writeAsFormattedText sets only Unix/Linux line endings

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

writeAsFormattedText sets only Unix/Linux line endings

Papadopoulos, Konstantinos

Hi all,

 

We are developing an application using Flink DataSet API focusing on generating a CSV file from a dataset of POJOs using writeAsFormattedText and a custom TextFormatter.

During the testing of our application, we observed that the files generated consist of Unix line endings (i.e., ‘\n’) even if the execution environment is Windows (i.e., expected line endings ‘\r\n’).

Since this is critical to our solution, is there any way to configure the line endings of the files generated?

 

Thanks in advance,

Konstantinos

Reply | Threaded
Open this post in threaded view
|

RE: writeAsFormattedText sets only Unix/Linux line endings

Papadopoulos, Konstantinos

Kind reminder

 

From: Papadopoulos, Konstantinos <[hidden email]>
Sent: Monday, May 06, 2019 5:31 PM
To: [hidden email]
Subject: writeAsFormattedText sets only Unix/Linux line endings

 

Hi all,

 

We are developing an application using Flink DataSet API focusing on generating a CSV file from a dataset of POJOs using writeAsFormattedText and a custom TextFormatter.

During the testing of our application, we observed that the files generated consist of Unix line endings (i.e., ‘\n’) even if the execution environment is Windows (i.e., expected line endings ‘\r\n’).

Since this is critical to our solution, is there any way to configure the line endings of the files generated?

 

Thanks in advance,

Konstantinos

Reply | Threaded
Open this post in threaded view
|

Re: writeAsFormattedText sets only Unix/Linux line endings

Chesnay Schepler
The line-ending is hard-coded into the TextOutputFormat. You will have to either extend this class and override #writeRecord(), or convert your POJOs to a Tuple and use the CsvOutputFormat, which supports setting the line delimiter (called recordDelimiter).

On 09/05/2019 08:32, Papadopoulos, Konstantinos wrote:

Kind reminder

 

From: Papadopoulos, Konstantinos [hidden email]
Sent: Monday, May 06, 2019 5:31 PM
To: [hidden email]
Subject: writeAsFormattedText sets only Unix/Linux line endings

 

Hi all,

 

We are developing an application using Flink DataSet API focusing on generating a CSV file from a dataset of POJOs using writeAsFormattedText and a custom TextFormatter.

During the testing of our application, we observed that the files generated consist of Unix line endings (i.e., ‘\n’) even if the execution environment is Windows (i.e., expected line endings ‘\r\n’).

Since this is critical to our solution, is there any way to configure the line endings of the files generated?

 

Thanks in advance,

Konstantinos