Read CSV file and and create customized field

[Soheil Pourbafrani]

Hi friends,

I'm going to read a CSV file that has 3 columns. I want the final loaded datatype to have other columns inferred by that 3 columns. For example, I would split the first column of the CSV file and create 3 new columns.

The problem is I did not find a straightforward approach for that. Here is what I have so far:

env.readCsvFile("pathToCsv")
        .fieldDelimiter(",")
        .ignoreFirstLine()
        .ignoreInvalidLines()
        .type(String.class, String.class, String.class)
        .print();

So is there any way to show the readCSVFile how to split CSV records or I should add an extra map function after loading CSV to create my desired schema?

Thanks

[Chesnay Schepler]

You should add an extra map function.

On 16/01/2020 17:10, Soheil Pourbafrani wrote:

Hi friends,

I'm going to read a CSV file that has 3 columns. I want the final loaded datatype to have other columns inferred by that 3 columns. For example, I would split the first column of the CSV file and create 3 new columns.

The problem is I did not find a straightforward approach for that. Here is what I have so far:

env.readCsvFile("pathToCsv")
        .fieldDelimiter(",")
        .ignoreFirstLine()
        .ignoreInvalidLines()
        .type(String.class, String.class, String.class)
        .print();

So is there any way to show the readCSVFile how to split CSV records or I should add an extra map function after loading CSV to create my desired schema?

Thanks