Re: Discarding header from CSV file

Posted by nsengupta on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Discarding-header-from-CSV-file-tp6474p6476.html

Hello Chiwan,

I was just about to post to declare my ignorance, because I searched again and realized that I failed to spot ReadCsvFile ! :-) You have been faster than me!

Yes, I should use ReadCsvFile so that I get all the facilities built in. 

Many thanks for pointing out.

-- N

Nirmalya Sengupta about.me/sengupta.nirmalya

On Wed, Apr 27, 2016 at 7:19 AM, Chiwan Park-2 [via Apache Flink User Mailing List archive.] <[hidden email]> wrote:

Hi, Nirmalya

I recommend readCsvFile() method rather than readTextFile() to read CSV file. readCsvFile() provides some features for CSV file such as ignoreFirstLine() (what you are looking for), ignoreComments(), and etc.

If you have to use readTextFile() method, I think, you can ignore column headers by calling zipWithIndex method and filtering it based on the index.

Regards,
Chiwan Park

> On Apr 27, 2016, at 10:32 AM, nsengupta <[hidden email]> wrote:
>
> What is the recommended way of discarding the Column Header(s) from a CSV
> file, if I am using
>
> /environment.readTextFile(....)
> /
> facility? Obviously, we don't know beforehand, which of the nodes will read
> the Header(s)? So, we cannot use usual tricks like drop(1)?
>
> I don't recall well: has this been discussed and closed earlier in this
> forum? If so, can someone point that out to me please?
>
> -- Nirmalya
>
>
>
> --
> View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Discarding-header-from-CSV-file-tp6474.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.

---

If you reply to this email, your message will be added to the discussion below:

http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Discarding-header-from-CSV-file-tp6474p6475.html

To unsubscribe from Discarding header from CSV file, click here.
NAML

--