Table Api and CSV builder

[Karim Amer]

Hi there

I Have a CSV file with the timestamp deconstructed into 3 fields and I was wondering what is the best way to  specify the those 3 fields are the event  time ? Should I make extend  CsvTableSource and do the preprocessing or can CsvTableSource.builder() handle it. Or is there a better way in general to tackle this obstacle.

Thanks

[Stefan Richter]

Hi,

I think you just need to specify a custom watermark extractor that constructs the watermark from the 3 fields, as described here: https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/event_timestamp_extractors.html.

Best,

Stefan

Am 07.03.2018 um 00:52 schrieb Karim Amer <[hidden email]>:

Hi there

I Have a CSV file with the timestamp deconstructed into 3 fields and I was wondering what is the best way to  specify the those 3 fields are the event  time ? Should I make extend  CsvTableSource and do the preprocessing or can CsvTableSource.builder() handle it. Or is there a better way in general to tackle this obstacle.

Thanks

[Timo Walther]

Hi Karim,

the CsvTableSource and its builder are currently not able to specify event-time or processing-time. I'm sure this will change in the near future. Until then I would recommend to either extend it yourself or use the DataStream API first to do the parsing and watermarking and then convert it into a table.

This issue covers your case: https://issues.apache.org/jira/browse/FLINK-8536

Regards,
Timo



Am 3/7/18 um 4:23 PM schrieb Stefan Richter:

Hi,

I think you just need to specify a custom watermark extractor that constructs the watermark from the 3 fields, as described here: https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/event_timestamp_extractors.html.

Best,

Stefan

Am 07.03.2018 um 00:52 schrieb Karim Amer <[hidden email]>:

Hi there

I Have a CSV file with the timestamp deconstructed into 3 fields and I was wondering what is the best way to  specify the those 3 fields are the event  time ? Should I make extend  CsvTableSource and do the preprocessing or can CsvTableSource.builder() handle it. Or is there a better way in general to tackle this obstacle.

Thanks