Streaming from a file

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Streaming from a file

Vishwas Siravara
Hi guys,
Is it possible for flink to stream from a unix file system line by line, when I use readTextFile(path) - Reads text files, i.e. files that respect the TextInputFormat specification, line-by-line and returns them as Strings.
The entire contents of the file comes as a datastream, over which I can iterate to extract each line as a string. Is it possible to stream a line at a time ? Thanks for all the help so far. 


Thanks,
Vishwas 
Reply | Threaded
Open this post in threaded view
|

Re: Streaming from a file

Zhu Zhu
Hi Vishwas,

Not sure whether I understand your needs correctly.
I think currently readTextFile(path) does return a DataStream<String>. From the code it is emitting one line once it is read from the file, thus in a line-by-line streaming pattern.

Thanks,
Zhu Zhu

Vishwas Siravara <[hidden email]> 于2019年8月1日周四 下午11:50写道:
Hi guys,
Is it possible for flink to stream from a unix file system line by line, when I use readTextFile(path) - Reads text files, i.e. files that respect the TextInputFormat specification, line-by-line and returns them as Strings.
The entire contents of the file comes as a datastream, over which I can iterate to extract each line as a string. Is it possible to stream a line at a time ? Thanks for all the help so far. 


Thanks,
Vishwas