How Flink read files from local filesystem

[Soheil Pourbafrani]

Hi,

Suppose we have a Flink single node cluster with multiple slots and some input files exist in local file system. In this case where we have no distributed file system to dedicate each file's block to taskmanagers, how Flink will read the file? Do all the task managers will open the file separately and read some dedicated portion of the file in parallel?

[Tillman Peng]

You can use env.readTextFile(path) which accepts path to a directory and
reads all files in that directory producing record for each line.

on 2020/1/15 17:58, Soheil Pourbafrani wrote:
> Suppose we have a Flink single node cluster with multiple slots and some
> input files exist in local file system. In this case where we have no
> distributed file system to dedicate each file's block to taskmanagers,
> how Flink will read the file? Do all the task managers will open the
> file separately and read some dedicated portion of the file in parallel?