k8s FileNotFoundException

[Kaepke, Marc]

Hi everyone,

I always get an FileNotFoundException by following the kubernetes setup guide [1].

I moved my jar and my input file onto the job manager pod 

After that I join the job manager pod by using: kubectl exec -it <jobmanager> - - /bin/bash

With ls I can see both files. WordCount-example worked well.

Caused by: java.io.IOException: Error opening the Input Split file:/opt/flink/wiki-vote.txt [0,1095061]: /opt/flink/wiki-vote.txt (No such file or directory)

at org.apache.flink.api.common.io.FileInputFormat.open(FileInputFormat.java:706)

at org.apache.flink.api.common.io.DelimitedInputFormat.open(DelimitedInputFormat.java:477)

at org.apache.flink.api.common.io.GenericCsvInputFormat.open(GenericCsvInputFormat.java:301)

at org.apache.flink.api.java.io.CsvInputFormat.open(CsvInputFormat.java:48)

at org.apache.flink.api.java.io.CsvInputFormat.open(CsvInputFormat.java:31)

at org.apache.flink.runtime.operators.DataSourceTask.invoke(DataSourceTask.java:145)

at org.apache.flink.runtime.taskmanager.Task.run(Task.java:702)

at java.lang.Thread.run(Thread.java:748)

Caused by: java.io.FileNotFoundException: /opt/flink/wiki-vote.txt (No such file or directory)

at java.io.FileInputStream.open0(Native Method)

at java.io.FileInputStream.open(FileInputStream.java:195)

at java.io.FileInputStream.<init>(FileInputStream.java:138)

at org.apache.flink.core.fs.local.LocalDataInputStream.<init>(LocalDataInputStream.java:49)

at org.apache.flink.core.fs.local.LocalFileSystem.open(LocalFileSystem.java:141)

at org.apache.flink.api.common.io.FileInputFormat$InputSplitOpenThread.run(FileInputFormat.java:866)

root@flink-jobmanager-2891032829-pqgwq:/opt/flink# 

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.3/setup/kubernetes.html

Any idea?!

Marc

[Aljoscha Krettek]

Is the file also available on the TaskManager containers? These are doing the actual reading, in fact.

In general, I would suggest to use a distributed file system as input for a Flink Job if you want to process files.

Best,

Aljoscha

On 4. Aug 2017, at 14:13, Kaepke, Marc <[hidden email]> wrote:

Hi everyone,

I always get an FileNotFoundException by following the kubernetes setup guide [1].

I moved my jar and my input file onto the job manager pod 

After that I join the job manager pod by using: kubectl exec -it <jobmanager> - - /bin/bash

With ls I can see both files. WordCount-example worked well.

Caused by: java.io.IOException: Error opening the Input Split file:/opt/flink/wiki-vote.txt [0,1095061]: /opt/flink/wiki-vote.txt (No such file or directory)

at org.apache.flink.api.common.io.FileInputFormat.open(FileInputFormat.java:706)

at org.apache.flink.api.common.io.DelimitedInputFormat.open(DelimitedInputFormat.java:477)

at org.apache.flink.api.common.io.GenericCsvInputFormat.open(GenericCsvInputFormat.java:301)

at org.apache.flink.api.java.io.CsvInputFormat.open(CsvInputFormat.java:48)

at org.apache.flink.api.java.io.CsvInputFormat.open(CsvInputFormat.java:31)

at org.apache.flink.runtime.operators.DataSourceTask.invoke(DataSourceTask.java:145)

at org.apache.flink.runtime.taskmanager.Task.run(Task.java:702)

at java.lang.Thread.run(Thread.java:748)

Caused by: java.io.FileNotFoundException: /opt/flink/wiki-vote.txt (No such file or directory)

at java.io.FileInputStream.open0(Native Method)

at java.io.FileInputStream.open(FileInputStream.java:195)

at java.io.FileInputStream.<init>(FileInputStream.java:138)

at org.apache.flink.core.fs.local.LocalDataInputStream.<init>(LocalDataInputStream.java:49)

at org.apache.flink.core.fs.local.LocalFileSystem.open(LocalFileSystem.java:141)

at org.apache.flink.api.common.io.FileInputFormat$InputSplitOpenThread.run(FileInputFormat.java:866)

root@flink-jobmanager-2891032829-pqgwq:/opt/flink# 

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.3/setup/kubernetes.html

Any idea?!

Marc