Cannot get Python API to work at all; java.io.FileNotFoundException for Flink python resources

Posted by Ronkainen Jussi on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Cannot-get-Python-API-to-work-at-all-java-io-FileNotFoundException-for-Flink-python-resources-tp3144.html

Hi,

 

I’m trying to evaluate the Python API but get a java.io.FileNotFoundException when trying to run any Python script on pyflink. The contents of the Python script are irrelevant, doesn’t even need to be valid Python.

I made a local install of 0.9.1, copy-pasted the example Python script from https://ci.apache.org/projects/flink/flink-docs-release-0.9/apis/python.html to a file “~/python_src/wordcount.py” and tried to run it in Flink root directory (/home/flink/flink-0.9.1/):

 

./bin/pyflink2.sh ~/python_src/wordcount.py

 

I get a java.io.FileNotFoundException (trace below). The cause could be related to PythonPlanBinder line 93, which strips 7 characters from FLINK_DIR and after that appends /resources/python to the result, ending in an invalid path. If I understand correctly, for my  user “flink” the path should be /home/flink/flink-0.9.1/resources/python

 

Am I missing some setup option, using pyflink wrong, or could this be a bug? I also tried compiling from 0.10-SNAPSHOT but the problem persists.

I’m running Ubuntu 14.04  with openjdk 8 and Python 2.7.6.

 

Exception trace:

 

org.apache.flink.client.program.ProgramInvocationException: The main method caused an error.

        at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:452)

        at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:353)

        at org.apache.flink.client.program.Client.run(Client.java:315)

        at org.apache.flink.client.CliFrontend.executeProgram(CliFrontend.java:582)

        at org.apache.flink.client.CliFrontend.run(CliFrontend.java:288)

        at org.apache.flink.client.CliFrontend.parseParameters(CliFrontend.java:878)

        at org.apache.flink.client.CliFrontend.main(CliFrontend.java:920)

Caused by: java.io.FileNotFoundException: File /home/flink/flin/resources/python does not exist or the user running Flink ('flink') has insufficient permissions to access it.

        at org.apache.flink.core.fs.local.LocalFileSystem.getFileStatus(LocalFileSystem.java:107)

        at org.apache.flink.runtime.filecache.FileCache.copy(FileCache.java:242)

        at org.apache.flink.languagebinding.api.java.python.PythonPlanBinder.prepareFiles(PythonPlanBinder.java:138)

        at org.apache.flink.languagebinding.api.java.python.PythonPlanBinder.runPlan(PythonPlanBinder.java:108)

        at org.apache.flink.languagebinding.api.java.python.PythonPlanBinder.main(PythonPlanBinder.java:84)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:497)

        at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:437)

        ... 6 more

 

-        Jussi