run python job with flink 1.7

[Joshua Fan]

Hi all

When I run the python example in flink 1.7, it always got a excepthon.

The command is: ./bin/pyflink.sh ./examples/python/streaming/word_count.py

The return message is:

2019-05-17 11:43:22,900 INFO  org.apache.hadoop.yarn.client.RMProxy                         - Connecting to ResourceManager at data01.hj.shbt.qihoo.net/10.203.82.17:8832

Starting execution of program

Traceback (most recent call last):

  File "/tmp/flink_plan_0a1aed4b-4155-4cfa-b2ba-8083a4a61f6e/plan.py", line 21, in <module>

    from org.apache.flink.api.common.functions import FlatMapFunction

ImportError: No module named org.apache.flink.api.common.functions

Failed to run plan: Plan file caused an error. Check log-files for details.

The program didn't contain a Flink job. Perhaps you forgot to call execute() on the execution environment.

Can not find any help in google.

Appreciate your help very much.

Sincerely

Joshua

[Joshua Fan]

when I look into the log file, it turned out that flink cannot get the plan when create the plan file, full log message is below.

107 2019-05-17 12:24:56.950 [main] ERROR org.apache.flink.python.api.PythonPlanBinder  - Failed to run plan.

108 java.lang.RuntimeException: Plan file caused an error. Check log-files for details.

109         at org.apache.flink.python.api.streaming.plan.PythonPlanStreamer.preparePlanMode(PythonPlanStreamer.java:107)

110         at org.apache.flink.python.api.PythonPlanBinder.runPlan(PythonPlanBinder.java:178)

111         at org.apache.flink.python.api.PythonPlanBinder.main(PythonPlanBinder.java:98)

112         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

113         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

114         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

115         at java.lang.reflect.Method.invoke(Method.java:498)

116         at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:529)

117         at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:421)

118         at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:427)

119         at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:813)

120         at org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:287)

121         at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:213)

122         at org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1050)

123         at org.apache.flink.client.cli.CliFrontend.lambda$main$11(CliFrontend.java:1126)

124         at java.security.AccessController.doPrivileged(Native Method)

125         at javax.security.auth.Subject.doAs(Subject.java:422)

126         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)

127         at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)

128         at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1126)

129 2019-05-17 12:24:56.951 [main] INFO  org.apache.flink.runtime.rest.RestClient  - Shutting down rest endpoint.

130 2019-05-17 12:24:56.959 [main] INFO  org.apache.flink.runtime.rest.RestClient  - Rest endpoint shutdown complete.

I am not familiar with python. Thanks for your help.

On Fri, May 17, 2019 at 11:47 AM Joshua Fan <[hidden email]> wrote:

Hi all

When I run the python example in flink 1.7, it always got a excepthon.

The command is: ./bin/pyflink.sh ./examples/python/streaming/word_count.py

The return message is:

2019-05-17 11:43:22,900 INFO  org.apache.hadoop.yarn.client.RMProxy                         - Connecting to ResourceManager at data01.hj.shbt.qihoo.net/10.203.82.17:8832

Starting execution of program

Traceback (most recent call last):

  File "/tmp/flink_plan_0a1aed4b-4155-4cfa-b2ba-8083a4a61f6e/plan.py", line 21, in <module>

    from org.apache.flink.api.common.functions import FlatMapFunction

ImportError: No module named org.apache.flink.api.common.functions

Failed to run plan: Plan file caused an error. Check log-files for details.

The program didn't contain a Flink job. Perhaps you forgot to call execute() on the execution environment.

Can not find any help in google.

Appreciate your help very much.

Sincerely

Joshua