Debugging Python-Api fails with NoClassDefFoundError

[Mathias Peters]

Hi,

I just wanted to debug a custom python script using your python dataset
api. Running the PythonPlanBinder in Intellij IDEA gives me the
subjected error. I took a fresh clone, built it with mvn clean install
-DskipTest, and imported everything in idea. Using an older version this
worked fine, so assume no(t the usual noob) errors on my side 

Submitting a python script via console works.

The full stack trace looks like this:

Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/flink/api/java/typeutils/TupleTypeInfoBase
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:264)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:122)
Caused by: java.lang.ClassNotFoundException:
org.apache.flink.api.java.typeutils.TupleTypeInfoBase
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 3 more


Thanks for the help.

best

Mathias

[Ted Yu]

This class is in flink-core jar.

Have you verified that the jar is on classpath ?

Cheers

On Wed, Jan 4, 2017 at 12:16 PM, Mathias Peters <[hidden email]> wrote:

Hi,

I just wanted to debug a custom python script using your python dataset
api. Running the PythonPlanBinder in Intellij IDEA gives me the
subjected error. I took a fresh clone, built it with mvn clean install
-DskipTest, and imported everything in idea. Using an older version this
worked fine, so assume no(t the usual noob) errors on my side 

Submitting a python script via console works.

The full stack trace looks like this:

Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/flink/api/java/typeutils/TupleTypeInfoBase
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:264)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:122)
Caused by: java.lang.ClassNotFoundException:
org.apache.flink.api.java.typeutils.TupleTypeInfoBase
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 3 more


Thanks for the help.

best

Mathias

[Mathias Peters]

Yes, it is. Also, the project import in Idea has worked so far.

Cheers

On 04.01.2017 21:52, Ted Yu wrote:

This class is in flink-core jar.

Have you verified that the jar is on classpath ?

Cheers

On Wed, Jan 4, 2017 at 12:16 PM, Mathias Peters <[hidden email]> wrote:

Hi,

I just wanted to debug a custom python script using your python dataset
api. Running the PythonPlanBinder in Intellij IDEA gives me the
subjected error. I took a fresh clone, built it with mvn clean install
-DskipTest, and imported everything in idea. Using an older version this
worked fine, so assume no(t the usual noob) errors on my side 

Submitting a python script via console works.

The full stack trace looks like this:

Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/flink/api/java/typeutils/TupleTypeInfoBase
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:264)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:122)
Caused by: java.lang.ClassNotFoundException:
org.apache.flink.api.java.typeutils.TupleTypeInfoBase
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 3 more


Thanks for the help.

best

Mathias

[Chesnay Schepler]

Hello,

all Flink dependencies of the Python APi are marked as provided in the pom.xml similar to most connectors.
By removing the provided tags in the pom.xml you should be able to run the PythonPlanBinder from the IDE.

This was done to exclude these dependencies in the flink-python jar; since we copy this into the /lib folder
of flink-dist we would otherwise include some parts of flink twice.

Maybe we could use the shade-plugin to exclude all flink-dependencies in the jar, then we could remove the
provided tags and it should work from the IDE.

Feel free to open a JIRA for this.

Regards,
Chesnay


On 05.01.2017 08:25, Mathias Peters wrote:

Yes, it is. Also, the project import in Idea has worked so far.

Cheers

On 04.01.2017 21:52, Ted Yu wrote:

This class is in flink-core jar.

Have you verified that the jar is on classpath ?

Cheers

On Wed, Jan 4, 2017 at 12:16 PM, Mathias Peters <[hidden email]> wrote:

Hi,

I just wanted to debug a custom python script using your python dataset
api. Running the PythonPlanBinder in Intellij IDEA gives me the
subjected error. I took a fresh clone, built it with mvn clean install
-DskipTest, and imported everything in idea. Using an older version this
worked fine, so assume no(t the usual noob) errors on my side 

Submitting a python script via console works.

The full stack trace looks like this:

Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/flink/api/java/typeutils/TupleTypeInfoBase
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:264)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:122)
Caused by: java.lang.ClassNotFoundException:
org.apache.flink.api.java.typeutils.TupleTypeInfoBase
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 3 more


Thanks for the help.

best

Mathias