(DEPRECATED) Apache Flink User Mailing List archive.

Cluster installation gives java.lang.NoClassDefFoundError for everything

Classic

List

Threaded

6 messages Options

Camelia Elena Ciolac

Cluster installation gives java.lang.NoClassDefFoundError for everything

Hello,

I am configuring Flink to run on a cluster with NFS.
I have the Flink 0.9.1 distribution in some path in NFS and I added that path in ~/.bashrc as FLINK_HOME, and also included the $FLINK_HOME/lib folder to $PATH.
I have the slaves file and the yaml file configured correctly with the nodes involved.

Still, when I run the following command I get the following errors:

$FLINK_HOME/bin/start-cluster.sh

(tried also
cd $FLINK_HOME
./bin/start-cluster.sh)

--> in /log/flink-theuser-jobmanager-thenode.out
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/flink/runtime/jobmanager/JobManager
Caused by: java.lang.ClassNotFoundException: org.apache.flink.runtime.jobmanager.JobManager
    at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
Could not find the main class: org.apache.flink.runtime.jobmanager.JobManager. Program will exit.

-->in /log/flink-theuser-taskmanager-theslavenode.out
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/flink/runtime/taskmanager/TaskManager
Caused by: java.lang.ClassNotFoundException: org.apache.flink.runtime.taskmanager.TaskManager
    at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
Could not find the main class: org.apache.flink.runtime.taskmanager.TaskManager. Program will exit.

And lastly, in my script I have the run of a job which gives:

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/flink/client/CliFrontend
Caused by: java.lang.ClassNotFoundException: org.apache.flink.client.CliFrontend
    at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
Could not find the main class: org.apache.flink.client.CliFrontend. Program will exit.

What could be the cause and which is the solution?

Looking forward to your answer as soon as possible.
Many thanks,
Camelia

Stephan Ewen

Re: Cluster installation gives java.lang.NoClassDefFoundError for everything

The distributed "start-cluster.sh" script works only, if the code is accessible under the same path on all machines, which must be the same path as on the machine where you invoke the script.

Otherwise the paths for remote shell commands will be wrong, and the classpaths will be wrong as a result. That means none of the Flink classes are in the JVM classpath.

I don't think any of these scripts evaluate a "FLINK_HOME" variable.

Can you try and make sure that Flink is on all machines under the same directory structure, and that you start it from there?

On Mon, Nov 9, 2015 at 2:32 PM, Camelia Elena Ciolac <[hidden email]> wrote:

Hello,

I am configuring Flink to run on a cluster with NFS.
I have the Flink 0.9.1 distribution in some path in NFS and I added that path in ~/.bashrc as FLINK_HOME, and also included the $FLINK_HOME/lib folder to $PATH.
I have the slaves file and the yaml file configured correctly with the nodes involved.

Still, when I run the following command I get the following errors:

$FLINK_HOME/bin/start-cluster.sh

(tried also
cd $FLINK_HOME
./bin/start-cluster.sh)

--> in /log/flink-theuser-jobmanager-thenode.out
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/flink/runtime/jobmanager/JobManager
Caused by: java.lang.ClassNotFoundException: org.apache.flink.runtime.jobmanager.JobManager
    at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
Could not find the main class: org.apache.flink.runtime.jobmanager.JobManager. Program will exit.

-->in /log/flink-theuser-taskmanager-theslavenode.out
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/flink/runtime/taskmanager/TaskManager
Caused by: java.lang.ClassNotFoundException: org.apache.flink.runtime.taskmanager.TaskManager
    at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
Could not find the main class: org.apache.flink.runtime.taskmanager.TaskManager. Program will exit.

And lastly, in my script I have the run of a job which gives:

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/flink/client/CliFrontend
Caused by: java.lang.ClassNotFoundException: org.apache.flink.client.CliFrontend
    at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
Could not find the main class: org.apache.flink.client.CliFrontend. Program will exit.

What could be the cause and which is the solution?

Looking forward to your answer as soon as possible.
Many thanks,
Camelia

Camelia Elena Ciolac

RE: Cluster installation gives java.lang.NoClassDefFoundError for everything

In reply to this post by Camelia Elena Ciolac

Good morning,

Thank you Stephan!

I keep on testing and in the meantime I'm wondering if the Java version on the cluster may be part of the issue:
java version "1.6.0_36"
OpenJDK Runtime Environment (IcedTea6 1.13.8) (rhel-1.13.8.1.el6_7-x86_64)
OpenJDK 64-Bit Server VM (build 23.25-b01, mixed mode)
?

The cluster has shared network file system, so no problem in this aspect.
I keep on trying and I've put some echo's for debugging in the Flink start scripts.

Best regards,
Camelia

From: Camelia Elena Ciolac
Sent: Monday, November 09, 2015 2:32 PM
To: [hidden email]
Subject: Cluster installation gives java.lang.NoClassDefFoundError for everything

Stephan Ewen

Re: Cluster installation gives java.lang.NoClassDefFoundError for everything

Hi!

Flink requires at least Java 1.7, so one of the reasons could also be that the classes are rejected for an incompatible version (class format 1.7, JVM does not understand it since it is only version 1.6).

That could explain things...

Greetings,

Stephan

On Wed, Nov 11, 2015 at 9:01 AM, Camelia Elena Ciolac <[hidden email]> wrote:

Good morning,

Thank you Stephan!

I keep on testing and in the meantime I'm wondering if the Java version on the cluster may be part of the issue:
java version "1.6.0_36"
OpenJDK Runtime Environment (IcedTea6 1.13.8) (rhel-1.13.8.1.el6_7-x86_64)
OpenJDK 64-Bit Server VM (build 23.25-b01, mixed mode)
?

The cluster has shared network file system, so no problem in this aspect.
I keep on trying and I've put some echo's for debugging in the Flink start scripts.

Best regards,
Camelia

From: Camelia Elena Ciolac
Sent: Monday, November 09, 2015 2:32 PM
To: [hidden email]
Subject: Cluster installation gives java.lang.NoClassDefFoundError for everything

Hello,

I am configuring Flink to run on a cluster with NFS.
I have the Flink 0.9.1 distribution in some path in NFS and I added that path in ~/.bashrc as FLINK_HOME, and also included the $FLINK_HOME/lib folder to $PATH.
I have the slaves file and the yaml file configured correctly with the nodes involved.

Still, when I run the following command I get the following errors:

$FLINK_HOME/bin/start-cluster.sh

(tried also
cd $FLINK_HOME
./bin/start-cluster.sh)

--> in /log/flink-theuser-jobmanager-thenode.out
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/flink/runtime/jobmanager/JobManager
Caused by: java.lang.ClassNotFoundException: org.apache.flink.runtime.jobmanager.JobManager
    at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
Could not find the main class: org.apache.flink.runtime.jobmanager.JobManager. Program will exit.

-->in /log/flink-theuser-taskmanager-theslavenode.out
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/flink/runtime/taskmanager/TaskManager
Caused by: java.lang.ClassNotFoundException: org.apache.flink.runtime.taskmanager.TaskManager
    at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
Could not find the main class: org.apache.flink.runtime.taskmanager.TaskManager. Program will exit.

And lastly, in my script I have the run of a job which gives:

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/flink/client/CliFrontend
Caused by: java.lang.ClassNotFoundException: org.apache.flink.client.CliFrontend
    at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
Could not find the main class: org.apache.flink.client.CliFrontend. Program will exit.

What could be the cause and which is the solution?

Looking forward to your answer as soon as possible.
Many thanks,
Camelia

Maximilian Michels

Re: Cluster installation gives java.lang.NoClassDefFoundError for everything

In reply to this post by Camelia Elena Ciolac

Hi Camelia,

Flink 0.9.X supports Java 6. So this can't be the issue.

Out of curiosity, I gave it a spin on a Linux machine with OpenJDK 6. I was able to start the command-line interface, job manager and task managers.

java version "1.6.0_36"
OpenJDK Runtime Environment (IcedTea6 1.13.8) (6b36-1.13.8-0ubuntu1~14.04)
OpenJDK 64-Bit Server VM (build 23.25-b01, mixed mode)

I think the error must be caused by your NFS setup. Let's start with ./bin/flink. For instance, can you access "/users/camelia/thecluster/flink-0.9.1/lib/flink-dist-0.9.1.jar" from the machine where you run ./bin/flink?

Best,

Max

On Wed, Nov 11, 2015 at 10:41 AM, Camelia Elena Ciolac <[hidden email]> wrote:

Hello,

As promised, I come back with debugging details.
So:

******* In start-cluster.sh , the following echo's

echo "start-cluster ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
echo $HOSTLIST
echo "start-cluster ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
echo $FLINK_BIN_DIR
echo "start-cluster ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"

# cluster mode, bring up job manager locally and a task manager on every slave host
"$FLINK_BIN_DIR"/jobmanager.sh start cluster batch

==> gave:

start-cluster ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/users/camelia/thecluster/flink-0.9.1/conf/slaves
start-cluster ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/users/camelia/thecluster/flink-0.9.1/bin
start-cluster ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

******* In jobmanager.sh , the following echo's

echo "Starting Job Manager"
        echo "jobmanager ~~~~~~~~~~~~~~~~~~~~~"
        pwd
        echo "jobmanager ~~~~~~~~~~~~~~~~~~~~~"
        echo ${FLINK_ENV_JAVA_OPTS}
        echo "jobmanager ~~~~~~~~~~~~~~~~~~~~~"
        echo $FLINK_JM_CLASSPATH
        echo "jobmanager ~~~~~~~~~~~~~~~~~~~~~"
        echo $INTERNAL_HADOOP_CLASSPATHS
        echo "jobmanager ~~~~~~~~~~~~~~~~~~~~"
        echo $FLINK_CONF_DIR
        echo "~~~~~~~~~~~~~~~~~~~~"
        $JAVA_RUN $JVM_ARGS ${FLINK_ENV_JAVA_OPTS} "${log_setting[@]}" -classpath "`manglePathList "$FLINK_JM_CLASSPATH:$INTERNAL_HADOOP_CLASSPATHS"`" org.apache.flink.runtime.jobmanager.JobManager --configDir "$FLINK_CONF_DIR" --executionMode $EXECUTIONMODE --streamingMode "$STREAMINGMODE" > "$out" 2>&1 < /dev/null &



==> gave:

jobmanager ~~~~~~~~~~~~~~~~~~~~~
/users/camelia/thecluster
jobmanager ~~~~~~~~~~~~~~~~~~~~~

jobmanager ~~~~~~~~~~~~~~~~~~~~~
/users/camelia/thecluster/flink-0.9.1/lib/flink-dist-0.9.1.jar:/users/camelia/thecluster/flink-0.9.1/lib/flink-python-0.9.1.jar
jobmanager ~~~~~~~~~~~~~~~~~~~~~
::
jobmanager ~~~~~~~~~~~~~~~~~~~~
/users/camelia/thecluster/flink-0.9.1/conf
~~~~~~~~~~~~~~~~~~~~

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/flink/runtime/jobmanager/JobManager
Caused by: java.lang.ClassNotFoundException: org.apache.flink.runtime.jobmanager.JobManager
    at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
Could not find the main class: org.apache.flink.runtime.jobmanager.JobManager. Program will exit.

******* In taskmanager.sh, the following echo's

echo Starting task manager on host $HOSTNAME
        echo "taskmanager ~~~~~~~~~~~~~~~~~~~~~"
        pwd
        echo "taskmanager ~~~~~~~~~~~~~~~~~~~~~"
        echo ${FLINK_ENV_JAVA_OPTS}
        echo "taskmanager ~~~~~~~~~~~~~~~~~~~~~"
        echo $FLINK_JM_CLASSPATH
        echo "taskmanager ~~~~~~~~~~~~~~~~~~~~~"
        echo $INTERNAL_HADOOP_CLASSPATHS
        echo "taskmanager ~~~~~~~~~~~~~~~~~~~~"
        echo $FLINK_CONF_DIR
        echo "~~~~~~~~~~~~~~~~~~~~"
        $JAVA_RUN $JVM_ARGS ${FLINK_ENV_JAVA_OPTS} "${log_setting[@]}" -classpath "`manglePathList "$FLINK_TM_CLASSPATH:$INTERNAL_HADOOP_CLASSPATHS"`" org.apache.flink.runtime.taskmanager.TaskManager --configDir "$FLINK_CONF_DIR" --streamingMode "$STREAMINGMODE" > "$out" 2>&1 < /dev/null &

==> gave:

taskmanager ~~~~~~~~~~~~~~~~~~~~~
/users/camelia/thecluster
taskmanager ~~~~~~~~~~~~~~~~~~~~~

taskmanager ~~~~~~~~~~~~~~~~~~~~~

taskmanager ~~~~~~~~~~~~~~~~~~~~~
::
taskmanager ~~~~~~~~~~~~~~~~~~~~
/users/camelia/thecluster/flink-0.9.1/conf
~~~~~~~~~~~~~~~~~~~~

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/flink/runtime/taskmanager/TaskManager
Caused by: java.lang.ClassNotFoundException: org.apache.flink.runtime.taskmanager.TaskManager
    at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
Could not find the main class: org.apache.flink.runtime.taskmanager.TaskManager. Program will exit.

******* In flink , the following echo's

echo "flink ~~~~~~~~~~~~~~~~~~~~~"
pwd
echo "flink ~~~~~~~~~~~~~~~~~~~~~"
echo $CC_CLASSPATH
echo "flink ~~~~~~~~~~~~~~~~~~~~~"
echo $INTERNAL_HADOOP_CLASSPATHS
echo "~~~~~~~~~~~~~~~~~~~~"
$JAVA_RUN $JVM_ARGS "${log_setting[@]}" -classpath "`manglePathList "$CC_CLASSPATH:$INTERNAL_HADOOP_CLASSPATHS"`" org.apache.flink.client.CliFrontend "$@"

==> gave:

flink ~~~~~~~~~~~~~~~~~~~~~
/users/camelia/thecluster
flink ~~~~~~~~~~~~~~~~~~~~~
/users/camelia/thecluster/flink-0.9.1/lib/flink-dist-0.9.1.jar:/users/camelia/thecluster/flink-0.9.1/lib/flink-python-0.9.1.jar
flink ~~~~~~~~~~~~~~~~~~~~~
::
~~~~~~~~~~~~~~~~~~~~
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/flink/client/CliFrontend
Caused by: java.lang.ClassNotFoundException: org.apache.flink.client.CliFrontend
    at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
Could not find the main class: org.apache.flink.client.CliFrontend. Program will exit.

I hope that now you have a better image of the issue and I hope you can provide some guidance to solve it.
I mention that I use the simplest scenario possible, just 2 nodes (1 master and 1 slave) without Hadoop for this test.
Also, I have a hostname command inside the script that shows me that I'm on the master node when running these scripts.

Thank you very much!
Camelia

From: Camelia Elena Ciolac
Sent: Wednesday, November 11, 2015 9:01 AM
To: [hidden email]
Subject: RE: Cluster installation gives java.lang.NoClassDefFoundError for everything

Good morning,

Thank you Stephan!

I keep on testing and in the meantime I'm wondering if the Java version on the cluster may be part of the issue:
java version "1.6.0_36"
OpenJDK Runtime Environment (IcedTea6 1.13.8) (rhel-1.13.8.1.el6_7-x86_64)
OpenJDK 64-Bit Server VM (build 23.25-b01, mixed mode)
?

The cluster has shared network file system, so no problem in this aspect.
I keep on trying and I've put some echo's for debugging in the Flink start scripts.

Best regards,
Camelia

From: Camelia Elena Ciolac
Sent: Monday, November 09, 2015 2:32 PM
To: [hidden email]
Subject: Cluster installation gives java.lang.NoClassDefFoundError for everything

Hello,

I am configuring Flink to run on a cluster with NFS.
I have the Flink 0.9.1 distribution in some path in NFS and I added that path in ~/.bashrc as FLINK_HOME, and also included the $FLINK_HOME/lib folder to $PATH.
I have the slaves file and the yaml file configured correctly with the nodes involved.

Still, when I run the following command I get the following errors:

$FLINK_HOME/bin/start-cluster.sh

(tried also
cd $FLINK_HOME
./bin/start-cluster.sh)

--> in /log/flink-theuser-jobmanager-thenode.out
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/flink/runtime/jobmanager/JobManager
Caused by: java.lang.ClassNotFoundException: org.apache.flink.runtime.jobmanager.JobManager
    at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
Could not find the main class: org.apache.flink.runtime.jobmanager.JobManager. Program will exit.

-->in /log/flink-theuser-taskmanager-theslavenode.out
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/flink/runtime/taskmanager/TaskManager
Caused by: java.lang.ClassNotFoundException: org.apache.flink.runtime.taskmanager.TaskManager
    at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
Could not find the main class: org.apache.flink.runtime.taskmanager.TaskManager. Program will exit.

And lastly, in my script I have the run of a job which gives:

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/flink/client/CliFrontend
Caused by: java.lang.ClassNotFoundException: org.apache.flink.client.CliFrontend
    at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
Could not find the main class: org.apache.flink.client.CliFrontend. Program will exit.

What could be the cause and which is the solution?

Looking forward to your answer as soon as possible.
Many thanks,
Camelia

rmetzger0

Re: Cluster installation gives java.lang.NoClassDefFoundError for everything

Is "jar tf /users/camelia/thecluster/flink-0.9.1/lib/flink-dist-0.9.1.jar" listing for example the "org/apache/flink/client/CliFrontend" class?

On Wed, Nov 11, 2015 at 12:09 PM, Maximilian Michels <[hidden email]> wrote:

Hi Camelia,

Flink 0.9.X supports Java 6. So this can't be the issue.

Out of curiosity, I gave it a spin on a Linux machine with OpenJDK 6. I was able to start the command-line interface, job manager and task managers.

java version "1.6.0_36"
OpenJDK Runtime Environment (IcedTea6 1.13.8) (6b36-1.13.8-0ubuntu1~14.04)
OpenJDK 64-Bit Server VM (build 23.25-b01, mixed mode)

I think the error must be caused by your NFS setup. Let's start with ./bin/flink. For instance, can you access "/users/camelia/thecluster/flink-0.9.1/lib/flink-dist-0.9.1.jar" from the machine where you run ./bin/flink?

Best,
Max

On Wed, Nov 11, 2015 at 10:41 AM, Camelia Elena Ciolac <[hidden email]> wrote:

Hello,

As promised, I come back with debugging details.
So:

******* In start-cluster.sh , the following echo's

echo "start-cluster ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
echo $HOSTLIST
echo "start-cluster ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
echo $FLINK_BIN_DIR
echo "start-cluster ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"

# cluster mode, bring up job manager locally and a task manager on every slave host
"$FLINK_BIN_DIR"/jobmanager.sh start cluster batch

==> gave:

start-cluster ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/users/camelia/thecluster/flink-0.9.1/conf/slaves
start-cluster ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/users/camelia/thecluster/flink-0.9.1/bin
start-cluster ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

******* In jobmanager.sh , the following echo's

echo "Starting Job Manager"
        echo "jobmanager ~~~~~~~~~~~~~~~~~~~~~"
        pwd
        echo "jobmanager ~~~~~~~~~~~~~~~~~~~~~"
        echo ${FLINK_ENV_JAVA_OPTS}
        echo "jobmanager ~~~~~~~~~~~~~~~~~~~~~"
        echo $FLINK_JM_CLASSPATH
        echo "jobmanager ~~~~~~~~~~~~~~~~~~~~~"
        echo $INTERNAL_HADOOP_CLASSPATHS
        echo "jobmanager ~~~~~~~~~~~~~~~~~~~~"
        echo $FLINK_CONF_DIR
        echo "~~~~~~~~~~~~~~~~~~~~"
        $JAVA_RUN $JVM_ARGS ${FLINK_ENV_JAVA_OPTS} "${log_setting[@]}" -classpath "`manglePathList "$FLINK_JM_CLASSPATH:$INTERNAL_HADOOP_CLASSPATHS"`" org.apache.flink.runtime.jobmanager.JobManager --configDir "$FLINK_CONF_DIR" --executionMode $EXECUTIONMODE --streamingMode "$STREAMINGMODE" > "$out" 2>&1 < /dev/null &



==> gave:

jobmanager ~~~~~~~~~~~~~~~~~~~~~
/users/camelia/thecluster
jobmanager ~~~~~~~~~~~~~~~~~~~~~

jobmanager ~~~~~~~~~~~~~~~~~~~~~
/users/camelia/thecluster/flink-0.9.1/lib/flink-dist-0.9.1.jar:/users/camelia/thecluster/flink-0.9.1/lib/flink-python-0.9.1.jar
jobmanager ~~~~~~~~~~~~~~~~~~~~~
::
jobmanager ~~~~~~~~~~~~~~~~~~~~
/users/camelia/thecluster/flink-0.9.1/conf
~~~~~~~~~~~~~~~~~~~~

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/flink/runtime/jobmanager/JobManager
Caused by: java.lang.ClassNotFoundException: org.apache.flink.runtime.jobmanager.JobManager
    at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
Could not find the main class: org.apache.flink.runtime.jobmanager.JobManager. Program will exit.

******* In taskmanager.sh, the following echo's

echo Starting task manager on host $HOSTNAME
        echo "taskmanager ~~~~~~~~~~~~~~~~~~~~~"
        pwd
        echo "taskmanager ~~~~~~~~~~~~~~~~~~~~~"
        echo ${FLINK_ENV_JAVA_OPTS}
        echo "taskmanager ~~~~~~~~~~~~~~~~~~~~~"
        echo $FLINK_JM_CLASSPATH
        echo "taskmanager ~~~~~~~~~~~~~~~~~~~~~"
        echo $INTERNAL_HADOOP_CLASSPATHS
        echo "taskmanager ~~~~~~~~~~~~~~~~~~~~"
        echo $FLINK_CONF_DIR
        echo "~~~~~~~~~~~~~~~~~~~~"
        $JAVA_RUN $JVM_ARGS ${FLINK_ENV_JAVA_OPTS} "${log_setting[@]}" -classpath "`manglePathList "$FLINK_TM_CLASSPATH:$INTERNAL_HADOOP_CLASSPATHS"`" org.apache.flink.runtime.taskmanager.TaskManager --configDir "$FLINK_CONF_DIR" --streamingMode "$STREAMINGMODE" > "$out" 2>&1 < /dev/null &

==> gave:

taskmanager ~~~~~~~~~~~~~~~~~~~~~
/users/camelia/thecluster
taskmanager ~~~~~~~~~~~~~~~~~~~~~

taskmanager ~~~~~~~~~~~~~~~~~~~~~

taskmanager ~~~~~~~~~~~~~~~~~~~~~
::
taskmanager ~~~~~~~~~~~~~~~~~~~~
/users/camelia/thecluster/flink-0.9.1/conf
~~~~~~~~~~~~~~~~~~~~

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/flink/runtime/taskmanager/TaskManager
Caused by: java.lang.ClassNotFoundException: org.apache.flink.runtime.taskmanager.TaskManager
    at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
Could not find the main class: org.apache.flink.runtime.taskmanager.TaskManager. Program will exit.

******* In flink , the following echo's

echo "flink ~~~~~~~~~~~~~~~~~~~~~"
pwd
echo "flink ~~~~~~~~~~~~~~~~~~~~~"
echo $CC_CLASSPATH
echo "flink ~~~~~~~~~~~~~~~~~~~~~"
echo $INTERNAL_HADOOP_CLASSPATHS
echo "~~~~~~~~~~~~~~~~~~~~"
$JAVA_RUN $JVM_ARGS "${log_setting[@]}" -classpath "`manglePathList "$CC_CLASSPATH:$INTERNAL_HADOOP_CLASSPATHS"`" org.apache.flink.client.CliFrontend "$@"

==> gave:

flink ~~~~~~~~~~~~~~~~~~~~~
/users/camelia/thecluster
flink ~~~~~~~~~~~~~~~~~~~~~
/users/camelia/thecluster/flink-0.9.1/lib/flink-dist-0.9.1.jar:/users/camelia/thecluster/flink-0.9.1/lib/flink-python-0.9.1.jar
flink ~~~~~~~~~~~~~~~~~~~~~
::
~~~~~~~~~~~~~~~~~~~~
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/flink/client/CliFrontend
Caused by: java.lang.ClassNotFoundException: org.apache.flink.client.CliFrontend
    at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
Could not find the main class: org.apache.flink.client.CliFrontend. Program will exit.

I hope that now you have a better image of the issue and I hope you can provide some guidance to solve it.
I mention that I use the simplest scenario possible, just 2 nodes (1 master and 1 slave) without Hadoop for this test.
Also, I have a hostname command inside the script that shows me that I'm on the master node when running these scripts.

Thank you very much!
Camelia

From: Camelia Elena Ciolac
Sent: Wednesday, November 11, 2015 9:01 AM
To: [hidden email]
Subject: RE: Cluster installation gives java.lang.NoClassDefFoundError for everything

Good morning,

Thank you Stephan!

I keep on testing and in the meantime I'm wondering if the Java version on the cluster may be part of the issue:
java version "1.6.0_36"
OpenJDK Runtime Environment (IcedTea6 1.13.8) (rhel-1.13.8.1.el6_7-x86_64)
OpenJDK 64-Bit Server VM (build 23.25-b01, mixed mode)
?

The cluster has shared network file system, so no problem in this aspect.
I keep on trying and I've put some echo's for debugging in the Flink start scripts.

Best regards,
Camelia

From: Camelia Elena Ciolac
Sent: Monday, November 09, 2015 2:32 PM
To: [hidden email]
Subject: Cluster installation gives java.lang.NoClassDefFoundError for everything

Hello,

I am configuring Flink to run on a cluster with NFS.
I have the Flink 0.9.1 distribution in some path in NFS and I added that path in ~/.bashrc as FLINK_HOME, and also included the $FLINK_HOME/lib folder to $PATH.
I have the slaves file and the yaml file configured correctly with the nodes involved.

Still, when I run the following command I get the following errors:

$FLINK_HOME/bin/start-cluster.sh

(tried also
cd $FLINK_HOME
./bin/start-cluster.sh)

--> in /log/flink-theuser-jobmanager-thenode.out
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/flink/runtime/jobmanager/JobManager
Caused by: java.lang.ClassNotFoundException: org.apache.flink.runtime.jobmanager.JobManager
    at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
Could not find the main class: org.apache.flink.runtime.jobmanager.JobManager. Program will exit.

-->in /log/flink-theuser-taskmanager-theslavenode.out
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/flink/runtime/taskmanager/TaskManager
Caused by: java.lang.ClassNotFoundException: org.apache.flink.runtime.taskmanager.TaskManager
    at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
Could not find the main class: org.apache.flink.runtime.taskmanager.TaskManager. Program will exit.

And lastly, in my script I have the run of a job which gives:

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/flink/client/CliFrontend
Caused by: java.lang.ClassNotFoundException: org.apache.flink.client.CliFrontend
    at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
Could not find the main class: org.apache.flink.client.CliFrontend. Program will exit.

What could be the cause and which is the solution?

Looking forward to your answer as soon as possible.
Many thanks,
Camelia