yarnship option

[Mikhail Pryakhin]

Hi all!

I'm playing with flink streaming job on yarn cluster. The job consumes events from kafka and prints them to the standard out.
The job uses flink-connector-kafka-0.9_2.11-1.2.1.jar library that is passed via the --yarnship option.
Here is the way I run the job:

export HADOOP_USER_NAME=hdfs; \
export HADOOP_CONF_DIR=/etc/hadoop/conf/; \
/opt/flink-1.2.1/bin/flink run \
        -yst \
        -yt /home/user/job-libs  \
        -m yarn-cluster \
        -yn 3 \
        -c com.flink.Test \
        flink-test_2.11-1.0.0-SNAPSHOT.jar

Finally the job fails complaing that it can't find the class:
java.lang.NoClassDefFoundError: org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer09

I looked through the jobmanager.log and found that the flink-connector-kafka-0.9_2.11-1.2.1.jar library is added to the classpath:

2017-05-22 17:41:59,637 INFO  org.apache.flink.yarn.YarnApplicationMasterRunner             -  Classpath: job-libs/flink-connector-kafka-0.9_2.11-1.2.1.jar:<another jars...>

Could please help to figure out why the class org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer09 can't be loaded inspite of the jar containing this class was added to the classpath

----
Mike Pryakhin

[rmetzger0]

Hi,

this issue is unexpected :) Can you double check if the job-libs/flink-connector-kafka-0.9_2.11-1.2.1.jar contains org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer09 ?

Also, were there any previous Kafka09 related exceptions in the log??

From this SO answer, it seems that this is not really the classical classNotFoundException, but a bit differenT: https://stackoverflow.com/a/5756989/568695

On Mon, May 22, 2017 at 5:12 PM, Mikhail Pryakhin <[hidden email]> wrote:

Hi all!

I'm playing with flink streaming job on yarn cluster. The job consumes events from kafka and prints them to the standard out.
The job uses flink-connector-kafka-0.9_2.11-1.2.1.jar library that is passed via the --yarnship option.
Here is the way I run the job:

export HADOOP_USER_NAME=hdfs; \
export HADOOP_CONF_DIR=/etc/hadoop/conf/; \
/opt/flink-1.2.1/bin/flink run \
        -yst \
        -yt /home/user/job-libs  \
        -m yarn-cluster \
        -yn 3 \
        -c com.flink.Test \
        flink-test_2.11-1.0.0-SNAPSHOT.jar

Finally the job fails complaing that it can't find the class:
java.lang.NoClassDefFoundError: org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer09

I looked through the jobmanager.log and found that the flink-connector-kafka-0.9_2.11-1.2.1.jar library is added to the classpath:

2017-05-22 17:41:59,637 INFO  org.apache.flink.yarn.YarnApplicationMasterRunner             -  Classpath: job-libs/flink-connector-kafka-0.9_2.11-1.2.1.jar:<another jars...>

Could please help to figure out why the class org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer09 can't be loaded inspite of the jar containing this class was added to the classpath

----
Mike Pryakhin

[Mikhail Pryakhin]

Hi Robert!

Thanks a lot for your reply!

>Can you double check if the job-libs/flink-connector-kafka-0.9_2.11-1.2.1.jar contains org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer09 ?

The jar does contain the class org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer09.class (The contents of the jar file is listed below)

> Also, were there any previous Kafka09 related exceptions in the log??    

No, it was the very first exception… You can find log file attached. 

the flink-connector-kafka-0.9_2.11-1.2.1.jar contains the following:

root@host:~# unzip -l job-libs/flink-connector-kafka-0.9_2.11-1.2.1.jar

Archive:  job-libs/flink-connector-kafka-0.9_2.11-1.2.1.jar

  Length      Date    Time    Name

---------  ---------- -----   ----

        0  2017-04-11 01:59   META-INF/

      443  2017-04-11 01:59   META-INF/MANIFEST.MF

     1451  2017-04-11 01:59   META-INF/DEPENDENCIES

    11358  2017-04-11 01:59   META-INF/LICENSE

      182  2017-04-11 01:59   META-INF/NOTICE

        0  2017-04-11 01:59   org/

        0  2017-04-11 01:59   org/apache/

        0  2017-04-11 01:59   org/apache/flink/

        0  2017-04-11 01:59   org/apache/flink/streaming/

        0  2017-04-11 01:59   org/apache/flink/streaming/connectors/

        0  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/

    11554  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer09.class

     2603  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/Kafka09JsonTableSink.class

     4463  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/FlinkKafkaProducer09.class

     2693  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/Kafka09TableSource.class

        0  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/internal/

     1125  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/internal/KafkaConsumerCallBridge.class

      570  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/internal/Handover$WakeupException.class

    10268  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/internal/KafkaConsumerThread.class

      570  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/internal/Handover$ClosedException.class

     3018  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/internal/Handover.class

    11854  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/internal/Kafka09Fetcher.class

     2182  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/internal/KafkaConsumerThread$CommitCallback.class

      311  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/internal/KafkaConsumerThread$1.class

     2329  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/Kafka09JsonTableSource.class

        0  2017-04-11 01:59   META-INF/maven/

        0  2017-04-11 01:59   META-INF/maven/org.apache.flink/

        0  2017-04-11 01:59   META-INF/maven/org.apache.flink/flink-connector-kafka-0.9_2.11/

     6039  2017-04-11 01:59   META-INF/maven/org.apache.flink/flink-connector-kafka-0.9_2.11/pom.xml

      131  2017-04-11 01:59   META-INF/maven/org.apache.flink/flink-connector-kafka-0.9_2.11/pom.properties

        0  2017-04-11 01:59   META-INF/maven/org.apache.flink/force-shading/

     3285  2017-04-11 01:59   META-INF/maven/org.apache.flink/force-shading/pom.xml

      114  2017-04-11 01:59   META-INF/maven/org.apache.flink/force-shading/pom.properties

---------                     -------

    76543                     33 files

------

Mike Pryakhin

On 22 May 2017, at 22:14, Robert Metzger <[hidden email]> wrote:

Hi,

this issue is unexpected :) Can you double check if the job-libs/flink-connector-kafka-0.9_2.11-1.2.1.jar contains org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer09 ?

Also, were there any previous Kafka09 related exceptions in the log??

From this SO answer, it seems that this is not really the classical classNotFoundException, but a bit differenT: https://stackoverflow.com/a/5756989/568695

On Mon, May 22, 2017 at 5:12 PM, Mikhail Pryakhin <[hidden email]> wrote:

Hi all!

I'm playing with flink streaming job on yarn cluster. The job consumes events from kafka and prints them to the standard out.
The job uses flink-connector-kafka-0.9_2.11-1.2.1.jar library that is passed via the --yarnship option.
Here is the way I run the job:

export HADOOP_USER_NAME=hdfs; \
export HADOOP_CONF_DIR=/etc/hadoop/conf/; \
/opt/flink-1.2.1/bin/flink run \
        -yst \
        -yt /home/user/job-libs  \
        -m yarn-cluster \
        -yn 3 \
        -c com.flink.Test \
        flink-test_2.11-1.0.0-SNAPSHOT.jar

Finally the job fails complaing that it can't find the class:
java.lang.NoClassDefFoundError: org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer09

I looked through the jobmanager.log and found that the flink-connector-kafka-0.9_2.11-1.2.1.jar library is added to the classpath:

2017-05-22 17:41:59,637 INFO  org.apache.flink.yarn.YarnApplicationMasterRunner             -  Classpath: job-libs/flink-connector-kafka-0.9_2.11-1.2.1.jar:<another jars...>

Could please help to figure out why the class org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer09 can't be loaded inspite of the jar containing this class was added to the classpath

----
Mike Pryakhin

[Mikhail Pryakhin]

Hi Robert!

You were right.. it was my fault, I didn’t copy all required dependencies for the flink-connector-kafka. That made impossible to load the class.

Everything works as expected but only if I put required dependencies inside the flink lib folder, but when dependencies are located in a different folder which is passed in the yarnship command line option then they seem not to be put on the TM's classpath, and the job fails with ClassNotFoundException. 

Is there anything else I need to provide to make libraries in that folder to be on the TM's classpath?

Many thanks,

——

Mike Pryakhin

On 22 May 2017, at 23:06, Mikhail Pryakhin <[hidden email]> wrote:

Hi Robert!

Thanks a lot for your reply!

>Can you double check if the job-libs/flink-connector-kafka-0.9_2.11-1.2.1.jar contains org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer09 ?

The jar does contain the class org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer09.class (The contents of the jar file is listed below)

> Also, were there any previous Kafka09 related exceptions in the log??    

No, it was the very first exception… You can find log file attached. 

the flink-connector-kafka-0.9_2.11-1.2.1.jar contains the following:

root@host:~# unzip -l job-libs/flink-connector-kafka-0.9_2.11-1.2.1.jar

Archive:  job-libs/flink-connector-kafka-0.9_2.11-1.2.1.jar

  Length      Date    Time    Name

---------  ---------- -----   ----

        0  2017-04-11 01:59   META-INF/

      443  2017-04-11 01:59   META-INF/MANIFEST.MF

     1451  2017-04-11 01:59   META-INF/DEPENDENCIES

    11358  2017-04-11 01:59   META-INF/LICENSE

      182  2017-04-11 01:59   META-INF/NOTICE

        0  2017-04-11 01:59   org/

        0  2017-04-11 01:59   org/apache/

        0  2017-04-11 01:59   org/apache/flink/

        0  2017-04-11 01:59   org/apache/flink/streaming/

        0  2017-04-11 01:59   org/apache/flink/streaming/connectors/

        0  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/

    11554  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer09.class

     2603  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/Kafka09JsonTableSink.class

     4463  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/FlinkKafkaProducer09.class

     2693  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/Kafka09TableSource.class

        0  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/internal/

     1125  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/internal/KafkaConsumerCallBridge.class

      570  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/internal/Handover$WakeupException.class

    10268  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/internal/KafkaConsumerThread.class

      570  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/internal/Handover$ClosedException.class

     3018  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/internal/Handover.class

    11854  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/internal/Kafka09Fetcher.class

     2182  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/internal/KafkaConsumerThread$CommitCallback.class

      311  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/internal/KafkaConsumerThread$1.class

     2329  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/Kafka09JsonTableSource.class

        0  2017-04-11 01:59   META-INF/maven/

        0  2017-04-11 01:59   META-INF/maven/org.apache.flink/

        0  2017-04-11 01:59   META-INF/maven/org.apache.flink/flink-connector-kafka-0.9_2.11/

     6039  2017-04-11 01:59   META-INF/maven/org.apache.flink/flink-connector-kafka-0.9_2.11/pom.xml

      131  2017-04-11 01:59   META-INF/maven/org.apache.flink/flink-connector-kafka-0.9_2.11/pom.properties

        0  2017-04-11 01:59   META-INF/maven/org.apache.flink/force-shading/

     3285  2017-04-11 01:59   META-INF/maven/org.apache.flink/force-shading/pom.xml

      114  2017-04-11 01:59   META-INF/maven/org.apache.flink/force-shading/pom.properties

---------                     -------

    76543                     33 files

<flink-root-client-dmpkit-dev-dn1.log>

------

Mike Pryakhin

On 22 May 2017, at 22:14, Robert Metzger <[hidden email]> wrote:

Hi,

this issue is unexpected :) Can you double check if the job-libs/flink-connector-kafka-0.9_2.11-1.2.1.jar contains org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer09 ?

Also, were there any previous Kafka09 related exceptions in the log??

From this SO answer, it seems that this is not really the classical classNotFoundException, but a bit differenT: https://stackoverflow.com/a/5756989/568695

On Mon, May 22, 2017 at 5:12 PM, Mikhail Pryakhin <[hidden email]> wrote:

Hi all!

I'm playing with flink streaming job on yarn cluster. The job consumes events from kafka and prints them to the standard out.
The job uses flink-connector-kafka-0.9_2.11-1.2.1.jar library that is passed via the --yarnship option.
Here is the way I run the job:

export HADOOP_USER_NAME=hdfs; \
export HADOOP_CONF_DIR=/etc/hadoop/conf/; \
/opt/flink-1.2.1/bin/flink run \
        -yst \
        -yt /home/user/job-libs  \
        -m yarn-cluster \
        -yn 3 \
        -c com.flink.Test \
        flink-test_2.11-1.0.0-SNAPSHOT.jar

Finally the job fails complaing that it can't find the class:
java.lang.NoClassDefFoundError: org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer09

I looked through the jobmanager.log and found that the flink-connector-kafka-0.9_2.11-1.2.1.jar library is added to the classpath:

2017-05-22 17:41:59,637 INFO  org.apache.flink.yarn.YarnApplicationMasterRunner             -  Classpath: job-libs/flink-connector-kafka-0.9_2.11-1.2.1.jar:<another jars...>

Could please help to figure out why the class org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer09 can't be loaded inspite of the jar containing this class was added to the classpath

----
Mike Pryakhin

[rmetzger0]

Hi Mike,

I would recommend you to build a "fat jar" containing your application code and all required dependencies.

On Tue, May 23, 2017 at 10:33 AM, Mikhail Pryakhin <[hidden email]> wrote:

Hi Robert!

You were right.. it was my fault, I didn’t copy all required dependencies for the flink-connector-kafka. That made impossible to load the class.

Everything works as expected but only if I put required dependencies inside the flink lib folder, but when dependencies are located in a different folder which is passed in the yarnship command line option then they seem not to be put on the TM's classpath, and the job fails with ClassNotFoundException. 

Is there anything else I need to provide to make libraries in that folder to be on the TM's classpath?

Many thanks,

——

Mike Pryakhin

On 22 May 2017, at 23:06, Mikhail Pryakhin <[hidden email]> wrote:

Hi Robert!

Thanks a lot for your reply!

>Can you double check if the job-libs/flink-connector-kafka-0.9_2.11-1.2.1.jar contains org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer09 ?

The jar does contain the class org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer09.class (The contents of the jar file is listed below)

> Also, were there any previous Kafka09 related exceptions in the log??    

No, it was the very first exception… You can find log file attached. 

the flink-connector-kafka-0.9_2.11-1.2.1.jar contains the following:

root@host:~# unzip -l job-libs/flink-connector-kafka-0.9_2.11-1.2.1.jar

Archive:  job-libs/flink-connector-kafka-0.9_2.11-1.2.1.jar

  Length      Date    Time    Name

---------  ---------- -----   ----

        0  2017-04-11 01:59   META-INF/

      443  2017-04-11 01:59   META-INF/MANIFEST.MF

     1451  2017-04-11 01:59   META-INF/DEPENDENCIES

    11358  2017-04-11 01:59   META-INF/LICENSE

      182  2017-04-11 01:59   META-INF/NOTICE

        0  2017-04-11 01:59   org/

        0  2017-04-11 01:59   org/apache/

        0  2017-04-11 01:59   org/apache/flink/

        0  2017-04-11 01:59   org/apache/flink/streaming/

        0  2017-04-11 01:59   org/apache/flink/streaming/connectors/

        0  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/

    11554  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer09.class

     2603  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/Kafka09JsonTableSink.class

     4463  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/FlinkKafkaProducer09.class

     2693  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/Kafka09TableSource.class

        0  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/internal/

     1125  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/internal/KafkaConsumerCallBridge.class

      570  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/internal/Handover$WakeupException.class

    10268  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/internal/KafkaConsumerThread.class

      570  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/internal/Handover$ClosedException.class

     3018  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/internal/Handover.class

    11854  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/internal/Kafka09Fetcher.class

     2182  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/internal/KafkaConsumerThread$CommitCallback.class

      311  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/internal/KafkaConsumerThread$1.class

     2329  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/Kafka09JsonTableSource.class

        0  2017-04-11 01:59   META-INF/maven/

        0  2017-04-11 01:59   META-INF/maven/org.apache.flink/

        0  2017-04-11 01:59   META-INF/maven/org.apache.flink/flink-connector-kafka-0.9_2.11/

     6039  2017-04-11 01:59   META-INF/maven/org.apache.flink/flink-connector-kafka-0.9_2.11/pom.xml

      131  2017-04-11 01:59   META-INF/maven/org.apache.flink/flink-connector-kafka-0.9_2.11/pom.properties

        0  2017-04-11 01:59   META-INF/maven/org.apache.flink/force-shading/

     3285  2017-04-11 01:59   META-INF/maven/org.apache.flink/force-shading/pom.xml

      114  2017-04-11 01:59   META-INF/maven/org.apache.flink/force-shading/pom.properties

---------                     -------

    76543                     33 files

<flink-root-client-dmpkit-dev-dn1.log>

------

Mike Pryakhin

On 22 May 2017, at 22:14, Robert Metzger <[hidden email]> wrote:

Hi,

this issue is unexpected :) Can you double check if the job-libs/flink-connector-kafka-0.9_2.11-1.2.1.jar contains org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer09 ?

Also, were there any previous Kafka09 related exceptions in the log??

From this SO answer, it seems that this is not really the classical classNotFoundException, but a bit differenT: https://stackoverflow.com/a/5756989/568695

On Mon, May 22, 2017 at 5:12 PM, Mikhail Pryakhin <[hidden email]> wrote:

Hi all!

I'm playing with flink streaming job on yarn cluster. The job consumes events from kafka and prints them to the standard out.
The job uses flink-connector-kafka-0.9_2.11-1.2.1.jar library that is passed via the --yarnship option.
Here is the way I run the job:

export HADOOP_USER_NAME=hdfs; \
export HADOOP_CONF_DIR=/etc/hadoop/conf/; \
/opt/flink-1.2.1/bin/flink run \
        -yst \
        -yt /home/user/job-libs  \
        -m yarn-cluster \
        -yn 3 \
        -c com.flink.Test \
        flink-test_2.11-1.0.0-SNAPSHOT.jar

Finally the job fails complaing that it can't find the class:
java.lang.NoClassDefFoundError: org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer09

I looked through the jobmanager.log and found that the flink-connector-kafka-0.9_2.11-1.2.1.jar library is added to the classpath:

2017-05-22 17:41:59,637 INFO  org.apache.flink.yarn.YarnApplicationMasterRunner             -  Classpath: job-libs/flink-connector-kafka-0.9_2.11-1.2.1.jar:<another jars...>

Could please help to figure out why the class org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer09 can't be loaded inspite of the jar containing this class was added to the classpath

----
Mike Pryakhin

[Mikhail Pryakhin]

Hi Robert,

Thanks for your advice but this option doesn’t work for us because our job depends on third party jars with transitive dependencies to the same jar but of different versions and it hardly possible to say which version will participate in the fat jar..

Anyway thanks a lot for your help.

—— 

Mike Pryakhin

On 23 May 2017, at 16:04, Robert Metzger <[hidden email]> wrote:

Hi Mike,

I would recommend you to build a "fat jar" containing your application code and all required dependencies.

On Tue, May 23, 2017 at 10:33 AM, Mikhail Pryakhin <[hidden email]> wrote:

Hi Robert!

You were right.. it was my fault, I didn’t copy all required dependencies for the flink-connector-kafka. That made impossible to load the class.

Everything works as expected but only if I put required dependencies inside the flink lib folder, but when dependencies are located in a different folder which is passed in the yarnship command line option then they seem not to be put on the TM's classpath, and the job fails with ClassNotFoundException. 

Is there anything else I need to provide to make libraries in that folder to be on the TM's classpath?

Many thanks,

——

Mike Pryakhin

On 22 May 2017, at 23:06, Mikhail Pryakhin <[hidden email]> wrote:

Hi Robert!

Thanks a lot for your reply!

>Can you double check if the job-libs/flink-connector-kafka-0.9_2.11-1.2.1.jar contains org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer09 ?

The jar does contain the class org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer09.class (The contents of the jar file is listed below)

> Also, were there any previous Kafka09 related exceptions in the log??    

No, it was the very first exception… You can find log file attached. 

the flink-connector-kafka-0.9_2.11-1.2.1.jar contains the following:

root@host:~# unzip -l job-libs/flink-connector-kafka-0.9_2.11-1.2.1.jar

Archive:  job-libs/flink-connector-kafka-0.9_2.11-1.2.1.jar

  Length      Date    Time    Name

---------  ---------- -----   ----

        0  2017-04-11 01:59   META-INF/

      443  2017-04-11 01:59   META-INF/MANIFEST.MF

     1451  2017-04-11 01:59   META-INF/DEPENDENCIES

    11358  2017-04-11 01:59   META-INF/LICENSE

      182  2017-04-11 01:59   META-INF/NOTICE

        0  2017-04-11 01:59   org/

        0  2017-04-11 01:59   org/apache/

        0  2017-04-11 01:59   org/apache/flink/

        0  2017-04-11 01:59   org/apache/flink/streaming/

        0  2017-04-11 01:59   org/apache/flink/streaming/connectors/

        0  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/

    11554  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer09.class

     2603  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/Kafka09JsonTableSink.class

     4463  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/FlinkKafkaProducer09.class

     2693  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/Kafka09TableSource.class

        0  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/internal/

     1125  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/internal/KafkaConsumerCallBridge.class

      570  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/internal/Handover$WakeupException.class

    10268  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/internal/KafkaConsumerThread.class

      570  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/internal/Handover$ClosedException.class

     3018  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/internal/Handover.class

    11854  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/internal/Kafka09Fetcher.class

     2182  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/internal/KafkaConsumerThread$CommitCallback.class

      311  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/internal/KafkaConsumerThread$1.class

     2329  2017-04-11 01:59   org/apache/flink/streaming/connectors/kafka/Kafka09JsonTableSource.class

        0  2017-04-11 01:59   META-INF/maven/

        0  2017-04-11 01:59   META-INF/maven/org.apache.flink/

        0  2017-04-11 01:59   META-INF/maven/org.apache.flink/flink-connector-kafka-0.9_2.11/

     6039  2017-04-11 01:59   META-INF/maven/org.apache.flink/flink-connector-kafka-0.9_2.11/pom.xml

      131  2017-04-11 01:59   META-INF/maven/org.apache.flink/flink-connector-kafka-0.9_2.11/pom.properties

        0  2017-04-11 01:59   META-INF/maven/org.apache.flink/force-shading/

     3285  2017-04-11 01:59   META-INF/maven/org.apache.flink/force-shading/pom.xml

      114  2017-04-11 01:59   META-INF/maven/org.apache.flink/force-shading/pom.properties

---------                     -------

    76543                     33 files

<flink-root-client-dmpkit-dev-dn1.log>

------

Mike Pryakhin

On 22 May 2017, at 22:14, Robert Metzger <[hidden email]> wrote:

Hi,

this issue is unexpected :) Can you double check if the job-libs/flink-connector-kafka-0.9_2.11-1.2.1.jar contains org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer09 ?

Also, were there any previous Kafka09 related exceptions in the log??

From this SO answer, it seems that this is not really the classical classNotFoundException, but a bit differenT: https://stackoverflow.com/a/5756989/568695

On Mon, May 22, 2017 at 5:12 PM, Mikhail Pryakhin <[hidden email]> wrote:

Hi all!

I'm playing with flink streaming job on yarn cluster. The job consumes events from kafka and prints them to the standard out.
The job uses flink-connector-kafka-0.9_2.11-1.2.1.jar library that is passed via the --yarnship option.
Here is the way I run the job:

export HADOOP_USER_NAME=hdfs; \
export HADOOP_CONF_DIR=/etc/hadoop/conf/; \
/opt/flink-1.2.1/bin/flink run \
        -yst \
        -yt /home/user/job-libs  \
        -m yarn-cluster \
        -yn 3 \
        -c com.flink.Test \
        flink-test_2.11-1.0.0-SNAPSHOT.jar

Finally the job fails complaing that it can't find the class:
java.lang.NoClassDefFoundError: org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer09

I looked through the jobmanager.log and found that the flink-connector-kafka-0.9_2.11-1.2.1.jar library is added to the classpath:

2017-05-22 17:41:59,637 INFO  org.apache.flink.yarn.YarnApplicationMasterRunner             -  Classpath: job-libs/flink-connector-kafka-0.9_2.11-1.2.1.jar:<another jars...>

Could please help to figure out why the class org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer09 can't be loaded inspite of the jar containing this class was added to the classpath

----
Mike Pryakhin