java.lang.NoClassDefFoundError --- FlinkKafkaConsumer

[syed]

hi
I am trying to add a kafka source to the standard WordCount application
available with flink, So that the application may count words from the kafka
source. I added the source like this

 *DataStream<String> text;

if(params.has("topic")&&params.has("bootstrap.servers")&&params.has("zookeeper.connect")
 &&params.has("group.id")) {
     text = env.addSource(new
FlinkKafkaConsumer<String>(params.get("topic"), new SimpleStringSchema(),
params.getProperties())
             .setStartFromEarliest());
 }
else if (params.has("input")) {
     // read the text file from given input path
     text = env.readTextFile(params.get("input"));
 } else {
     System.out.println("Executing WordCount example with default input data
set.");
     System.out.println("Use --input to specify file input.");
     // get default test text data
     text = env.fromElements(WordCountData.WORDS);
 }*

When running the application I got the following exception

java.lang.NoClassDefFoundError:
org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer
        at WordCount.main(WordCount.java:79)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at
org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:529)
        at
org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:421)
        at
org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:423)
        at
org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:813)
        at org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:287)
        at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:213)
        at
org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1050)
        at
org.apache.flink.client.cli.CliFrontend.lambda$main$11(CliFrontend.java:1126)
        at
org.apache.flink.runtime.security.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
        at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1126)
Caused by: java.lang.ClassNotFoundException:
org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer
        at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        ... 15 more

Please guide me how to fix this exception.
Kind Regards;
Syed



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[Yun Tang]

Hi Syed

Have you ever build your user application jar package with all dependencies as [1] suggested or copy related connector jar package from flink-home/opt/ folder to flink-home/lib/ folder?

[1] https://ci.apache.org/projects/flink/flink-docs-stable/dev/projectsetup/dependencies.html#appendix-template-for-building-a-jar-with-dependencies

Best

Yun Tang

---

From: syed <[hidden email]>
Sent: Thursday, June 13, 2019 16:28
To: [hidden email]
Subject: java.lang.NoClassDefFoundError --- FlinkKafkaConsumer

 

hi
I am trying to add a kafka source to the standard WordCount application
available with flink, So that the application may count words from the kafka
source. I added the source like this

 *DataStream<String> text;

if(params.has("topic")&&params.has("bootstrap.servers")&&params.has("zookeeper.connect")
 &&params.has("group.id")) {
     text = env.addSource(new
FlinkKafkaConsumer<String>(params.get("topic"), new SimpleStringSchema(),
params.getProperties())
             .setStartFromEarliest());
 }
else if (params.has("input")) {
     // read the text file from given input path
     text = env.readTextFile(params.get("input"));
 } else {
     System.out.println("Executing WordCount example with default input data
set.");
     System.out.println("Use --input to specify file input.");
     // get default test text data
     text = env.fromElements(WordCountData.WORDS);
 }*

When running the application I got the following exception

java.lang.NoClassDefFoundError:
org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer
        at WordCount.main(WordCount.java:79)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at
org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:529)
        at
org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:421)
        at
org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:423)
        at
org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:813)
        at org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:287)
        at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:213)
        at
org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1050)
        at
org.apache.flink.client.cli.CliFrontend.lambda$main$11(CliFrontend.java:1126)
        at
org.apache.flink.runtime.security.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
        at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1126)
Caused by: java.lang.ClassNotFoundException:
org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer
        at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        ... 15 more

Please guide me how to fix this exception.
Kind Regards;
Syed



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/