Re: Flink 0.10.2 and Kafka 0.8.1

Posted by rmetzger0 on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Flink-0-10-2-and-Kafka-0-8-1-tp6154p6165.html

Hi,

did you check your user jar if it contains the Kafka classes?

Are you building a fat jar? Are you manually excluding any dependencies?

Flink's 0.10.2 Kafka connector depends on Kafka 0.8.2.0 [1] which in turn depends on kafka-clients 0.8.2.0 [2]. And the "kafka-clients" dependency also contains the org.apache.kafka.common.Node class (the LegacyFetcher needs that class).

[1] https://repo1.maven.org/maven2/org/apache/flink/flink-connector-kafka/0.10.2/flink-connector-kafka-0.10.2.pom

[2] https://repo1.maven.org/maven2/org/apache/kafka/kafka_2.10/0.8.2.0/kafka_2.10-0.8.2.0.pom

On Mon, Apr 18, 2016 at 7:42 AM, Balaji Rajagopalan <[hidden email]> wrote:

I had fought with 0.8.0.2 kafka and flink 0.10.2 scala version 2.11, was never able to get it working confounded with noclassdeffounderror, moved to flink 1.0.0 with kafka 0.8.0.2  scala version 2.11 things worked for me, if moving to flink 1.0.0 is an option for you do so. 

balaji 

On Mon, Apr 18, 2016 at 3:19 AM, Robert Schmidtke <[hidden email]> wrote:

Hi everyone,

I have a Kafka cluster running on version 0.8.1, hence I'm using the FlinkKafkaConsumer081. When running my program, I saw a NoClassDefFoundError for org.apache.kafka.common.Node. So I packaged my binaries according to https://ci.apache.org/projects/flink/flink-docs-release-0.10/apis/cluster_execution.html#linking-with-modules-not-contained-in-the-binary-distribution, however I'm still seeing the error.

I played around a bit and it turns out I have to package kafka-clients v. 0.8.2.0 instead of kafka_2.10 v. 0.8.1 with my program. Is there an error in the documentation or have I not figured out something properly?

Thanks!

Robert

--

My GPG Key ID: 336E2680