Duplication error on Kafka Connector Libraries

[Kevin Kwon]

Hi community, I'm testing out 1.12-SNAPSHOT in master branch

I built my application with library 'flink-connector-kafka' but when I start the app, I get

Caused by: org.apache.kafka.common.KafkaException: class org.apache.kafka.common.serialization.ByteArrayDeserializer is not an instance of org.apache.kafka.common.serialization.Deserializer

while constructing KafkaConsumer class. Is this normal behavior?

[Arvid Heise-3]

The most common cause of such issues is usually class loading.

You probably have added the flink-connector-kafka also to flink-dist/libs. But the connector is only meant to be bundled with your job jar afaik. Right now, you have the Kafka classes loaded in the user code classloader and in the system classloader of Flink itself.

On Fri, Nov 27, 2020 at 12:24 AM Kevin Kwon <[hidden email]> wrote:

Hi community, I'm testing out 1.12-SNAPSHOT in master branch

I built my application with library 'flink-connector-kafka' but when I start the app, I get

Caused by: org.apache.kafka.common.KafkaException: class org.apache.kafka.common.serialization.ByteArrayDeserializer is not an instance of org.apache.kafka.common.serialization.Deserializer

while constructing KafkaConsumer class. Is this normal behavior?

--