Kafka exception "Unable to find a leader for partitions"

[Shannon Carey]

Does anyone have a guess what might cause this exception?

java.lang.RuntimeException: Unable to find a leader for partitions: [FetchPartition {topic=usersignals, partition=1, offset=2825838}]

at org.apache.flink.streaming.connectors.kafka.internals.LegacyFetcher.findLeaderForPartitions(LegacyFetcher.java:323)

at org.apache.flink.streaming.connectors.kafka.internals.LegacyFetcher.run(LegacyFetcher.java:162)

at org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer08.run(FlinkKafkaConsumer08.java:316)

at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:78)

at org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run(SourceStreamTask.java:56)

at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:225)

at org.apache.flink.runtime.taskmanager.Task.run(Task.java:559)

at java.lang.Thread.run(Thread.java:745)

[rmetzger0]

Hi Shannon,

Some questions:

which Flink version are you using?

Can you provide me with some more logs, in particular the log entries before this event from the Kafka connector.

Also, it is possible that the Kafka broker was in an erroneous state?

Did the error happen after weeks of data consumption, or early?

And last question: how many of the available brokers did you put into the bootstrap.server property ?

Regards,

Robert

On Wed, Jun 8, 2016 at 7:11 PM, Shannon Carey <[hidden email]> wrote:

Does anyone have a guess what might cause this exception?

java.lang.RuntimeException: Unable to find a leader for partitions: [FetchPartition {topic=usersignals, partition=1, offset=2825838}]

at org.apache.flink.streaming.connectors.kafka.internals.LegacyFetcher.findLeaderForPartitions(LegacyFetcher.java:323)

at org.apache.flink.streaming.connectors.kafka.internals.LegacyFetcher.run(LegacyFetcher.java:162)

at org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer08.run(FlinkKafkaConsumer08.java:316)

at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:78)

at org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run(SourceStreamTask.java:56)

at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:225)

at org.apache.flink.runtime.taskmanager.Task.run(Task.java:559)

at java.lang.Thread.run(Thread.java:745)