Kafka Fetch Failed / DisconnectException

[Jonas Gröger]

Hi!

According to the output, I'm having some problems with the KafkaConsumer09. It reports the following on stdout:

[error] 2017-01-18 17:33:22,857 DEBUG org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient  - Cancelled FETCH request ClientRequest(expectResponse=true, callback=org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient$RequestFutureCompletionHandler@7a2c5a92, request=RequestSend(header={api_key=1,api_version=1,correlation_id=1441,client_id=consumer-6}, body={replica_id=-1,max_wait_time=500,min_bytes=1,topics=[{topic=my.topic,partitions=[{partition=4,fetch_offset=1426476,max_bytes=1048576},{partition=10,fetch_offset=1428289,max_bytes=1048576}]}]}), createdTimeMs=1484757202854, sendTimeMs=0) with correlation id 1441 due to node 1008 being disconnected
[error] 2017-01-18 17:33:22,857 DEBUG org.apache.kafka.clients.consumer.internals.Fetcher           - Fetch failed
[error] org.apache.kafka.common.errors.DisconnectException
[error] 2017-01-18 17:33:22,857 DEBUG org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient  - Cancelled FETCH request ClientRequest(expectResponse=true, callback=org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient$RequestFutureCompletionHandler@2ba5554c, request=RequestSend(header={api_key=1,api_version=1,correlation_id=1442,client_id=consumer-6}, body={replica_id=-1,max_wait_time=500,min_bytes=1,topics=[{topic=my.topic,partitions=[{partition=18,fetch_offset=1402333,max_bytes=1048576},{partition=6,fetch_offset=1403272,max_bytes=1048576}]}]}), createdTimeMs=1484757202854, sendTimeMs=0) with correlation id 1442 due to node 1007 being disconnected
[error] 2017-01-18 17:33:22,857 DEBUG org.apache.kafka.clients.consumer.internals.Fetcher           - Fetch failed
[error] org.apache.kafka.common.errors.DisconnectException
[error] 2017-01-18 17:33:22,857 DEBUG org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient  - Cancelled FETCH request ClientRequest(expectResponse=true, callback=org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient$RequestFutureCompletionHandler@4ad26e7f, request=RequestSend(header={api_key=1,api_version=1,correlation_id=1443,client_id=consumer-6}, body={replica_id=-1,max_wait_time=500,min_bytes=1,topics=[{topic=my.topic,partitions=[{partition=11,fetch_offset=1464080,max_bytes=1048576}]}]}), createdTimeMs=1484757202854, sendTimeMs=0) with correlation id 1443 due to node 1006 being disconnected
[error] 2017-01-18 17:33:22,857 DEBUG org.apache.kafka.clients.consumer.internals.Fetcher           - Fetch failed
[error] org.apache.kafka.common.errors.DisconnectException
[error] 2017-01-18 17:33:22,955 DEBUG org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient  - Cancelled FETCH request ClientRequest(expectResponse=true, callback=org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient$RequestFutureCompletionHandler@21b56cad, request=RequestSend(header={api_key=1,api_version=1,correlation_id=1444,client_id=consumer-6}, body={replica_id=-1,max_wait_time=500,min_bytes=1,topics=[{topic=my.topic,partitions=[{partition=4,fetch_offset=1426476,max_bytes=1048576},{partition=10,fetch_offset=1428289,max_bytes=1048576}]}]}), createdTimeMs=1484757202857, sendTimeMs=0) with correlation id 1444 due to node 1008 being disconnected
[error] 2017-01-18 17:33:22,955 DEBUG org.apache.kafka.clients.consumer.internals.Fetcher           - Fetch failed
[error] org.apache.kafka.common.errors.DisconnectException
[error] 2017-01-18 17:33:22,955 DEBUG org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient  - Cancelled FETCH request ClientRequest(expectResponse=true, callback=org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient$RequestFutureCompletionHandler@39d55ac3, request=RequestSend(header={api_key=1,api_version=1,correlation_id=1445,client_id=consumer-6}, body={replica_id=-1,max_wait_time=500,min_bytes=1,topics=[{topic=my.topic,partitions=[{partition=18,fetch_offset=1402333,max_bytes=1048576},{partition=6,fetch_offset=1403272,max_bytes=1048576}]}]}), createdTimeMs=1484757202857, sendTimeMs=0) with correlation id 1445 due to node 1007 being disconnected
[error] 2017-01-18 17:33:22,955 DEBUG org.apache.kafka.clients.consumer.internals.Fetcher           - Fetch failed
[error] org.apache.kafka.common.errors.DisconnectException
[error] 2017-01-18 17:33:22,955 DEBUG org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient  - Cancelled FETCH request ClientRequest(expectResponse=true, callback=org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient$RequestFutureCompletionHandler@38db7780, request=RequestSend(header={api_key=1,api_version=1,correlation_id=1446,client_id=consumer-6}, body={replica_id=-1,max_wait_time=500,min_bytes=1,topics=[{topic=my.topic,partitions=[{partition=11,fetch_offset=1464080,max_bytes=1048576}]}]}), createdTimeMs=1484757202857, sendTimeMs=0) with correlation id 1446 due to node 1006 being disconnected
[error] 2017-01-18 17:33:22,955 DEBUG org.apache.kafka.clients.consumer.internals.Fetcher           - Fetch failed

Is that something I should worry about?

-- Jonas

[Fabian Hueske-2]

Hi Jonas,

your mail did not include the error message. Can you send it again?

Thanks, Fabian

2017-01-18 17:37 GMT+01:00 Jonas <[hidden email]>:

Hi!

According to the output, I'm having some problems with the KafkaConsumer09.
It reports the following on stdout:



Is that something I should worry about?

-- Jonas



--
View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Kafka-Fetch-Failed-DisconnectException-tp11142.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.

[Jonas Gröger]

Hallo Fabian,

that IS the error message. The job continues to run without restarting. There is not really more to see from the logs. Its just something that occurs often in the logs so I might want to check this out :)

-- Jonas