Re: KakfaConsumer
Posted by
Nagarjun Guraja on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/KakfaConsumer-tp25913p25914.html
Hi Vishal,
Flink does checkpoint to Kafka(Offset commits) by default which could be disabled. Look
here for more information.
Regards,
Nagarjun
Success is not final, failure is not fatal: it is the courage to continue that counts.
- Winston Churchill -
On Mon, Feb 4, 2019 at 3:31 PM Vishal Santoshi <
[hidden email]> wrote:
A simple query
Does Flink' KafkaConnector flush the current offsets to kafka on a SP ? Note that the I do koow that Flink consumes data from Kafka topics and periodically checkpoints using Flink's distributed checkpointing mechanism. In case of failure, Flink will restore the records from checkpoint directory and will start reading data from Kafka offset after that. In this case however I want to restart a radically different pipe but from an offset that is part of the SP and the hope is that I could just depend on the kafka offsets of the consumer group.
Thanks