Question regarding refreshing KafkaConsumer in FlinkKafkaConnector

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Question regarding refreshing KafkaConsumer in FlinkKafkaConnector

JaxonTheGamer AndCooker
Hi Flink support,

I'm trying to use FlinkKafkaConnector010 to do the streaming from KafkaBroker Sasl. Currently, I got problem with Token refreshing every 24hours that's why I want restart the KafkaConsumer every 24hours. 

Since KafkaConsumer is wrapped by FlinkKafkaConnector010, I'm trying to re-new FlinkKafkaConnector010 every 24h. However, since the Flink job is already submitted the re-new FlinkKfakaConnector010 is not taking effected to the current running Flink job. Could you please let me know if there is any feature to do a scheduler to restart Flink job? Or you have any idea to have it work in my case? 

Thanks a lot for your help!

Best regards,

Le Hoang

Le Hoang Nguyen
Cell Phone: 312-480-0568

Reply | Threaded
Open this post in threaded view
|

Re: Question regarding refreshing KafkaConsumer in FlinkKafkaConnector

Tzu-Li (Gordon) Tai
Hi,

AFAIK, there is no built-in feature for scheduling / orchestrating
submission of Flink jobs.
However, you should be able to easily use tools like cron jobs to do that.
It should work by just taking a savepoint of your running job, and then
resuming for that, and you do this periodically.

Cheers,
Gordon



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/