High availability flink job

[Vishwas Siravara]

Hi guys,

I have a flink job running in standalone mode with a parallelism of >1 , that produces data to a kafka sink. My topic is replicated with a replication factor of 2. Now suppose one of the kafka brokers goes down , then will my streaming job fail ? Is there a way where in I can continue processing until that broker node comes up, also the sink is partitioned by a key. 

Thanks,

Vishwas 

[PoolakkalMukkath, Shakir]

I do not think you will have any problem with one broker going down as long as you have provided enough brokers on the bootstrap.

 

Thanks,

Shakir

 

From: Vishwas Siravara <[hidden email]>
Date: Sunday, September 15, 2019 at 2:52 PM
To: user <[hidden email]>
Subject: [EXTERNAL] High availability flink job

 

Hi guys,

I have a flink job running in standalone mode with a parallelism of >1 , that produces data to a kafka sink. My topic is replicated with a replication factor of 2. Now suppose one of the kafka brokers goes down , then will my streaming job fail ? Is there a way where in I can continue processing until that broker node comes up, also the sink is partitioned by a key. 

 

Thanks,

Vishwas