Getting "ProducerFenced" exception while using flink kafka producer

[Gaurav Luthra]

Hi,

I have two flink jobs, both the jobs are using Flink Kafka Producer and Flink Kafka Consumer running in Exactly-Once mode.

Parallelism of both the jobs is one.

Both the jobs are same in number of operators and type of operators.

When we start one job then that job runs fine. But as soon as we start the second job then both the jobs start failing with "ProducerFenced" Exception at runtime.

As per our understanding we think that both the jobs get the same value of transactional ID (assigned by flink), that is required to run the job in Exactly-Once mode.

I think, Flink calculates the transactional ID with concatenation of operator names and subtask ID. For our case in both the jobs we have same operators, and both jobs runs with parallelism of one, so both jobs get subtask ID. 

Question:

Kindly provide the solution for this exception, Kindly show some light on our understanding the reason of this exception. And kindly tell how can we get different transactional ID for two jobs.