Publish heartbeat messages in all Kafka partitions

[Alexey Trenikhun]

Hello,

We need to publish heartbeat messages in all topic partitions. Is possible to produce single message and then somehow broadcast it to all partitions from FlinkKafkaProducer? Or only way that message source knows number of existing partitions and sends 1 message to 1 partition?

Thanks,
Alexey

[Arvid Heise-4]

Hi Alexey,

I don't see a way to do it with one message in FlinkKafkaProducer. So you have to multiply the heartbeat yourself. I'd imagine the easiest way would be to query the number of partitions of the output topic (it's static in Kafka) in the heartbeat producer and already produce as all records.

Best,

Arvid

On Fri, Jan 29, 2021 at 1:13 AM Alexey Trenikhun <[hidden email]> wrote:

Hello,

We need to publish heartbeat messages in all topic partitions. Is possible to produce single message and then somehow broadcast it to all partitions from FlinkKafkaProducer? Or only way that message source knows number of existing partitions and sends 1 message to 1 partition?

Thanks,
Alexey