Re: Per Partition Watermarking source idleness

Posted by Eduardo Winpenny Tejedor on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Per-Partition-Watermarking-source-idleness-tp29596p29613.html

Hi Prakhar,

Everything is probably working as expected, if a partition does not receive any messages then the watermark of the operator does not advance (as it is the minimum across all partitions).

You'll need to define a strategy for the watermark to advance even when no messages are received for a particular partition.

Regards,

Eduardo

On Fri, 23 Aug 2019, 10:35 Prakhar Mathur, <[hidden email]> wrote:

Hi,

We are using flink v1.6. We are facing data loss issues while consuming data from older offsets in Kafka with windowing. We are exploring per partition watermarking strategy. But we noticed that when we are trying to consume from multiple topics and if any of the partition is not receiving data it just blocks everything. Do we have a known solution for this?