Does Flink merge streams deterministically?

[amran dean]

The Flink Kafka consumer's watermark extractor claims:.

"The watermark extractor will run per Kafka partition, watermarks will be merged across partitions in the same way as in the Flink runtime, when streams are merged."

Suppose we have simple situation where two streams merge into one. Is the ordering of the merged stream deterministic upon retries? 

My pipeline relies on this property to guarantee determinism due to how watermarks are emitted.

Thanks for any input

[Till Rohrmann]

Hi Amran,

in the general case, the merging of two streams is not deterministic.

Cheers,

Till

On Thu, Nov 7, 2019 at 3:24 AM amran dean <[hidden email]> wrote:

The Flink Kafka consumer's watermark extractor claims:.

"The watermark extractor will run per Kafka partition, watermarks will be merged across partitions in the same way as in the Flink runtime, when streams are merged."

Suppose we have simple situation where two streams merge into one. Is the ordering of the merged stream deterministic upon retries? 

My pipeline relies on this property to guarantee determinism due to how watermarks are emitted.

Thanks for any input