Watermark on connected stream

[Kien Truong]

Hi,

If I connect two stream with different watermark, how are the watermark
of the resulting stream determined ?


Best regards,

Kien

[Piotr Nowojski]

Hi,

As you can see in org.apache.flink.streaming.api.operators.AbstractStreamOperator#processWatermark1 it takes a minimum of both of the inputs.

Piotrek
 

> On 19 Oct 2017, at 14:06, Kien Truong <[hidden email]> wrote:
>
> Hi,
>
> If I connect two stream with different watermark, how are the watermark of the resulting stream determined ?
>
>
> Best regards,
>
> Kien
>

[Tzu-Li (Gordon) Tai]

Hi Kien,

The watermark of an operator with multiple inputs will be determined by the current minimum watermark across all inputs.

Cheers,

Gordon

On 19 October 2017 at 8:06:11 PM, Kien Truong ([hidden email]) wrote:

Hi, 

If I connect two stream with different watermark, how are the watermark 
of the resulting stream determined ? 


Best regards, 

Kien 

[Tzu-Li (Gordon) Tai]

Ah, sorry for the duplicate answer, I didn’t see Piotr’s reply. Slight delay on the mail client.

On 19 October 2017 at 11:05:01 PM, Tzu-Li (Gordon) Tai ([hidden email]) wrote:

Hi Kien,

The watermark of an operator with multiple inputs will be determined by the current minimum watermark across all inputs.

Cheers,

Gordon

On 19 October 2017 at 8:06:11 PM, Kien Truong ([hidden email]) wrote:

Hi, 

If I connect two stream with different watermark, how are the watermark 
of the resulting stream determined ? 


Best regards, 

Kien 

[aitozi]

Hi,

You can see the field in AbstractStreamOperator

// We keep track of watermarks from both inputs, the combined input is the
minimum
// Once the minimum advances we emit a new watermark for downstream
operators
private long combinedWatermark = Long.MIN_VALUE;

it will chose the Min watermark from the connect stream



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[Piotr Nowojski]

With aitozi we have a hat trick oO

 

On 19 Oct 2017, at 17:08, Tzu-Li (Gordon) Tai <[hidden email]> wrote:

Ah, sorry for the duplicate answer, I didn’t see Piotr’s reply. Slight delay on the mail client.

On 19 October 2017 at 11:05:01 PM, Tzu-Li (Gordon) Tai ([hidden email]) wrote:

Hi Kien,

The watermark of an operator with multiple inputs will be determined by the current minimum watermark across all inputs.

Cheers,

Gordon

On 19 October 2017 at 8:06:11 PM, Kien Truong ([hidden email]) wrote:

Hi, 

If I connect two stream with different watermark, how are the watermark 
of the resulting stream determined ? 


Best regards, 

Kien