Issue on watermark meaning

[Lorenzo Affetti]

Hi everybody,

I want to signal that I think there is a mismatch between what is the meaning of emitting a watermark between the code and the documentation:

from Flink docs: A watermark with a certain timestamp denotes the knowledge that no event with timestamp lower than the timestamp of the watermark will ever arrive.

from Flink code (Output.java):
A watermark specifies that no element with a timestamp older or equal to the watermark timestamp will be emitted in the future.

I think that the documentation should be fixed to match "lower or equal to" because it is what the system really does, I think.

Thank you

--

Lorenzo Affetti

[Till Rohrmann]

Hi Lorenzo,

you're right that we should stick to the same terminology between the online documentation and the code, otherwise it's confusing. In this case, though, a lower numeric timestamp is equivalent to an older event. The older an element is, the lower is its timestamp. 

However, there is a another problem with the formulation. In the first sentence it's said no event with a lower (=older) timestamp whereas in the second sentence it's additional said that no event with the same timestamp as the watermark will be emitted. This is not consistent. We'll fix that.

Cheers,

Till

On Fri, Jan 29, 2016 at 10:49 AM, Lorenzo Affetti <[hidden email]> wrote:

Hi everybody,

I want to signal that I think there is a mismatch between what is the meaning of emitting a watermark between the code and the documentation:

from Flink docs: A watermark with a certain timestamp denotes the knowledge that no event with timestamp lower than the timestamp of the watermark will ever arrive.

from Flink code (Output.java):
A watermark specifies that no element with a timestamp older or equal to the watermark timestamp will be emitted in the future.

I think that the documentation should be fixed to match "lower or equal to" because it is what the system really does, I think.

Thank you

--

Lorenzo Affetti