Re: Timestamp and key preservation over operators
Posted by
Fabian Hueske-2 on
May 03, 2019; 7:44am
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Timestamp-and-key-preservation-over-operators-tp27627p27698.html
Hi Averell,
Yes, timestamps and watermarks do not (completely) move together.
The watermark should always be lower than the timestamps of the currently processed records.
Otherwise, the records might be processed as late records (depending on the logic).
The easiest way to check the timestamp of a message is using a ProcessFunction.
The Context of the processElement() method has a timestamp() method that returns the timestamp of the current record.
Best, Fabian
Thank you Fabian.
I have one more question about timestamp:
In the previous email, you asked how did I check the timestamp - I don't
have an answer. Then I only checked the watermark, not the timestamp. I had
the (wrong) assumption that watermarks advance along with timestamps.
Today I played with that early trigger window, putting the output into a
table, and found that the timestamp is set to the window's end-time, but the
watermark seems not. (My window is [10:00-10:15), my incoming msgs both have
time-stamp of 10:00, which trigger one early output with timestamp
10:14:59.999, but the watermark stays at 10:00)
Thus, my question: what is the easiest way to check the timestamp of a
message?
Thanks and regards,
Averell
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/