Re: Timestamp and key preservation over operators
Posted by
Guowei Ma on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Timestamp-and-key-preservation-over-operators-tp27627p27635.html
Hi,
Most operators will preserve the input elements timestamp if it has.
Window is a special case. The timestamp of elements emitted by window is the maxTimestamp of the Window which is triggered. Different Window will have different implementation.(GlobalWindow/TimeWindow/CustomizedWindow).
Keyby just shuffle data. I think it could not affect the element's timestamp.
Hope this could help.
Best,
Hello,
I extracted timestamps using BoundedOutOfOrdernessTimestampExtractor from my
sources, have a WindowFunction, and found that my timestamps has been lost.
To do another Window operation, I need to extract timestamp again. I tried
to find a document for that but haven't found one.
Could you please help tell which type of operators would preserve records'
timestamp?
The same question for keyed stream. I have been using the same key
throughout my flow, but with many tranformations (using different operators,
including coProcessFunction, and converting my data between different
classes), and I have been trying to use
DataStreamUtils.reinterpretAsKeyedStream. Is it safe to assume that as long
as I dont do transformation on key, I could use that
reinterpretAsKeyedStream function?
Thanks and best regards,
Averell
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/