time window and reduce problem

[ゞ野蠻遊戲χ]

Deal All

       I use the reduce operator after the TimeWindow operator, whether to output one element after all elements in the current window have been calculated according to reduce logic？

Thanks

Jiazhi

[Arvid Heise-3]

Hi Jiazhi,

not sure if I got the question correctly, but the reduce function will be repeatedly applied to all elements in your tumbling window until only one final aggregate per key and window remains.

So in your case, you would get the user with the max id per UserBehavior(?) for each 30 secs window.

Please note that the window is calculated based on your watermark if you use event time [1]. If you use a watermark strategy to allow late events [2], your windows are triggered later by the allowed lateness.

So if you have a tumbling window of 30 secs and an allowed lateness of 10 secs, you would get window results after

40 secs: outputs results of [0:00, 0:30)

1:10: outputs results of [0:30, 0:60)

1:40: outputs results of [1:00, 1:30)

...

Feel free to ask more questions.

[1] https://ci.apache.org/projects/flink/flink-docs-stable/dev/event_time.html

[2] https://ci.apache.org/projects/flink/flink-docs-release-1.11/dev/event_timestamp_extractors.html#fixed-amount-of-lateness

On Sat, Sep 12, 2020 at 11:53 AM ゞ野蠻遊戲χ <[hidden email]> wrote:

Deal All

       I use the reduce operator after the TimeWindow operator, whether to output one element after all elements in the current window have been calculated according to reduce logic？

Thanks

Jiazhi

--