when the second window fire after a window operator

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

when the second window fire after a window operator

LiZhe
I have some operators like this:

(1) val DS1 = source.keyBy(_.code).window(TumblingEventTimeWindows.of(Time.minutes(1))).apply(..);

DS1.sink(...)

(2) val DS2 = DS1.flatMap(...);
(3) val DS3 = DS2.keyBy(_.id).window(TumblingEventTimeWindows.of(Time.minutes(1))).apply(...).

I want to know when the 2nd window described in (3) fire?

For example, I did a stats every 1 min on "code", and then, did a flatMap() function, transfer DS1 to DS2, the field "code" and "id" is a many-to-many relationship, then, I want to used another window for "id" to do another stats every 1 min.

I know the watermark has passed the end of window after step (1), so when the 2nd window fire and what is the internal fire mechanism for this situation?

Thanks.