Re: Cannot see all events in window apply() for big input

Posted by Hung on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Cannot-see-all-events-in-window-apply-for-big-input-tp9945p9985.html

Hi,

Would the issue be events are too out of ordered and the watermark is global?

We want to count event per event type per day, and the data looks like:

eventA, 10-29-XX
eventB,, 11-02-XX
eventB,, 11-02-XX
eventB,, 11-03-XX
eventB,, 11-04-XX
....
....
eventA, 10-29-XX
eventA, 10-30-XX
eventA, 10-30-XX
.
.
.
eventA, 11-04-XX


eventA is much much larger than eventB,
and it looks like we lost the count of eventA at 10-29 and 10-30 while we have count of eventA at 11-04-XX.
Could it be the problem that watermark is gloabal rather than per event?

Best,

Sendoh