Count sliding window does not work as expected

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Count sliding window does not work as expected

Soheil Pourbafrani
Hi,

I need some sliding windowing strategy that fills the window with the count of 400 and for every 100 incoming data, process the last 400 data. For example, suppose we have a data stream of count 160000. For count window of 400 and sliding of 100, I expect it output 1597 stream:

160000 - 400 = 159600    // the first output
159600 / 100 = 1596        //  sliding every 100 input data
 overall output = 1596 + 1 = 1597

But calling the Flink built-in count window, count (400, 100), on the keyed stream, resulted in some strange behavior! Actually, my process is to merge windowed streams. I expected every output to have 400 merged streams in it, but in the Flink output, it contains 100 to 1000 merged streams!

Can someone elaborate on how Sliding count window works in Flink?