Trying to comprehend rolling windows + event time

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Trying to comprehend rolling windows + event time

nsengupta
Hello lofifnc <[hidden email]>

I am keen to hear more about this particular thread of discussion. However, just a silly question: in the first case, why do you say that 'Each 5 times, as expected'! What causes them to appear 5 times? I don't see any _repeat()_ or _repeatAll()_ in the gist you have shared. 

What am I missing? 

--- Nirmalya

--
Software Technologist
http://www.linkedin.com/in/nirmalyasengupta
"If you have built castles in the air, your work need not be lost. That is where they should be.
Now put the foundation under them."
Reply | Threaded
Open this post in threaded view
|

Re: Trying to comprehend rolling windows + event time

lofifnc
Hi Nirmalya,

The aggregates will be printed 5 times because I have a rolling window with the length of 5 minutes, which will shift 1 minute forward after each evaluation. Because my input is within the time interval of 0 to 1 minute, it is perfectly aligned with the windows and will fit completely into 5 different windows, thus will be evaluated 5 times.

I have created another gist, where the start and end time of each window is shown along with it's content.
The single record I'm emitting into the data flow will fit into 5 different overlapping windows.  
https://gist.github.com/lofifnc/36861d8f37b166cc4863

If you've haven't already read them, I can really recommend the two articles, Tyler Akidau has published on o'reilly radar, they have some really great explanations and visualisations in them:
https://www.oreilly.com/people/09f01-tyler-akidau

Best,
Alex