Re: emit partial state in window (streaming)

Posted by Luis Mariano Guerra on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/emit-partial-state-in-window-streaming-tp9759p9880.html

On Thu, Nov 3, 2016 at 7:05 PM, Kostas Kloudas <[hidden email]> wrote:

Hi Luis,

You cannot have event-time early firings on both chained window operators.
The reason is that each early result from the first window operator will have a timestamp equal to window.maxTimestamp-1.
So in the second windowing operator, they will be buffered until the watermark signaling the end of the window arrives.

so, how do I get windowAll and partial results? do I have to remove the partial calculations and do it all in one node/thread? is there another way?
 

Now for the second point, I think that what you have understood is correct.
The "ctx.registerEventTimeTimer(window.getEnd())” registers a timer to call the onEventTime().

then what else calls onEventTime? because if a register the event time timer inside of it something else calls it.

 

Cheers,
Kostas

> On Nov 3, 2016, at 3:29 PM, Luis Mariano Guerra <[hidden email]> wrote:
>
> On Thu, Nov 3, 2016 at 2:06 PM, Kostas Kloudas <[hidden email]> wrote:
> Hi Luis,
>
> Can you try to comment the whole final windowing and see if this is works?
> This includes the following lines:
>
>   .windowAll(TumblingEventTimeWindows.of(Time.of(windowTime, timeUnit)))
>   .trigger(new PartialWindowTrigger<>(partialWindowTime, timeUnit, windowTime, timeUnit))
>   .apply(creator.create(), windowAllFold, windowAllMerge);
>
>
> commenting it emits on fire, how do I make the trigger "go thorough" the windowAll, or if not possible, how can I join the substreams in one stream and respect the trigger?
>
> An additional note is that I would go for registering an event time timer at the onEventTime
> instead of checking the timestamp on the onElement(). This is because with your implementation,
> in order to fire a computation, you always have to wait for an element outside the partial window interval to arrive.
>
> then I think I understood the purpose of registering the event time timer wrong, isn't "ctx.registerEventTimeTimer(window.getEnd())" called to register a timer to call onEventTime?
>
>
> Cheers,
> Kostas
>
>> On Nov 3, 2016, at 11:31 AM, Luis Mariano Guerra <[hidden email]> wrote:
>>
>>   .windowAll(TumblingEventTimeWindows.of(Time.of(windowTime, timeUnit)))
>>                 //.trigger(new PartialWindowTrigger<>(partialWindowTime, timeUnit, windowTime, timeUnit))
>>                 .apply(creator.create(), windowAllFold, windowAllMerge);
>
>