ContinuousProcessingTimeTrigger on empty

[Xiang Zhang]

Hi,

I want to have a trigger fires every 5 seconds in processing time even when no event comes. I tried

datastream.windowAll(GlobalWindows.create())
                .trigger(ContinuousProcessingTimeTrigger.of(Time.seconds(5)))
                .apply { MY_APPLY_FUNCTION}

However, ContinuousProcessingTimeTrigger only fires when there is a event. I want it fires even for a empty 5 seconds time window. Should I user ContinuousProcessingTimeTrigger?

Xiang

[Kostas Kloudas]

Hi Xiang,

Currently this is not supported by the trigger provided by Flink, as
a window with no data, is a non-existing window for Flink.

What you could do is emit periodically dummy elements from your source (so
that all windows have at least one element) and make sure that your windowing
function ignores them when computing the final result. But I am not sure
how you can do it in processing time.

Kostas

> On Jul 11, 2016, at 6:28 PM, Xiang Zhang <[hidden email]> wrote:
>
> Hi,
>
> I want to have a trigger fires every 5 seconds in processing time even when
> no event comes. I tried
>
> datastream.windowAll(GlobalWindows.create())
>
> .trigger(ContinuousProcessingTimeTrigger.of(Time.seconds(5)))
>                .apply { MY_APPLY_FUNCTION}
>
> However, ContinuousProcessingTimeTrigger only fires when there is a event. I
> want it fires even for a empty 5 seconds time window. Should I user
> ContinuousProcessingTimeTrigger?
>
> Xiang
>
>
>
> --
> View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/ContinuousProcessingTimeTrigger-on-empty-tp7914.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.

[Xiang Zhang]

Hi Kostas,

Yes, so I tried GlobalWindows. Is it possible to trigger every 5 mins on GlobalWindows? From the comments in the source for ContinuousProcessingTimeTrigger, it says:
 * A {@link Trigger} that continuously fires based on a given time interval as measured by
 * the clock of the machine on which the job is running.

So what does ContinuousProcessingTimeTrigger do in GlobalWindows?

Xiang

[Kostas Kloudas]

Hi Xiang,

According to your code, you just put all your elements (no splitting by key) into a single infinite window,
and you apply your window function every 5min (after the first element had arrived).

The combination of the two means that if you have elements arriving at steady pace
of 1 element/min, and your function just counts the already seen elements. then the
result will be 5, 10, 15 …

Under the hood, when the first element arrives the trigger registers the first time to fire after 5 min.
Then, for every firing, the trigger registers another timer to fire after 5min, and so on.

Your problem is that the first timer is set on the first element.
If you control your source, why don’t you put a dummy element in the beginning?
This will instantiate the global window and set the first timer.

Kostas

> On Jul 11, 2016, at 8:39 PM, Xiang Zhang <[hidden email]> wrote:
>
> Hi Kostas,
>
> Yes, so I tried GlobalWindows. Is it possible to trigger every 5 mins on
> GlobalWindows? From the comments in the source for
> ContinuousProcessingTimeTrigger, it says:
> * A {@link Trigger} that continuously fires based on a given time interval
> as measured by
> * the clock of the machine on which the job is running.
>
> So what does ContinuousProcessingTimeTrigger do in GlobalWindows?
>
> Xiang
>
>
>
>
> --
> View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/ContinuousProcessingTimeTrigger-on-empty-tp7914p7917.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.