Custom trigger to trigger for late events

[Poornapragna Ts]

Hi,

I have a simple requirement where i want to have 10 second window with allow late events upto 1 hour.

Existing TumblingEventTimeWindows with EventTimeTrigger will work for this.

But the EventTimeTrigger, triggers for every incoming event after watermark has passed windows max time. I don't want this behaviour. Even for late events, I want to fire for every 10 seconds.

For this, I thought of writing custom trigger, which will be similar to EventTimeTrigger, but instead of firing on every late event, it will register timer in onElement method for upcoming 10th second.

With this setup, I have some questions.

1) When we register timers to context, is it compulsory to delete them on clear() call?

2) Will these triggers be stored in fault tolerance state? So that deleting is must.

3) Will it be problematic, if I call delete trigger for unregistered time( i.e., if I call delete for time T1 for which I had not registered before.)

4) Without implementing custom trigger, can it be achieved?

5) Lets say, late event came at 255 second so I will register a timer to trigger at 260(next 10th second). If a failure happens before that time, then restarting from the checkpoint, Will it trigger when watermark reaches 260? That means will the trigger be recovered when we restart from failure.

Thanks,

Poornapragna T S

[Congxian Qiu]

Hi Poornapragna

I'll try to answer your questions

1. you don't need to delete the timer manually(it will be deleted after fired), but you can delete the timers manually if you want.

2. AFAIK, triggers  would not be snapshot out, but the timers will be snapshot out

Poornapragna Ts <[hidden email]> 于2020年5月29日周五 上午12:38写道：

Hi,

I have a simple requirement where i want to have 10 second window with allow late events upto 1 hour.

Existing TumblingEventTimeWindows with EventTimeTrigger will work for this.

But the EventTimeTrigger, triggers for every incoming event after watermark has passed windows max time. I don't want this behaviour. Even for late events, I want to fire for every 10 seconds.

For this, I thought of writing custom trigger, which will be similar to EventTimeTrigger, but instead of firing on every late event, it will register timer in onElement method for upcoming 10th second.

With this setup, I have some questions.

1) When we register timers to context, is it compulsory to delete them on clear() call?

2) Will these triggers be stored in fault tolerance state? So that deleting is must.

3) Will it be problematic, if I call delete trigger for unregistered time( i.e., if I call delete for time T1 for which I had not registered before.)

4) Without implementing custom trigger, can it be achieved?

5) Lets say, late event came at 255 second so I will register a timer to trigger at 260(next 10th second). If a failure happens before that time, then restarting from the checkpoint, Will it trigger when watermark reaches 260? That means will the trigger be recovered when we restart from failure.

Thanks,

Poornapragna T S