Window elements for certain period for delayed processing

[simpleusr]

Hi,

My ultimate requirement is to stop processing of certain events between
00:00:00 and 01:00:00 for each day (Time is in HH:mm:SS format).

I am flink newbie and I thought only option to delay elements is to collect
them in a window between 00:00:00 and 01:00:00 for each day.

TumblingEventTimeWindows for one hour seems the possible candidate but as I
understand this window will be present for all times, i.e (startTime,
startTime+1hour), (startTime+1hour, startTime+2hour), (startTime+2hour,
startTime+3hour)...

How will it be possible to start a window at 00:00:00 each day and stop the
window at 01:00:00?

Or is there any other option to keep elements in the job between 00:00:00
and 01:00:00 and then start to process them after 01:00:00?

Best regards



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[Congxian Qiu]

Hi, simpleusr

Maybe custom trigger[1] can be helpful.

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.7/dev/stream/operators/windows.html#triggers

Best, Congxian

On Feb 15, 2019, 13:15 +0800, simpleusr <[hidden email]>, wrote:

Hi,

My ultimate requirement is to stop processing of certain events between
00:00:00 and 01:00:00 for each day (Time is in HH:mm:SS format).

I am flink newbie and I thought only option to delay elements is to collect
them in a window between 00:00:00 and 01:00:00 for each day.

TumblingEventTimeWindows for one hour seems the possible candidate but as I
understand this window will be present for all times, i.e (startTime,
startTime+1hour), (startTime+1hour, startTime+2hour), (startTime+2hour,
startTime+3hour)...

How will it be possible to start a window at 00:00:00 each day and stop the
window at 01:00:00?

Or is there any other option to keep elements in the job between 00:00:00
and 01:00:00 and then start to process them after 01:00:00?

Best regards



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[Fabian Hueske-2]

Hi,

I would not use a window for that.

Implementing the logic with a ProcessFunction seems more straight-forward.

The function simply collects all events between 00:00 and 01:00 in a ListState and emits them when the time passes 01:00.

All other records are simply forwarded.

Best, Fabian

Am Fr., 15. Feb. 2019 um 07:02 Uhr schrieb Congxian Qiu <[hidden email]>:

Hi, simpleusr

Maybe custom trigger[1] can be helpful.

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.7/dev/stream/operators/windows.html#triggers

Best, Congxian

On Feb 15, 2019, 13:15 +0800, simpleusr <[hidden email]>, wrote:

Hi,

My ultimate requirement is to stop processing of certain events between
00:00:00 and 01:00:00 for each day (Time is in HH:mm:SS format).

I am flink newbie and I thought only option to delay elements is to collect
them in a window between 00:00:00 and 01:00:00 for each day.

TumblingEventTimeWindows for one hour seems the possible candidate but as I
understand this window will be present for all times, i.e (startTime,
startTime+1hour), (startTime+1hour, startTime+2hour), (startTime+2hour,
startTime+3hour)...

How will it be possible to start a window at 00:00:00 each day and stop the
window at 01:00:00?

Or is there any other option to keep elements in the job between 00:00:00
and 01:00:00 and then start to process them after 01:00:00?

Best regards



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[simpleusr]

Many Thanks Fabian

I will start to investigate ProcessFunction

Regards



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/