Allowed lateness + side output late data, what's included?

[Juho Autio]

If I use a non-zero value for allowedLateness and also sideOutputLateData, does the late data output contain also the events that were triggered in the bounds of allowed lateness? By looking at the docs I can't be sure which way it is.

Code example:

.timeWindow(Time.days(1))

.allowedLateness(Time.minutes(1))

.sideOutputLateData(lateDataTag)

Also I know that data that arrives more than 1 minute late is written to the late data side output.

If some data arrives 0-1 minute late, I know that the time window triggers again with that data added. Is that data also written to the late data side output or not?

Thanks!

[Chesnay Schepler]

Data that arrives within the allowed lateness should not be written to the side output.

On 11.04.2018 11:12, Juho Autio wrote:

If I use a non-zero value for allowedLateness and also sideOutputLateData, does the late data output contain also the events that were triggered in the bounds of allowed lateness? By looking at the docs I can't be sure which way it is.

Code example:

.timeWindow(Time.days(1))

.allowedLateness(Time.minutes(1))

.sideOutputLateData(lateDataTag)

Also I know that data that arrives more than 1 minute late is written to the late data side output.

If some data arrives 0-1 minute late, I know that the time window triggers again with that data added. Is that data also written to the late data side output or not?

Thanks!

[Juho Autio]

Thanks!

On Wed, Apr 11, 2018 at 12:59 PM, Chesnay Schepler <[hidden email]> wrote:

Data that arrives within the allowed lateness should not be written to the side output.

On 11.04.2018 11:12, Juho Autio wrote:

If I use a non-zero value for allowedLateness and also sideOutputLateData, does the late data output contain also the events that were triggered in the bounds of allowed lateness? By looking at the docs I can't be sure which way it is.

Code example:

.timeWindow(Time.days(1))

.allowedLateness(Time.minutes(1))

.sideOutputLateData(lateDataTag)

Also I know that data that arrives more than 1 minute late is written to the late data side output.

If some data arrives 0-1 minute late, I know that the time window triggers again with that data added. Is that data also written to the late data side output or not?

Thanks!