Behaviour of Windows on Shutdown

[snntr]

Hi everyone,

if a DataStream is created with .fromElements(...) all windows emit all
buffered records at the end of the stream. I have two questions about this:

1) Is this only the case for streams created with .fromElements() or
does this happen in any streaming application on shutdown?

2) Is there a configuration option to disable this behaviour, such that
buffered events remaining in windows are just discarded?

In our application it is critical, that only events, which were
explicitly fired are emitted from the windows.

Cheers and thank you,

Konstantin

[Aljoscha Krettek]

Hi,
did the first mail from me not arrive? I’m sending it again:

we changed it a while back to not emit any buffered elements at the end because we noticed that that would be a more natural behavior. This must be an oversight on our part. I’ll make sure that the 1.0 release will have the correct behavior.

> On 17 Feb 2016, at 16:18, Konstantin Knauf <[hidden email]> wrote:
>
> Hi everyone,
>
> if a DataStream is created with .fromElements(...) all windows emit all
> buffered records at the end of the stream. I have two questions about this:
>
> 1) Is this only the case for streams created with .fromElements() or
> does this happen in any streaming application on shutdown?
>
> 2) Is there a configuration option to disable this behaviour, such that
> buffered events remaining in windows are just discarded?
>
> In our application it is critical, that only events, which were
> explicitly fired are emitted from the windows.
>
> Cheers and thank you,
>
> Konstantin
>

[snntr]

Haha :) It arrived, somehow my first mail got sent twice.

On 17.02.2016 20:45, Aljoscha Krettek wrote:

> Hi,
> did the first mail from me not arrive? I’m sending it again:
>
> we changed it a while back to not emit any buffered elements at the end because we noticed that that would be a more natural behavior. This must be an oversight on our part. I’ll make sure that the 1.0 release will have the correct behavior.
>
>> On 17 Feb 2016, at 16:18, Konstantin Knauf <[hidden email]> wrote:
>>
>> Hi everyone,
>>
>> if a DataStream is created with .fromElements(...) all windows emit all
>> buffered records at the end of the stream. I have two questions about this:
>>
>> 1) Is this only the case for streams created with .fromElements() or
>> does this happen in any streaming application on shutdown?
>>
>> 2) Is there a configuration option to disable this behaviour, such that
>> buffered events remaining in windows are just discarded?
>>
>> In our application it is critical, that only events, which were
>> explicitly fired are emitted from the windows.
>>
>> Cheers and thank you,
>>
>> Konstantin
>>
>
>

--
Konstantin Knauf * [hidden email] * [phone]
TNG Technology Consulting GmbH, Betastr. 13a, 85774 Unterföhring
Geschäftsführer: Henrik Klagges, Christoph Stock, Dr. Robert Dahlke
Sitz: Unterföhring * Amtsgericht München * HRB 135082