Hi, There is possibly an issue with EventTimeSessionWindows where a gap is specified for considering items in the same session. Here the logic is, if two adjacent items have a difference in event timestamps of more than the gap then the items are considered to be in separate session. The issue is, what happens if the gap between streaming records is never (or for a very long time) less than the session gap. This is likely to lead to a race condition. Is this a bug? How do we deal with this to process windows in finite time? Please could you suggest. Thanks.

[Sujit Sakre]

Hi,

We are using Flink 1.1.4 version.

There is possibly an issue with EventTimeSessionWindows where a gap is specified for considering items in the same session. Here the logic is, if two adjacent items have a difference in event timestamps of more than the gap then the items are considered to be in separate session. The issue is, what happens if the gap between streaming records is never  (or for a very long time) less than the session gap. This is likely to lead to a race condition.

Is this a bug? How do we deal with this to process windows in finite time?

Please could you suggest.

Thanks.

This email is sent on behalf of Northgate Public Services (UK) Limited and its associated companies including Rave Technologies (India) Pvt Limited (together "Northgate Public Services") and is strictly confidential and intended solely for the addressee(s). 

If you are not the intended recipient of this email you must: (i) not disclose, copy or distribute its contents to any other person nor use its contents in any way or you may be acting unlawfully;  (ii) contact Northgate Public Services immediately on [phone] quoting the name of the sender and the addressee then delete it from your system.

Northgate Public Services has taken reasonable precautions to ensure that no viruses are contained in this email, but does not accept any responsibility once this email has been transmitted.  You should scan attachments (if any) for viruses.

Northgate Public Services (UK) Limited, registered in England and Wales under number 00968498 with a registered address of Peoplebuilding 2, Peoplebuilding Estate, Maylands Avenue, Hemel Hempstead, Hertfordshire, HP2 4NN.  Rave Technologies (India) Pvt Limited, registered in India under number 117068 with a registered address of 2nd Floor, Ballard House, Adi Marzban Marg, Ballard Estate, Mumbai, Maharashtra, India, 400001.

[Jamie Grier]

If there is never a gap between elements larger than the session gap -- the window never ending would be the correct behavior.  So, if this is the case with some data stream I would not suggest to use session windows at all -- or I would use a smaller session gap.

Another alternative would be to use Session Windows along with a user-defined trigger that fires periodically whether the session has ended or not.  For example, if in the normal case the session window logic works well but sometimes you want to force an eval in case a "natural" session is too long.

-Jamie

On Mon, Jan 2, 2017 at 3:11 AM, Sujit Sakre <[hidden email]> wrote:

Hi,

We are using Flink 1.1.4 version.

There is possibly an issue with EventTimeSessionWindows where a gap is specified for considering items in the same session. Here the logic is, if two adjacent items have a difference in event timestamps of more than the gap then the items are considered to be in separate session. The issue is, what happens if the gap between streaming records is never  (or for a very long time) less than the session gap. This is likely to lead to a race condition.

Is this a bug? How do we deal with this to process windows in finite time?

Please could you suggest.

Thanks.

This email is sent on behalf of Northgate Public Services (UK) Limited and its associated companies including Rave Technologies (India) Pvt Limited (together "Northgate Public Services") and is strictly confidential and intended solely for the addressee(s). 

If you are not the intended recipient of this email you must: (i) not disclose, copy or distribute its contents to any other person nor use its contents in any way or you may be acting unlawfully;  (ii) contact Northgate Public Services immediately on <a href="tel:+44%201908%20264500" value="[phone]" target="_blank">[phone] quoting the name of the sender and the addressee then delete it from your system.

Northgate Public Services has taken reasonable precautions to ensure that no viruses are contained in this email, but does not accept any responsibility once this email has been transmitted.  You should scan attachments (if any) for viruses.

Northgate Public Services (UK) Limited, registered in England and Wales under number 00968498 with a registered address of Peoplebuilding 2, Peoplebuilding Estate, Maylands Avenue, Hemel Hempstead, Hertfordshire, HP2 4NN.  Rave Technologies (India) Pvt Limited, registered in India under number 117068 with a registered address of 2nd Floor, Ballard House, Adi Marzban Marg, Ballard Estate, Mumbai, Maharashtra, India, 400001.

--