CEP timeout occurs even for a successful match when using followedBy

[Moiz Jinia]

When using "next", this pattern works fine for the both a match as well as a timeout:

Pattern<Event, Event> pattern = Pattern.<Event>begin("start")

        .where(evt -> evt.value.equals("ni"))

        .next("last").where(evt -> evt.value.equals("ar")).within(Time.seconds(5));

1. "ni" then "ar" within 5 seconds - triggers match
2. "ni" then no "ar" within 5 seconds - triggers timeout

But with "followedBy", this does not behave as expected:

Pattern<Event, Event> pattern = Pattern.<Event>begin("start")

        .where(evt -> evt.value.equals("ni"))

        .followedBy("last").where(evt -> evt.value.equals("ar")).within(Time.seconds(5));

"ni" then "ar" within 5 seconds - triggers match and also triggers timeout.

Why is the timeout triggered when using followedBy (when there is a match)?

Version - 1.1.5.

[Dawid Wysakowicz]

Hi,

This is an expected behaviour. After the "ar" event there still may occur other "ar" event that will also trigger a match.

To be more generic in all versions prior to 1.3.0 there are two different consuming strategies:

• STRICT (the next operator) - that accepts only if the event occurs directly after the previous 
  
• SKIP TILL ANY (the followedBy operator) - it accepts any matching event following event if there were already an event that matched this pattern

Because after "ni" event we could match with some other "ar" events, the match is timeouted after 5 seconds.

In FLINK-6208 we introduced third consuming strategy:

• SKIP TILL NEXT(this is the strategy for followedBy right now) - the event does not have to occur directly after the previous one but only one event can be matched

and you can still use SKIP TILL ANY by using followedByAny. I believe the SKIP TILL NEXT strategy is the one you expected. 

You can check it on master branch. We did introduce lots of new features and bugfixes to CEP for 1.3.0 version so any comments,

tests or suggestions are welcome.

2017-04-29 12:14 GMT+02:00 Moiz S Jinia <[hidden email]>:

When using "next", this pattern works fine for the both a match as well as a timeout:

Pattern<Event, Event> pattern = Pattern.<Event>begin("start")

        .where(evt -> evt.value.equals("ni"))

        .next("last").where(evt -> evt.value.equals("ar")).within(Time.seconds(5));

1. "ni" then "ar" within 5 seconds - triggers match
2. "ni" then no "ar" within 5 seconds - triggers timeout

But with "followedBy", this does not behave as expected:

Pattern<Event, Event> pattern = Pattern.<Event>begin("start")

        .where(evt -> evt.value.equals("ni"))

        .followedBy("last").where(evt -> evt.value.equals("ar")).within(Time.seconds(5));

"ni" then "ar" within 5 seconds - triggers match and also triggers timeout.

Why is the timeout triggered when using followedBy (when there is a match)?

Version - 1.1.5.

[Moiz Jinia]

Thanks Dawid! 
Yes thats what i was expecting. I'll give it a try.

When do you expect 1.3.0 stable to be out?

Moiz

On Sat, Apr 29, 2017 at 9:20 PM, Dawid Wysakowicz <[hidden email]> wrote:

Hi,

This is an expected behaviour. After the "ar" event there still may occur other "ar" event that will also trigger a match.

To be more generic in all versions prior to 1.3.0 there are two different consuming strategies:

• STRICT (the next operator) - that accepts only if the event occurs directly after the previous 
  
• SKIP TILL ANY (the followedBy operator) - it accepts any matching event following event if there were already an event that matched this pattern

Because after "ni" event we could match with some other "ar" events, the match is timeouted after 5 seconds.

In FLINK-6208 we introduced third consuming strategy:

• SKIP TILL NEXT(this is the strategy for followedBy right now) - the event does not have to occur directly after the previous one but only one event can be matched

and you can still use SKIP TILL ANY by using followedByAny. I believe the SKIP TILL NEXT strategy is the one you expected. 

You can check it on master branch. We did introduce lots of new features and bugfixes to CEP for 1.3.0 version so any comments,

tests or suggestions are welcome.

2017-04-29 12:14 GMT+02:00 Moiz S Jinia <[hidden email]>:

When using "next", this pattern works fine for the both a match as well as a timeout:

Pattern<Event, Event> pattern = Pattern.<Event>begin("start")

        .where(evt -> evt.value.equals("ni"))

        .next("last").where(evt -> evt.value.equals("ar")).within(Time.seconds(5));

1. "ni" then "ar" within 5 seconds - triggers match
2. "ni" then no "ar" within 5 seconds - triggers timeout

But with "followedBy", this does not behave as expected:

Pattern<Event, Event> pattern = Pattern.<Event>begin("start")

        .where(evt -> evt.value.equals("ni"))

        .followedBy("last").where(evt -> evt.value.equals("ar")).within(Time.seconds(5));

"ni" then "ar" within 5 seconds - triggers match and also triggers timeout.

Why is the timeout triggered when using followedBy (when there is a match)?

Version - 1.1.5.

[Kostas Kloudas]

The 1.3 is scheduled for the beginning of June.

Cheers,

Kostas

On Apr 29, 2017, at 6:16 PM, Moiz S Jinia <[hidden email]> wrote:

Thanks Dawid! 
Yes thats what i was expecting. I'll give it a try.

When do you expect 1.3.0 stable to be out?

Moiz

On Sat, Apr 29, 2017 at 9:20 PM, Dawid Wysakowicz <[hidden email]> wrote:

Hi,

This is an expected behaviour. After the "ar" event there still may occur other "ar" event that will also trigger a match.

To be more generic in all versions prior to 1.3.0 there are two different consuming strategies:

• STRICT (the next operator) - that accepts only if the event occurs directly after the previous 
  
• SKIP TILL ANY (the followedBy operator) - it accepts any matching event following event if there were already an event that matched this pattern

Because after "ni" event we could match with some other "ar" events, the match is timeouted after 5 seconds.

In FLINK-6208 we introduced third consuming strategy:

• SKIP TILL NEXT(this is the strategy for followedBy right now) - the event does not have to occur directly after the previous one but only one event can be matched

and you can still use SKIP TILL ANY by using followedByAny. I believe the SKIP TILL NEXT strategy is the one you expected. 

You can check it on master branch. We did introduce lots of new features and bugfixes to CEP for 1.3.0 version so any comments,

tests or suggestions are welcome.

2017-04-29 12:14 GMT+02:00 Moiz S Jinia <[hidden email]>:

When using "next", this pattern works fine for the both a match as well as a timeout:

Pattern<Event, Event> pattern = Pattern.<Event>begin("start")

        .where(evt -> evt.value.equals("ni"))

        .next("last").where(evt -> evt.value.equals("ar")).within(Time.seconds(5));

1. "ni" then "ar" within 5 seconds - triggers match
2. "ni" then no "ar" within 5 seconds - triggers timeout

But with "followedBy", this does not behave as expected:

Pattern<Event, Event> pattern = Pattern.<Event>begin("start")

        .where(evt -> evt.value.equals("ni"))

        .followedBy("last").where(evt -> evt.value.equals("ar")).within(Time.seconds(5));

"ni" then "ar" within 5 seconds - triggers match and also triggers timeout.

Why is the timeout triggered when using followedBy (when there is a match)?

Version - 1.1.5.

[Moiz Jinia]

Oh ok thats a bit far off. Is there any chance of a backport of https://issues.apache.org/jira/browse/FLINK-6208 to the 1.2 branch? I require the SKIP_TILL_NEXT behaviour for a production use case that we want to use Flink for.

Moiz

On Sat, Apr 29, 2017 at 9:49 PM, Kostas Kloudas <[hidden email]> wrote:

The 1.3 is scheduled for the beginning of June.

Cheers,

Kostas

On Apr 29, 2017, at 6:16 PM, Moiz S Jinia <[hidden email]> wrote:

Thanks Dawid! 
Yes thats what i was expecting. I'll give it a try.

When do you expect 1.3.0 stable to be out?

Moiz

On Sat, Apr 29, 2017 at 9:20 PM, Dawid Wysakowicz <[hidden email]> wrote:

Hi,

This is an expected behaviour. After the "ar" event there still may occur other "ar" event that will also trigger a match.

To be more generic in all versions prior to 1.3.0 there are two different consuming strategies:

• STRICT (the next operator) - that accepts only if the event occurs directly after the previous 
  
• SKIP TILL ANY (the followedBy operator) - it accepts any matching event following event if there were already an event that matched this pattern

Because after "ni" event we could match with some other "ar" events, the match is timeouted after 5 seconds.

In FLINK-6208 we introduced third consuming strategy:

• SKIP TILL NEXT(this is the strategy for followedBy right now) - the event does not have to occur directly after the previous one but only one event can be matched

and you can still use SKIP TILL ANY by using followedByAny. I believe the SKIP TILL NEXT strategy is the one you expected. 

You can check it on master branch. We did introduce lots of new features and bugfixes to CEP for 1.3.0 version so any comments,

tests or suggestions are welcome.

2017-04-29 12:14 GMT+02:00 Moiz S Jinia <[hidden email]>:

When using "next", this pattern works fine for the both a match as well as a timeout:

Pattern<Event, Event> pattern = Pattern.<Event>begin("start")

        .where(evt -> evt.value.equals("ni"))

        .next("last").where(evt -> evt.value.equals("ar")).within(Time.seconds(5));

1. "ni" then "ar" within 5 seconds - triggers match
2. "ni" then no "ar" within 5 seconds - triggers timeout

But with "followedBy", this does not behave as expected:

Pattern<Event, Event> pattern = Pattern.<Event>begin("start")

        .where(evt -> evt.value.equals("ni"))

        .followedBy("last").where(evt -> evt.value.equals("ar")).within(Time.seconds(5));

"ni" then "ar" within 5 seconds - triggers match and also triggers timeout.

Why is the timeout triggered when using followedBy (when there is a match)?

Version - 1.1.5.

[Kostas Kloudas]

Hi Moiz,

The skip-till-next is a big change and backporting it does not seem feasible. 

Also this would require more general changes to the 1.2 to make it compatible with the previous 1.2 versions.

If you want you can already use the 1.3 version by downloading the master branch and writing your 

use-case against that. The changes until the final release are going to be minor hopefully and we can

always help you adjust your program accordingly.

Hope this helps,

Kostas

On Apr 29, 2017, at 6:23 PM, Moiz S Jinia <[hidden email]> wrote:

Oh ok thats a bit far off. Is there any chance of a backport of https://issues.apache.org/jira/browse/FLINK-6208 to the 1.2 branch? I require the SKIP_TILL_NEXT behaviour for a production use case that we want to use Flink for.

Moiz

On Sat, Apr 29, 2017 at 9:49 PM, Kostas Kloudas <[hidden email]> wrote:

The 1.3 is scheduled for the beginning of June.

Cheers,

Kostas

On Apr 29, 2017, at 6:16 PM, Moiz S Jinia <[hidden email]> wrote:

Thanks Dawid! 
Yes thats what i was expecting. I'll give it a try.

When do you expect 1.3.0 stable to be out?

Moiz

On Sat, Apr 29, 2017 at 9:20 PM, Dawid Wysakowicz <[hidden email]> wrote:

Hi,

This is an expected behaviour. After the "ar" event there still may occur other "ar" event that will also trigger a match.

To be more generic in all versions prior to 1.3.0 there are two different consuming strategies:

• STRICT (the next operator) - that accepts only if the event occurs directly after the previous 
  
• SKIP TILL ANY (the followedBy operator) - it accepts any matching event following event if there were already an event that matched this pattern

Because after "ni" event we could match with some other "ar" events, the match is timeouted after 5 seconds.

In FLINK-6208 we introduced third consuming strategy:

• SKIP TILL NEXT(this is the strategy for followedBy right now) - the event does not have to occur directly after the previous one but only one event can be matched

and you can still use SKIP TILL ANY by using followedByAny. I believe the SKIP TILL NEXT strategy is the one you expected. 

You can check it on master branch. We did introduce lots of new features and bugfixes to CEP for 1.3.0 version so any comments,

tests or suggestions are welcome.

2017-04-29 12:14 GMT+02:00 Moiz S Jinia <[hidden email]>:

When using "next", this pattern works fine for the both a match as well as a timeout:

Pattern<Event, Event> pattern = Pattern.<Event>begin("start")

        .where(evt -> evt.value.equals("ni"))

        .next("last").where(evt -> evt.value.equals("ar")).within(Time.seconds(5));

1. "ni" then "ar" within 5 seconds - triggers match
2. "ni" then no "ar" within 5 seconds - triggers timeout

But with "followedBy", this does not behave as expected:

Pattern<Event, Event> pattern = Pattern.<Event>begin("start")

        .where(evt -> evt.value.equals("ni"))

        .followedBy("last").where(evt -> evt.value.equals("ar")).within(Time.seconds(5));

"ni" then "ar" within 5 seconds - triggers match and also triggers timeout.

Why is the timeout triggered when using followedBy (when there is a match)?

Version - 1.1.5.

[Moiz Jinia]

Ok I'll try that. Its just that I'd rather use a stable version.
Are there any instructions for building binaries from latest sources?

Moiz

On Sat, Apr 29, 2017 at 10:09 PM, Kostas Kloudas <[hidden email]> wrote:

Hi Moiz,

The skip-till-next is a big change and backporting it does not seem feasible. 

Also this would require more general changes to the 1.2 to make it compatible with the previous 1.2 versions.

If you want you can already use the 1.3 version by downloading the master branch and writing your 

use-case against that. The changes until the final release are going to be minor hopefully and we can

always help you adjust your program accordingly.

Hope this helps,

Kostas

On Apr 29, 2017, at 6:23 PM, Moiz S Jinia <[hidden email]> wrote:

Oh ok thats a bit far off. Is there any chance of a backport of https://issues.apache.org/jira/browse/FLINK-6208 to the 1.2 branch? I require the SKIP_TILL_NEXT behaviour for a production use case that we want to use Flink for.

Moiz

On Sat, Apr 29, 2017 at 9:49 PM, Kostas Kloudas <[hidden email]> wrote:

The 1.3 is scheduled for the beginning of June.

Cheers,

Kostas

On Apr 29, 2017, at 6:16 PM, Moiz S Jinia <[hidden email]> wrote:

Thanks Dawid! 
Yes thats what i was expecting. I'll give it a try.

When do you expect 1.3.0 stable to be out?

Moiz

On Sat, Apr 29, 2017 at 9:20 PM, Dawid Wysakowicz <[hidden email]> wrote:

Hi,

This is an expected behaviour. After the "ar" event there still may occur other "ar" event that will also trigger a match.

To be more generic in all versions prior to 1.3.0 there are two different consuming strategies:

• STRICT (the next operator) - that accepts only if the event occurs directly after the previous 
  
• SKIP TILL ANY (the followedBy operator) - it accepts any matching event following event if there were already an event that matched this pattern

Because after "ni" event we could match with some other "ar" events, the match is timeouted after 5 seconds.

In FLINK-6208 we introduced third consuming strategy:

• SKIP TILL NEXT(this is the strategy for followedBy right now) - the event does not have to occur directly after the previous one but only one event can be matched

and you can still use SKIP TILL ANY by using followedByAny. I believe the SKIP TILL NEXT strategy is the one you expected. 

You can check it on master branch. We did introduce lots of new features and bugfixes to CEP for 1.3.0 version so any comments,

tests or suggestions are welcome.

2017-04-29 12:14 GMT+02:00 Moiz S Jinia <[hidden email]>:

When using "next", this pattern works fine for the both a match as well as a timeout:

Pattern<Event, Event> pattern = Pattern.<Event>begin("start")

        .where(evt -> evt.value.equals("ni"))

        .next("last").where(evt -> evt.value.equals("ar")).within(Time.seconds(5));

1. "ni" then "ar" within 5 seconds - triggers match
2. "ni" then no "ar" within 5 seconds - triggers timeout

But with "followedBy", this does not behave as expected:

Pattern<Event, Event> pattern = Pattern.<Event>begin("start")

        .where(evt -> evt.value.equals("ni"))

        .followedBy("last").where(evt -> evt.value.equals("ar")).within(Time.seconds(5));

"ni" then "ar" within 5 seconds - triggers match and also triggers timeout.

Why is the timeout triggered when using followedBy (when there is a match)?

Version - 1.1.5.

[Moiz Jinia]

I meant maven dependencies that i can use by generating them from sources.

On Sat, Apr 29, 2017 at 10:31 PM, Moiz S Jinia <[hidden email]> wrote:

Ok I'll try that. Its just that I'd rather use a stable version.
Are there any instructions for building binaries from latest sources?

Moiz

On Sat, Apr 29, 2017 at 10:09 PM, Kostas Kloudas <[hidden email]> wrote:

Hi Moiz,

The skip-till-next is a big change and backporting it does not seem feasible. 

Also this would require more general changes to the 1.2 to make it compatible with the previous 1.2 versions.

If you want you can already use the 1.3 version by downloading the master branch and writing your 

use-case against that. The changes until the final release are going to be minor hopefully and we can

always help you adjust your program accordingly.

Hope this helps,

Kostas

On Apr 29, 2017, at 6:23 PM, Moiz S Jinia <[hidden email]> wrote:

Oh ok thats a bit far off. Is there any chance of a backport of https://issues.apache.org/jira/browse/FLINK-6208 to the 1.2 branch? I require the SKIP_TILL_NEXT behaviour for a production use case that we want to use Flink for.

Moiz

On Sat, Apr 29, 2017 at 9:49 PM, Kostas Kloudas <[hidden email]> wrote:

The 1.3 is scheduled for the beginning of June.

Cheers,

Kostas

On Apr 29, 2017, at 6:16 PM, Moiz S Jinia <[hidden email]> wrote:

Thanks Dawid! 
Yes thats what i was expecting. I'll give it a try.

When do you expect 1.3.0 stable to be out?

Moiz

On Sat, Apr 29, 2017 at 9:20 PM, Dawid Wysakowicz <[hidden email]> wrote:

Hi,

This is an expected behaviour. After the "ar" event there still may occur other "ar" event that will also trigger a match.

To be more generic in all versions prior to 1.3.0 there are two different consuming strategies:

• STRICT (the next operator) - that accepts only if the event occurs directly after the previous 
  
• SKIP TILL ANY (the followedBy operator) - it accepts any matching event following event if there were already an event that matched this pattern

Because after "ni" event we could match with some other "ar" events, the match is timeouted after 5 seconds.

In FLINK-6208 we introduced third consuming strategy:

• SKIP TILL NEXT(this is the strategy for followedBy right now) - the event does not have to occur directly after the previous one but only one event can be matched

and you can still use SKIP TILL ANY by using followedByAny. I believe the SKIP TILL NEXT strategy is the one you expected. 

You can check it on master branch. We did introduce lots of new features and bugfixes to CEP for 1.3.0 version so any comments,

tests or suggestions are welcome.

2017-04-29 12:14 GMT+02:00 Moiz S Jinia <[hidden email]>:

When using "next", this pattern works fine for the both a match as well as a timeout:

Pattern<Event, Event> pattern = Pattern.<Event>begin("start")

        .where(evt -> evt.value.equals("ni"))

        .next("last").where(evt -> evt.value.equals("ar")).within(Time.seconds(5));

1. "ni" then "ar" within 5 seconds - triggers match
2. "ni" then no "ar" within 5 seconds - triggers timeout

But with "followedBy", this does not behave as expected:

Pattern<Event, Event> pattern = Pattern.<Event>begin("start")

        .where(evt -> evt.value.equals("ni"))

        .followedBy("last").where(evt -> evt.value.equals("ar")).within(Time.seconds(5));

"ni" then "ar" within 5 seconds - triggers match and also triggers timeout.

Why is the timeout triggered when using followedBy (when there is a match)?

Version - 1.1.5.

[Kostas Kloudas]

Hi Moiz,

Here are the instructions on how to build Flink from source:

https://ci.apache.org/projects/flink/flink-docs-release-1.2/setup/building.html

Kostas

On Apr 29, 2017, at 7:15 PM, Moiz S Jinia <[hidden email]> wrote:

I meant maven dependencies that i can use by generating them from sources.

On Sat, Apr 29, 2017 at 10:31 PM, Moiz S Jinia <[hidden email]> wrote:

Ok I'll try that. Its just that I'd rather use a stable version.
Are there any instructions for building binaries from latest sources?

Moiz

On Sat, Apr 29, 2017 at 10:09 PM, Kostas Kloudas <[hidden email]> wrote:

Hi Moiz,

The skip-till-next is a big change and backporting it does not seem feasible. 

Also this would require more general changes to the 1.2 to make it compatible with the previous 1.2 versions.

If you want you can already use the 1.3 version by downloading the master branch and writing your 

use-case against that. The changes until the final release are going to be minor hopefully and we can

always help you adjust your program accordingly.

Hope this helps,

Kostas

On Apr 29, 2017, at 6:23 PM, Moiz S Jinia <[hidden email]> wrote:

Oh ok thats a bit far off. Is there any chance of a backport of https://issues.apache.org/jira/browse/FLINK-6208 to the 1.2 branch? I require the SKIP_TILL_NEXT behaviour for a production use case that we want to use Flink for.

Moiz

On Sat, Apr 29, 2017 at 9:49 PM, Kostas Kloudas <[hidden email]> wrote:

The 1.3 is scheduled for the beginning of June.

Cheers,

Kostas

On Apr 29, 2017, at 6:16 PM, Moiz S Jinia <[hidden email]> wrote:

Thanks Dawid! 
Yes thats what i was expecting. I'll give it a try.

When do you expect 1.3.0 stable to be out?

Moiz

On Sat, Apr 29, 2017 at 9:20 PM, Dawid Wysakowicz <[hidden email]> wrote:

Hi,

This is an expected behaviour. After the "ar" event there still may occur other "ar" event that will also trigger a match.

To be more generic in all versions prior to 1.3.0 there are two different consuming strategies:

• STRICT (the next operator) - that accepts only if the event occurs directly after the previous 
  
• SKIP TILL ANY (the followedBy operator) - it accepts any matching event following event if there were already an event that matched this pattern

Because after "ni" event we could match with some other "ar" events, the match is timeouted after 5 seconds.

In FLINK-6208 we introduced third consuming strategy:

• SKIP TILL NEXT(this is the strategy for followedBy right now) - the event does not have to occur directly after the previous one but only one event can be matched

and you can still use SKIP TILL ANY by using followedByAny. I believe the SKIP TILL NEXT strategy is the one you expected. 

You can check it on master branch. We did introduce lots of new features and bugfixes to CEP for 1.3.0 version so any comments,

tests or suggestions are welcome.

2017-04-29 12:14 GMT+02:00 Moiz S Jinia <[hidden email]>:

When using "next", this pattern works fine for the both a match as well as a timeout:

Pattern<Event, Event> pattern = Pattern.<Event>begin("start")

        .where(evt -> evt.value.equals("ni"))

        .next("last").where(evt -> evt.value.equals("ar")).within(Time.seconds(5));

1. "ni" then "ar" within 5 seconds - triggers match
2. "ni" then no "ar" within 5 seconds - triggers timeout

But with "followedBy", this does not behave as expected:

Pattern<Event, Event> pattern = Pattern.<Event>begin("start")

        .where(evt -> evt.value.equals("ni"))

        .followedBy("last").where(evt -> evt.value.equals("ar")).within(Time.seconds(5));

"ni" then "ar" within 5 seconds - triggers match and also triggers timeout.

Why is the timeout triggered when using followedBy (when there is a match)?

Version - 1.1.5.

[Moiz Jinia]

Thanks! I downloaded and built 1.3-SNAPSHOT locally and was able to verify that followedBy now works as I want.

Moiz

On Sat, Apr 29, 2017 at 11:08 PM, Kostas Kloudas <[hidden email]> wrote:

Hi Moiz,

Here are the instructions on how to build Flink from source:

https://ci.apache.org/projects/flink/flink-docs-release-1.2/setup/building.html

Kostas

On Apr 29, 2017, at 7:15 PM, Moiz S Jinia <[hidden email]> wrote:

I meant maven dependencies that i can use by generating them from sources.

On Sat, Apr 29, 2017 at 10:31 PM, Moiz S Jinia <[hidden email]> wrote:

Ok I'll try that. Its just that I'd rather use a stable version.
Are there any instructions for building binaries from latest sources?

Moiz

On Sat, Apr 29, 2017 at 10:09 PM, Kostas Kloudas <[hidden email]> wrote:

Hi Moiz,

The skip-till-next is a big change and backporting it does not seem feasible. 

Also this would require more general changes to the 1.2 to make it compatible with the previous 1.2 versions.

If you want you can already use the 1.3 version by downloading the master branch and writing your 

use-case against that. The changes until the final release are going to be minor hopefully and we can

always help you adjust your program accordingly.

Hope this helps,

Kostas

On Apr 29, 2017, at 6:23 PM, Moiz S Jinia <[hidden email]> wrote:

Oh ok thats a bit far off. Is there any chance of a backport of https://issues.apache.org/jira/browse/FLINK-6208 to the 1.2 branch? I require the SKIP_TILL_NEXT behaviour for a production use case that we want to use Flink for.

Moiz

On Sat, Apr 29, 2017 at 9:49 PM, Kostas Kloudas <[hidden email]> wrote:

The 1.3 is scheduled for the beginning of June.

Cheers,

Kostas

On Apr 29, 2017, at 6:16 PM, Moiz S Jinia <[hidden email]> wrote:

Thanks Dawid! 
Yes thats what i was expecting. I'll give it a try.

When do you expect 1.3.0 stable to be out?

Moiz

On Sat, Apr 29, 2017 at 9:20 PM, Dawid Wysakowicz <[hidden email]> wrote:

Hi,

This is an expected behaviour. After the "ar" event there still may occur other "ar" event that will also trigger a match.

To be more generic in all versions prior to 1.3.0 there are two different consuming strategies:

• STRICT (the next operator) - that accepts only if the event occurs directly after the previous 
  
• SKIP TILL ANY (the followedBy operator) - it accepts any matching event following event if there were already an event that matched this pattern

Because after "ni" event we could match with some other "ar" events, the match is timeouted after 5 seconds.

In FLINK-6208 we introduced third consuming strategy:

• SKIP TILL NEXT(this is the strategy for followedBy right now) - the event does not have to occur directly after the previous one but only one event can be matched

and you can still use SKIP TILL ANY by using followedByAny. I believe the SKIP TILL NEXT strategy is the one you expected. 

You can check it on master branch. We did introduce lots of new features and bugfixes to CEP for 1.3.0 version so any comments,

tests or suggestions are welcome.

2017-04-29 12:14 GMT+02:00 Moiz S Jinia <[hidden email]>:

When using "next", this pattern works fine for the both a match as well as a timeout:

Pattern<Event, Event> pattern = Pattern.<Event>begin("start")

        .where(evt -> evt.value.equals("ni"))

        .next("last").where(evt -> evt.value.equals("ar")).within(Time.seconds(5));

1. "ni" then "ar" within 5 seconds - triggers match
2. "ni" then no "ar" within 5 seconds - triggers timeout

But with "followedBy", this does not behave as expected:

Pattern<Event, Event> pattern = Pattern.<Event>begin("start")

        .where(evt -> evt.value.equals("ni"))

        .followedBy("last").where(evt -> evt.value.equals("ar")).within(Time.seconds(5));

"ni" then "ar" within 5 seconds - triggers match and also triggers timeout.

Why is the timeout triggered when using followedBy (when there is a match)?

Version - 1.1.5.

[Kostas Kloudas]

Glad to hear that Moiz!

And thanks for helping us test out the library.

Kostas

On May 2, 2017, at 12:34 PM, Moiz S Jinia <[hidden email]> wrote:

Thanks! I downloaded and built 1.3-SNAPSHOT locally and was able to verify that followedBy now works as I want.

Moiz

On Sat, Apr 29, 2017 at 11:08 PM, Kostas Kloudas <[hidden email]> wrote:

Hi Moiz,

Here are the instructions on how to build Flink from source:

https://ci.apache.org/projects/flink/flink-docs-release-1.2/setup/building.html

Kostas

On Apr 29, 2017, at 7:15 PM, Moiz S Jinia <[hidden email]> wrote:

I meant maven dependencies that i can use by generating them from sources.

On Sat, Apr 29, 2017 at 10:31 PM, Moiz S Jinia <[hidden email]> wrote:

Ok I'll try that. Its just that I'd rather use a stable version.
Are there any instructions for building binaries from latest sources?

Moiz

On Sat, Apr 29, 2017 at 10:09 PM, Kostas Kloudas <[hidden email]> wrote:

Hi Moiz,

The skip-till-next is a big change and backporting it does not seem feasible. 

Also this would require more general changes to the 1.2 to make it compatible with the previous 1.2 versions.

If you want you can already use the 1.3 version by downloading the master branch and writing your 

use-case against that. The changes until the final release are going to be minor hopefully and we can

always help you adjust your program accordingly.

Hope this helps,

Kostas

On Apr 29, 2017, at 6:23 PM, Moiz S Jinia <[hidden email]> wrote:

Oh ok thats a bit far off. Is there any chance of a backport of https://issues.apache.org/jira/browse/FLINK-6208 to the 1.2 branch? I require the SKIP_TILL_NEXT behaviour for a production use case that we want to use Flink for.

Moiz

On Sat, Apr 29, 2017 at 9:49 PM, Kostas Kloudas <[hidden email]> wrote:

The 1.3 is scheduled for the beginning of June.

Cheers,

Kostas

On Apr 29, 2017, at 6:16 PM, Moiz S Jinia <[hidden email]> wrote:

Thanks Dawid! 
Yes thats what i was expecting. I'll give it a try.

When do you expect 1.3.0 stable to be out?

Moiz

On Sat, Apr 29, 2017 at 9:20 PM, Dawid Wysakowicz <[hidden email]> wrote:

Hi,

This is an expected behaviour. After the "ar" event there still may occur other "ar" event that will also trigger a match.

To be more generic in all versions prior to 1.3.0 there are two different consuming strategies:

• STRICT (the next operator) - that accepts only if the event occurs directly after the previous 
  
• SKIP TILL ANY (the followedBy operator) - it accepts any matching event following event if there were already an event that matched this pattern

Because after "ni" event we could match with some other "ar" events, the match is timeouted after 5 seconds.

In FLINK-6208 we introduced third consuming strategy:

• SKIP TILL NEXT(this is the strategy for followedBy right now) - the event does not have to occur directly after the previous one but only one event can be matched

and you can still use SKIP TILL ANY by using followedByAny. I believe the SKIP TILL NEXT strategy is the one you expected. 

You can check it on master branch. We did introduce lots of new features and bugfixes to CEP for 1.3.0 version so any comments,

tests or suggestions are welcome.

2017-04-29 12:14 GMT+02:00 Moiz S Jinia <[hidden email]>:

When using "next", this pattern works fine for the both a match as well as a timeout:

Pattern<Event, Event> pattern = Pattern.<Event>begin("start")

        .where(evt -> evt.value.equals("ni"))

        .next("last").where(evt -> evt.value.equals("ar")).within(Time.seconds(5));

1. "ni" then "ar" within 5 seconds - triggers match
2. "ni" then no "ar" within 5 seconds - triggers timeout

But with "followedBy", this does not behave as expected:

Pattern<Event, Event> pattern = Pattern.<Event>begin("start")

        .where(evt -> evt.value.equals("ni"))

        .followedBy("last").where(evt -> evt.value.equals("ar")).within(Time.seconds(5));

"ni" then "ar" within 5 seconds - triggers match and also triggers timeout.

Why is the timeout triggered when using followedBy (when there is a match)?

Version - 1.1.5.