FlinkCEP - Detect absence of a certain event

[Humberto Rodriguez Avila]

In the documentation of FlinkCEP, I found that I can enforce that a particular event doesn't occur between two other events using notFollowedBy or notNext.

However, I was wondering If I could detect the absence of a certain event after a time X. For example, if an event A is not followed by another event A within 10 seconds, fire an alert or do something.

Could be possible to define a FlinkCEP pattern to capture that situation?

Thanks in advance, Humberto

[Zhijiang(wangzhijiang999)]

Hi Humberto,

I guess Fuji is familiar with Flink CEP and he can answer your proposed question. I already cc him.

Best,

Zhijiang

------------------------------------------------------------------

From:Humberto Rodriguez Avila <[hidden email]>

Send Time:2020 Mar. 18 (Wed.) 17:31

To:user <[hidden email]>

Subject:FlinkCEP - Detect absence of a certain event

In the documentation of FlinkCEP, I found that I can enforce that a particular event doesn't occur between two other events using notFollowedBy or notNext.

However, I was wondering If I could detect the absence of a certain event after a time X. For example, if an event A is not followed by another event A within 10 seconds, fire an alert or do something.

Could be possible to define a FlinkCEP pattern to capture that situation?

Thanks in advance, Humberto

[Humberto Rodriguez Avila]

Hello Fuji and Zhijiang

Thanks for time! 

Fuji thanks for the links 👍🏻

I hope that in the future similar scenarios can be implemented in FlinkCEP. Other CEPs like ESPER support this particular type of negated patterns.

Best regards,

Humberto

On 19 Mar 2020, at 04:55, Shuai Xu <[hidden email]> wrote:

Hi Humberto,

Unfortunately, the answer is NO. Now FlinkCEP doesn't support notFollowedBy in the end of a Pattern, even with a time window. I have proposed an improvement to enable it. The detail can be found in https://lists.apache.org/thread.html/rc505728048663d672ad379578ac67d3219f6076986c80a2362802ebb%40%3Cdev.flink.apache.org%3E. However, I didn't receive enough responses, so this work is suspended now.

Zhijiang <[hidden email]> 于2020年3月19日周四 上午11:41写道：

Hi Humberto,

I guess Fuji is familiar with Flink CEP and he can answer your proposed question. I already cc him.

Best,

Zhijiang

------------------------------------------------------------------

From:Humberto Rodriguez Avila <[hidden email]>

Send Time:2020 Mar. 18 (Wed.) 17:31

To:user <[hidden email]>

Subject:FlinkCEP - Detect absence of a certain event

In the documentation of FlinkCEP, I found that I can enforce that a particular event doesn't occur between two other events using notFollowedBy or notNext.

However, I was wondering If I could detect the absence of a certain event after a time X. For example, if an event A is not followed by another event A within 10 seconds, fire an alert or do something.

Could be possible to define a FlinkCEP pattern to capture that situation?

Thanks in advance, Humberto

[David Anderson-2]

Humberto,

Although Flink CEP lacks notFollowedBy at the end of a Pattern, there is a way to implement this by exploiting the timeout feature. 

The Flink training includes an exercise [1] where the objective is to identify taxi rides with a START event that is not followed by an END event within two hours. You'll find a solution to this exercise that uses CEP in [2].

[1] https://training.ververica.com/exercises/longRides.html

[2] https://github.com/ververica/flink-training-exercises/blob/master/src/main/java/com/ververica/flinktraining/solutions/datastream_java/cep/LongRidesCEPSolution.java

Best,

David

On Thu, Mar 19, 2020 at 4:41 AM Zhijiang <[hidden email]> wrote:

Hi Humberto,

I guess Fuji is familiar with Flink CEP and he can answer your proposed question. I already cc him.

Best,

Zhijiang

------------------------------------------------------------------

From:Humberto Rodriguez Avila <[hidden email]>

Send Time:2020 Mar. 18 (Wed.) 17:31

To:user <[hidden email]>

Subject:FlinkCEP - Detect absence of a certain event

In the documentation of FlinkCEP, I found that I can enforce that a particular event doesn't occur between two other events using notFollowedBy or notNext.

However, I was wondering If I could detect the absence of a certain event after a time X. For example, if an event A is not followed by another event A within 10 seconds, fire an alert or do something.

Could be possible to define a FlinkCEP pattern to capture that situation?

Thanks in advance, Humberto

[Humberto Rodriguez Avila]

Thanks David, I will look at your references 👍🏻

On 19 Mar 2020, at 09:51, David Anderson <[hidden email]> wrote:

Humberto,

Although Flink CEP lacks notFollowedBy at the end of a Pattern, there is a way to implement this by exploiting the timeout feature. 

The Flink training includes an exercise [1] where the objective is to identify taxi rides with a START event that is not followed by an END event within two hours. You'll find a solution to this exercise that uses CEP in [2].

[1] https://training.ververica.com/exercises/longRides.html

[2] https://github.com/ververica/flink-training-exercises/blob/master/src/main/java/com/ververica/flinktraining/solutions/datastream_java/cep/LongRidesCEPSolution.java

Best,

David

On Thu, Mar 19, 2020 at 4:41 AM Zhijiang <[hidden email]> wrote:

Hi Humberto,

I guess Fuji is familiar with Flink CEP and he can answer your proposed question. I already cc him.

Best,

Zhijiang

------------------------------------------------------------------

From:Humberto Rodriguez Avila <[hidden email]>

Send Time:2020 Mar. 18 (Wed.) 17:31

To:user <[hidden email]>

Subject:FlinkCEP - Detect absence of a certain event

In the documentation of FlinkCEP, I found that I can enforce that a particular event doesn't occur between two other events using notFollowedBy or notNext.

However, I was wondering If I could detect the absence of a certain event after a time X. For example, if an event A is not followed by another event A within 10 seconds, fire an alert or do something.

Could be possible to define a FlinkCEP pattern to capture that situation?

Thanks in advance, Humberto

[Humberto Rodriguez Avila]

Thanks David, I will look at your references 👍🏻

On 19 Mar 2020, at 09:51, David Anderson <[hidden email]> wrote:

Humberto,

Although Flink CEP lacks notFollowedBy at the end of a Pattern, there is a way to implement this by exploiting the timeout feature. 

The Flink training includes an exercise [1] where the objective is to identify taxi rides with a START event that is not followed by an END event within two hours. You'll find a solution to this exercise that uses CEP in [2].

[1] https://training.ververica.com/exercises/longRides.html

[2] https://github.com/ververica/flink-training-exercises/blob/master/src/main/java/com/ververica/flinktraining/solutions/datastream_java/cep/LongRidesCEPSolution.java

Best,

David

On Thu, Mar 19, 2020 at 4:41 AM Zhijiang <[hidden email]> wrote:

Hi Humberto,

I guess Fuji is familiar with Flink CEP and he can answer your proposed question. I already cc him.

Best,

Zhijiang

------------------------------------------------------------------

From:Humberto Rodriguez Avila <[hidden email]>

Send Time:2020 Mar. 18 (Wed.) 17:31

To:user <[hidden email]>

Subject:FlinkCEP - Detect absence of a certain event

In the documentation of FlinkCEP, I found that I can enforce that a particular event doesn't occur between two other events using notFollowedBy or notNext.

However, I was wondering If I could detect the absence of a certain event after a time X. For example, if an event A is not followed by another event A within 10 seconds, fire an alert or do something.

Could be possible to define a FlinkCEP pattern to capture that situation?

Thanks in advance, Humberto