CEP Pattern detection progressively slower

[Kostas Kloudas]

I just realized that the conversation was not sent to the Mailing List, so I am resending it.

Kostas

Begin forwarded message:

From: Kostas Kloudas <[hidden email]>

Subject: Re: CEP Pattern detection progressively slower

Date: April 11, 2017 at 11:40:25 PM GMT+2

To: Dawid Wysakowicz <[hidden email]>

Cc: vijayakumar palaniappan <[hidden email]>

Hi,

As Dawid mentioned, this issue is fixed in the current master branch and it will be part of the Flink 1.3 version which is going to be released in the begining of May. This version is going to include a lot of new capabilities in the CEP library which has been through a lot of refactoring. 

I would encourage you to use the master to experiment with CEP in Flink and let us know what you think.

Thanks a lot for trying out the CEP library,

Kostas

On Apr 11, 2017 11:42, "Dawid Wysakowicz" <[hidden email]> wrote:

Unfortunately in version 1.2.0 list of keys is not pruned. In version 1.3.0 it will be fixed with FLINK-6032.

I don't know of any possible workarounds in 1.2.0. I include Kostas, who implemented FLINK-6032. Maybe he have some idea.

2017-04-11 19:57 GMT+02:00 vijayakumar palaniappan <[hidden email]>:

While trying to use Pattern detection on KeyedStream, list of unique keys encountered, keeps accumulating in AbstractKeyedCEPPatternOperator.java. On encountering watermarks these accumulated keys are looped over to detect patterns. This causes spike in CPU usage and progressively gets slower.

Is this a known issue

Are there any workarounds?

This happens with 1.2.0 version

--