Flink CEP dynamic patterns

[Abdallah Ghdiri]

As suggested by Matthias i am going to post my inquiry to the main thread. Can you please take a look at my question over at stack overflow and see if you have an answer its a quite important aspect of an ongoing project http://stackoverflow.com/questions/40935714/is-it-possible-to-add-new-patterns-in-flink-cep-after-calling-execute

[Till Rohrmann-2]

Hi Abdallah,

I've answered your question on SO. For the sake of completeness here is a copy:

At the moment Flink's CEP library does not support dynamic pattern changes out of the box. Thus, once you've defined your pattern and started your job, it will only process this defined pattern.

However, you can write your own operator implementing the TwoInputStreamOperator interface which receives on one input pattern definitions and on the other input the stream records (similar to a CoFlatMap function). For every new pattern you would then have to compile a new NFA on the operator and feed any new incoming stream elements to this NFA as well. That way, you could achieve your intended behaviour.

In the future, we will most likely add this feature to Flink's CEP library.

Cheers,

Till

On Sun, Dec 4, 2016 at 11:47 PM, Abdallah Ghdiri <[hidden email]> wrote:

As suggested by Matthias i am going to post my inquiry to the main thread. Can you please take a look at my question over at stack overflow and see if you have an answer its a quite important aspect of an ongoing project http://stackoverflow.com/questions/40935714/is-it-possible-to-add-new-patterns-in-flink-cep-after-calling-execute

[Abdallah Ghdiri]

thank you, i will investigate further

On Mon, Dec 5, 2016 at 10:36 AM, Till Rohrmann <[hidden email]> wrote:

Hi Abdallah,

I've answered your question on SO. For the sake of completeness here is a copy:

At the moment Flink's CEP library does not support dynamic pattern changes out of the box. Thus, once you've defined your pattern and started your job, it will only process this defined pattern.

However, you can write your own operator implementing the TwoInputStreamOperator interface which receives on one input pattern definitions and on the other input the stream records (similar to a CoFlatMap function). For every new pattern you would then have to compile a new NFA on the operator and feed any new incoming stream elements to this NFA as well. That way, you could achieve your intended behaviour.

In the future, we will most likely add this feature to Flink's CEP library.

Cheers,

Till

On Sun, Dec 4, 2016 at 11:47 PM, Abdallah Ghdiri <[hidden email]> wrote:

As suggested by Matthias i am going to post my inquiry to the main thread. Can you please take a look at my question over at stack overflow and see if you have an answer its a quite important aspect of an ongoing project http://stackoverflow.com/questions/40935714/is-it-possible-to-add-new-patterns-in-flink-cep-after-calling-execute