CEP Pattern limit

[Pedro Saraiva]

Hello,

I'm using CEP to match a stream against around 1000 different patterns.

To do this I create de patterns and then iterate and call CEP.pattern() for each. Later on, I merge the PatternStreams into one using datastream.union().

The problem is that i'm getting this exception:

AstTimeoutException: Ask timed out on Actor... after 10000ms. Sender null sent message of type LocalRpcInvocation.

I noticed that this exception is thrown when I reach around 500 patterns.

Is there a way to overcome this limit?

Kind regards,

Pedro Saraiva

[Fabian Hueske-2]

Hi Pedro,

each pattern gets translated into one or more Flink operators. Hence, your Flink program becomes *very* large and requires much more time to be deployed.

Hence, the timeout.

I'd try to limit the size your job by grouping your patterns and creating an own job for each group.

You can also increase the timeout threshold in the Flink configuration [1] but I'd still recommend to have smaller jobs.

Best,

Fabian

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.8/ops/config.html#distributed-coordination-via-akka

Am Do., 11. Juli 2019 um 15:21 Uhr schrieb Pedro Saraiva <[hidden email]>:

Hello,

I'm using CEP to match a stream against around 1000 different patterns.

To do this I create de patterns and then iterate and call CEP.pattern() for each. Later on, I merge the PatternStreams into one using datastream.union().

The problem is that i'm getting this exception:

AstTimeoutException: Ask timed out on Actor... after 10000ms. Sender null sent message of type LocalRpcInvocation.

I noticed that this exception is thrown when I reach around 500 patterns.

Is there a way to overcome this limit?

Kind regards,

Pedro Saraiva