Flink CEP not emitting timed out events properly

Posted by Biplob Biswas on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Flink-CEP-not-emitting-timed-out-events-properly-tp13794.html

Hi,

I am having some issues with FlinkCEP again. This time I am using processing time for my CEP job where I am reading from multiple kafka topics and using the pattern API to create a rule. I am outputting both, the matched events as well as timeout events.

Now my problem is, I am sending some event over one of the topics such that subsequent events wouldn't be generated within the time specified and I expect a timed out event.

But it is not generating the timed out event even after 2 minutes (specified interval) and it's only generating the previous timed out events when I am sending an extra message over the kafka topic.

I am not sure why is this happening, for example:

2> RecordReadEventType 1483278179000 044023a4-edec-439c-b221-806740972da2 []
2> RecordReadEventType 1483278179000 044023a4-edec-439c-b221-806740972da1 []
2> RecordReadEventType 1483278179000 044023a4-edec-439c-b221-806740972da5 []
2> Anomaly Events: {first=[RecordReadEventType 1483278179000 044023a4-edec-439c-b221-806740972da2 []]} @ 1497612386342
2> Anomaly Events: {first=[RecordReadEventType 1483278179000 044023a4-edec-439c-b221-806740972da1 []]} @ 1497612386342

in the example above, the anomaly events are generated only after sending the event with event id - 044023a4-edec-439c-b221-806740972da5

and that too the anomaly event for this particular event is not generated.

I suspected that the watermark was not updated automatically for the last event and it's only updated when a new event comes in the system. So, I added the setAutoWatermarkInterval(1000) to the code but no avail.


Thanks & Regards,
Biplob Biswas