Re: Flink CEP not emitting timed out events properly

Posted by Biplob Biswas on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Flink-CEP-not-emitting-timed-out-events-properly-tp13794p13999.html

Hi Kostas,

I ended up setting my
currentMaxTimestamp = Math.max(timestamp, currentMaxTimestamp);
to
currentMaxTimestamp = Math.min(timestamp, currentMaxTimestamp);

and  changing this :
if(firstEventFlag && (currentTime - systemTimeSinceLastModification > 10000)){
      systemTimeSinceLastModification = currentTime;
      currentMaxTimestamp = currentMaxTimestamp + 1000;
      //log.info("Current Max Time - {}, Last Modification Time - {}", currentMaxTimestamp, systemTimeSinceLastModification );
    }

to

if(firstEventFlag && (currentTime - systemTimeSinceLastModification > 20000)){
      systemTimeSinceLastModification = currentTime;
      currentMaxTimestamp = currentMaxTimestamp + 10000;
    }

it is working fine now such that I have all the events ... with timeout and with matches. Although I am afraid, this might not be the best way to do things(I am still investigating what I can do and change) because this change from max to min can lead to changes in the watermark which is not just ascending but descending as well (that's what I think can happen when an event with lower timestamp than the current watermark arrives) .... and from whatever I have read so far, watermarks should always move forward.... I haven't had any such behaviour so far but if it happens what should I expect? My job blowing up or some undefined behaviour?

Any inputs would be helpful.

BR,
Biplob