Event time, watermarks and windows

[Paul Joireman]

Hi all,

The  event timestamp and watermarks documentation (v. 1.1)  https://ci.apache.org/projects/flink/flink-docs-release-1.1/apis/streaming/event_timestamps_watermarks.html states that 

     The AssignerWithPeriodicWatermarks assigns timestamps and generates watermarks periodically (possibly depending on the stream elements, or purely     

      based on processing time).

     The interval (every n milliseconds) in which the watermark will be generated is defined via ExecutionConfig.setAutoWatermarkInterval(...). Each time, the 

     assigner’s getCurrentWatermark() method will be called, and a new Watermark will be emitted, if the returned Watermark is non-null and larger than the 

      previous Watermark.

It goes on to give an example of the BoundedOutOfOrdernessTimestampExtractor.   My question is that since the BoundedOutOfOrdernessTimestampExtractor is a sub-class of the AssignerWithPeriodicWatermarks is it necessary to call ExecutionConfig.setAutoWatermarkInterval(...) on the environment in order to actually generate watermarks or should they be generated "automatically" by the subsequent operators.

In other words, will an event time window still fire if this call (setAutoWatermarkInterval) is not present?

Regards,

Paul W Joireman

[Aljoscha Krettek]

Hi,

the call to setAutoWatermarkInterval() is still necessary to activate the watermark-generation mechanism. However, calling setStreamTimeCharacteristic(EventTime) will also set a good default value for the auto watermark interval.

Cheers,

Aljoscha 

On Mon, 24 Oct 2016 at 17:02 Paul Joireman <[hidden email]> wrote:

Hi all,

The  event timestamp and watermarks documentation (v. 1.1)  https://ci.apache.org/projects/flink/flink-docs-release-1.1/apis/streaming/event_timestamps_watermarks.html states that 

     The AssignerWithPeriodicWatermarks assigns timestamps and generates watermarks periodically (possibly depending on the stream elements, or purely     

      based on processing time).

     The interval (every n milliseconds) in which the watermark will be generated is defined via ExecutionConfig.setAutoWatermarkInterval(...). Each time, the 

     assigner’s getCurrentWatermark() method will be called, and a new Watermark will be emitted, if the returned Watermark is non-null and larger than the 

      previous Watermark.

It goes on to give an example of the BoundedOutOfOrdernessTimestampExtractor.   My question is that since the BoundedOutOfOrdernessTimestampExtractor is a sub-class of the AssignerWithPeriodicWatermarks is it necessary to call ExecutionConfig.setAutoWatermarkInterval(...) on the environment in order to actually generate watermarks or should they be generated "automatically" by the subsequent operators.

In other words, will an event time window still fire if this call (setAutoWatermarkInterval) is not present?

Regards,

Paul W Joireman