TumblingEventTimeWindows with time characteristic set to 'ProcessingTime'

[Rami Al-Isawi]

Hi,

Why this combination is not possible? even though I am setting "assignTimestampsAndWatermarks

“ correctly on the DataStream.

I would like Flink to be ticking on processing time, but also utilize the TumblingEventTimeWindows which is based on event time.

It is not possible because of :

java.lang.RuntimeException: Record has Long.MIN_VALUE timestamp (= no timestamp marker).

Regards,

-Rami

Disclaimer: This message and any attachments thereto are intended solely for the addressed recipient(s) and may contain confidential information. If you are not the intended recipient, please notify the sender by reply e-mail and delete the e-mail (including any attachments thereto) without producing, distributing or retaining any copies thereof. Any review, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient(s) is prohibited. Thank you.

[Till Rohrmann]

Hi Rami,

have you set the event time characteristic to event time with env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime);? Otherwise the event time windows won’t work.

Cheers,
Till

​

On Tue, Aug 16, 2016 at 2:42 PM, Al-Isawi Rami <[hidden email]> wrote:

Hi,

Why this combination is not possible? even though I am setting "assignTimestampsAndWatermarks

“ correctly on the DataStream.

I would like Flink to be ticking on processing time, but also utilize the TumblingEventTimeWindows which is based on event time.

It is not possible because of :

java.lang.RuntimeException: Record has Long.MIN_VALUE timestamp (= no timestamp marker).

Regards,

-Rami

Disclaimer: This message and any attachments thereto are intended solely for the addressed recipient(s) and may contain confidential information. If you are not the intended recipient, please notify the sender by reply e-mail and delete the e-mail (including any attachments thereto) without producing, distributing or retaining any copies thereof. Any review, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient(s) is prohibited. Thank you.

[Rami Al-Isawi]

Hi Till,

Yes, I understand that. I am just asking why. If I am assigning the timestamps. why can’t flink deal with the window based on the event time? i.e  TumblingEventTimeWindows. Why should I set the whole env to be ticking on event time?

It is just that I have some windows that needs to be processing time, and others needs to be event time.

Regard,

-Rami

On 17 Aug 2016, at 12:03, Till Rohrmann <[hidden email]> wrote:

Hi Rami,

have you set the event time characteristic to event time with env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime);? Otherwise the event time windows won’t work.

Cheers,
Till

​

On Tue, Aug 16, 2016 at 2:42 PM, Al-Isawi Rami <[hidden email]> wrote:

Hi,

Why this combination is not possible? even though I am setting "assignTimestampsAndWatermarks

“ correctly on the DataStream.

I would like Flink to be ticking on processing time, but also utilize the TumblingEventTimeWindows which is based on event time.

It is not possible because of :

java.lang.RuntimeException: Record has Long.MIN_VALUE timestamp (= no timestamp marker).

Regards,

-Rami

Disclaimer: This message and any attachments thereto are intended solely for the addressed recipient(s) and may contain confidential information. If you are not the intended recipient, please notify the sender by reply e-mail and delete the e-mail (including any attachments thereto) without producing, distributing or retaining any copies thereof. Any review, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient(s) is prohibited. Thank you.

Disclaimer: This message and any attachments thereto are intended solely for the addressed recipient(s) and may contain confidential information. If you are not the intended recipient, please notify the sender by reply e-mail and delete the e-mail (including any attachments thereto) without producing, distributing or retaining any copies thereof. Any review, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient(s) is prohibited. Thank you.

[Stephan Ewen]

You can still mix processing time and event time. You only need to set the base environment to event time in order to activate the timestamp/watermark transport.

On Wed, Aug 17, 2016 at 1:40 PM, Al-Isawi Rami <[hidden email]> wrote:

Hi Till,

Yes, I understand that. I am just asking why. If I am assigning the timestamps. why can’t flink deal with the window based on the event time? i.e  TumblingEventTimeWindows. Why should I set the whole env to be ticking on event time?

It is just that I have some windows that needs to be processing time, and others needs to be event time.

Regard,

-Rami

On 17 Aug 2016, at 12:03, Till Rohrmann <[hidden email]> wrote:

Hi Rami,

have you set the event time characteristic to event time with env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime);? Otherwise the event time windows won’t work.

Cheers,
Till

​

On Tue, Aug 16, 2016 at 2:42 PM, Al-Isawi Rami <[hidden email]> wrote:

Hi,

Why this combination is not possible? even though I am setting "assignTimestampsAndWatermarks

“ correctly on the DataStream.

I would like Flink to be ticking on processing time, but also utilize the TumblingEventTimeWindows which is based on event time.

It is not possible because of :

java.lang.RuntimeException: Record has Long.MIN_VALUE timestamp (= no timestamp marker).

Regards,

-Rami

Disclaimer: This message and any attachments thereto are intended solely for the addressed recipient(s) and may contain confidential information. If you are not the intended recipient, please notify the sender by reply e-mail and delete the e-mail (including any attachments thereto) without producing, distributing or retaining any copies thereof. Any review, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient(s) is prohibited. Thank you.

Disclaimer: This message and any attachments thereto are intended solely for the addressed recipient(s) and may contain confidential information. If you are not the intended recipient, please notify the sender by reply e-mail and delete the e-mail (including any attachments thereto) without producing, distributing or retaining any copies thereof. Any review, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient(s) is prohibited. Thank you.