Flink Event Time order

[Björn Zachrisson]

Hi,

According to the link https://pastebin.com/409Z0tSc

I'm trying to start building my flink application based on "taxi drives"-example.

I use a priority queue to add the events and emitting watermarks every period+1min where period normally is 30minutes. 

The thing is that the events seems to come in an chaotic order.

My expectation would be that they would come based on their original time since i emitt watermarks after each and every "item".

This is a proof of concept application.

Can anyone guide me to my errors here please?

Best Regards

[Aljoscha Krettek]

Is that linked code your actual code? I don’t see the watermark emission code or the priority queue code.

Best,

Aljoscha

On 1. May 2017, at 16:02, Björn Zachrisson <[hidden email]> wrote:

Hi,

According to the link https://pastebin.com/409Z0tSc

I'm trying to start building my flink application based on "taxi drives"-example.

I use a priority queue to add the events and emitting watermarks every period+1min where period normally is 30minutes. 

The thing is that the events seems to come in an chaotic order.

My expectation would be that they would come based on their original time since i emitt watermarks after each and every "item".

This is a proof of concept application.

Can anyone guide me to my errors here please?

Best Regards

[Aljoscha Krettek]

Ok, thanks for letting us know that you solved it!

On 4. May 2017, at 07:41, Björn Zachrisson <[hidden email]> wrote:

Hi,

No that is only a subset of the code.

I realized it was the multiple sink/parallelization that caused my "mystique behavior".

So when sorting each bucket-result for them self, the events came in correct order.

Thanks for replying! 

Best Regards

Björn 

2017-05-03 10:56 GMT+02:00 Aljoscha Krettek <[hidden email]>:

Is that linked code your actual code? I don’t see the watermark emission code or the priority queue code.

Best,

Aljoscha

On 1. May 2017, at 16:02, Björn Zachrisson <[hidden email]> wrote:

Hi,

According to the link https://pastebin.com/409Z0tSc

I'm trying to start building my flink application based on "taxi drives"-example.

I use a priority queue to add the events and emitting watermarks every period+1min where period normally is 30minutes. 

The thing is that the events seems to come in an chaotic order.

My expectation would be that they would come based on their original time since i emitt watermarks after each and every "item".

This is a proof of concept application.

Can anyone guide me to my errors here please?

Best Regards