Join two data streams on a given key and diffrent common window size.

[王凯]

hi all, flink 1.4 or 1.5 only support Join two data streams  as shown below：

but i have two data streams A and B, i want make A and B each set its own window size， not a common size；

if can do this ,i can join A (window size:1min , TumblingEventTimeWindow) and B (window size:5min，SlidingEventTimeWindow);

Why doesn't flink support this?  I found a similar example on github ( https://github.com/wangyangjun/flink-stream-join),like this：

 

 

 

[Fabian Hueske-2]

Hi,

The semantics of the joins offered by the DataStream API in Flink 1.4 and before as well as the upcoming 1.5 version are a bit messed up, IMO.

Since Flink 1.4, Flink SQL implements a better windowed join [1]. DataStream and SQL can be easily integrated with each other.
A similar implementation for the DataStream API is currently under review and will be added for Flink 1.6 [2] [3].

Best, Fabian

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/table/sql.html#joins
[2] https://github.com/apache/flink/pull/5342
[3] https://github.com/apache/flink/pull/5482

2018-04-22 10:41 GMT+02:00 lehaxing <[hidden email]>:

hi all, flink 1.4 or 1.5 only support Join two data streams  as shown below：

but i have two data streams A and B, i want make A and B each set its own window size， not a common size；

if can do this ,i can join A (window size:1min , TumblingEventTimeWindow) and B (window size:5min，SlidingEventTimeWindow);

Why doesn't flink support this?  I found a similar example on github ( https://github.com/wangyangjun/flink-stream-join),like this：

 

 

 

[王凯]

thanks a lot ,I'm sorry to recover so late，I will pay attention to this.

At 2018-04-23 19:03:11, "Fabian Hueske" <[hidden email]> wrote:

Hi,

The semantics of the joins offered by the DataStream API in Flink 1.4 and before as well as the upcoming 1.5 version are a bit messed up, IMO.

Since Flink 1.4, Flink SQL implements a better windowed join [1]. DataStream and SQL can be easily integrated with each other.
A similar implementation for the DataStream API is currently under review and will be added for Flink 1.6 [2] [3].

Best, Fabian

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/table/sql.html#joins
[2] https://github.com/apache/flink/pull/5342
[3] https://github.com/apache/flink/pull/5482

2018-04-22 10:41 GMT+02:00 lehaxing <[hidden email]>:

hi all, flink 1.4 or 1.5 only support Join two data streams  as shown below：

but i have two data streams A and B, i want make A and B each set its own window size， not a common size；

if can do this ,i can join A (window size:1min , TumblingEventTimeWindow) and B (window size:5min，SlidingEventTimeWindow);

Why doesn't flink support this?  I found a similar example on github ( https://github.com/wangyangjun/flink-stream-join),like this：