Joining Windowed Data Streams

[Philipp Goetze]

Hey community,

is there a possibility to join two windowed data streams, instead of joining two data streams on a window?

For example if one wants to implement Q1 of SRBench you load the data, create one window definition and then one would combine filters and joins on this window. But how would you do these Joins in Flink Streaming?

Best Regards,
Philipp

[Stephan Ewen]

Hi Philipp!

I don't think this currently works. We are starting to rework windowing and window streams, which means we can take feedback for the future implementation.

What kind of requirements would the benchmark query have?

Greetings,

Stephan

On Thu, Sep 17, 2015 at 11:02 AM, Philipp Goetze <[hidden email]> wrote:

Hey community,

is there a possibility to join two windowed data streams, instead of joining two data streams on a window?

For example if one wants to implement Q1 of SRBench you load the data, create one window definition and then one would combine filters and joins on this window. But how would you do these Joins in Flink Streaming?

Best Regards,
Philipp

[Philipp Goetze]

Hey Stephan,

the query would require to either join windowed data streams or doing self joins within one window. If you take a look at Q1 of SRBench you will see, that it is a quite short query. Another option would be to support SPARQL queries as Flink operator (so the WHERE clause).

Best Regards,
Philipp

On 18.09.2015 10:53, Stephan Ewen wrote:

Hi Philipp!

I don't think this currently works. We are starting to rework windowing and window streams, which means we can take feedback for the future implementation.

What kind of requirements would the benchmark query have?

Greetings,

Stephan

On Thu, Sep 17, 2015 at 11:02 AM, Philipp Goetze <[hidden email]> wrote:

Hey community,

is there a possibility to join two windowed data streams, instead of joining two data streams on a window?

For example if one wants to implement Q1 of SRBench you load the data, create one window definition and then one would combine filters and joins on this window. But how would you do these Joins in Flink Streaming?

Best Regards,
Philipp