Broadcast state with WindowedStream

[Aggarwal, Ajay]

Is it possible to use broadcast state with windowing? My job looks like below

 

                                inputStream

                                .keyBy("some-key")

                                .window(TumblingEventTimeWindows.of(Time.seconds(Properties.WINDOW_SIZE)))

                                .process(new MyProcessWindowFunction());

 

I wanted to introduce broadcast state that MyProcessWindowFunction can make use of.  The example here (https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/broadcast_state.html) covers connecting a Keyed and non-Keyed stream with the BroadcastStream. But in my case I need to connect WindowedStream with BroadcastStream, which isn’t seem to be supported.

 

Ajay

[Aggarwal, Ajay]

It sort of makes sense that broadcast state is not available with WindowedStream. But if I need some dynamic global state in MyProcessWindowFunction  what are my options?

 

Ajay

 

From: "Aggarwal, Ajay" <[hidden email]>
Date: Monday, March 4, 2019 at 4:36 PM
To: "[hidden email]" <[hidden email]>
Subject: Broadcast state with WindowedStream

 

 

Is it possible to use broadcast state with windowing? My job looks like below

 

                                inputStream

                                .keyBy("some-key")

                                .window(TumblingEventTimeWindows.of(Time.seconds(Properties.WINDOW_SIZE)))

                                .process(new MyProcessWindowFunction());

 

I wanted to introduce broadcast state that MyProcessWindowFunction can make use of.  The example here (https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/broadcast_state.html) covers connecting a Keyed and non-Keyed stream with the BroadcastStream. But in my case I need to connect WindowedStream with BroadcastStream, which isn’t seem to be supported.

 

Ajay

[Aggarwal, Ajay]

Still looking for ideas as to how I can use broadcast state in my use case.

 

From: "Aggarwal, Ajay" <[hidden email]>
Date: Monday, March 4, 2019 at 4:52 PM
To: "[hidden email]" <[hidden email]>
Subject: Re: Broadcast state with WindowedStream

It sort of makes sense that broadcast state is not available with WindowedStream. But if I need some dynamic global state in MyProcessWindowFunction  what are my options?

 

Ajay

 

From: "Aggarwal, Ajay" <[hidden email]>
Date: Monday, March 4, 2019 at 4:36 PM
To: "[hidden email]" <[hidden email]>
Subject: Broadcast state with WindowedStream

 

 

Is it possible to use broadcast state with windowing? My job looks like below

 

                                inputStream

                                .keyBy("some-key")

                                .window(TumblingEventTimeWindows.of(Time.seconds(Properties.WINDOW_SIZE)))

                                .process(new MyProcessWindowFunction());

 

I wanted to introduce broadcast state that MyProcessWindowFunction can make use of.  The example here (https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/broadcast_state.html) covers connecting a Keyed and non-Keyed stream with the BroadcastStream. But in my case I need to connect WindowedStream with BroadcastStream, which isn’t seem to be supported.

 

Ajay