Re: Broadcasting sets in Flink Streaming

Posted by Till Rohrmann on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Broadcasting-sets-in-Flink-Streaming-tp2504p2505.html

Hi Tamara,

I think this is not officially supported by Flink yet. However, I think that Gyula had once an example where he did something comparable. Maybe he can chime in here.

Cheers,

Till

On Tue, Aug 25, 2015 at 11:15 AM, Tamara Mendt <[hidden email]> wrote:

Hello,

I have been trying to use the function withBroadcastSet on a SingleOutputStreamOperator (map) the same way I would on a MapOperator for a DataSet. From what I see, this cannot be done. I wonder if there is some way to broadcast a DataSet to the tasks that are performing transformations on a DataStream?

I am basically pre-calculating some things with Flink which I later need for the transformations on the incoming data from the stream. So I want to broadcast the resulting datasets from the pre-calculations.

Any ideas on how to best approach this?

Thanks, cheers

Tamara.