Re: Broadcasting sets in Flink Streaming
Posted by
Stephan Ewen on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Broadcasting-sets-in-Flink-Streaming-tp2504p2506.html
You can do something very similar like broadcast sets like this:
Use a Co-Map function and connect your main data set regularly ("forward" partitioning) to one input and your broadcast set via "broadcast" to the other input. You can then retrieve the data in the two map functions separately.
This approach misses the logic that the broadcast data arrives fully before the non-broadcast data (you may receive events from the main data set before all broadcast data was received), but maybe you can work around that...