Hello,
Currently I have 2 streams and I enrich stream 1 with the second streams. To further enrich stream 1 we are planning to add 2 more streams so a total of 4 streams.
- Stream 1 read from Kafka
- Stream 2 read from Kafka
- Stream 3 will be read from Kafka - new
- Stream 4 will be read from ES . - new
1. It is possible to combine all 4 streams together - all the streams are of totally different types of stream.
2. Is it possible to get the Stream 3 and Stream 4 and store them in a global hash maps which can be made accessible to all workers/operators? while trying to enrich stream 1 with Stream 2. This is help not changing the current design much. So basically Stream 3 and Stream 4 goes into the global hash map and when Stream 1 is being enriched with stream2 it will do a get/lookup on the global hash map and thus also use stream 3 and 4 to enrich itself.
I don't know how to combine more than 2 streams with connect. Please let me know if there is a way to do that simultaneously and not in stages. Please help with code examples if any.
I am new to flink. Thanks for your help.
Regards
Siddhartha