How to connect more than 2 hetrogenous Streams!!

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

How to connect more than 2 hetrogenous Streams!!

Puneet Kinra-2
Hi 

Is there a way to connect more than 2 streams with different stream schema

--
Cheers 

Puneet Kinra

Mobile:+918800167808 | Skype : [hidden email]

e-mail :[hidden email]


Reply | Threaded
Open this post in threaded view
|

Re: How to connect more than 2 hetrogenous Streams!!

Hequn Cheng
Hi Puneet,

Flink doesn't support connecting more than 2 streams with different schema. There are ways I think might help you.
1. unify the schema and use union.
2. use multi join to join different streams.

Hope this helps.
Hequn 

On Thu, Jul 26, 2018 at 2:29 PM, Puneet Kinra <[hidden email]> wrote:
Hi 

Is there a way to connect more than 2 streams with different stream schema

--
Cheers 

Puneet Kinra

Mobile:+918800167808 | Skype : [hidden email]

e-mail :[hidden email]



Reply | Threaded
Open this post in threaded view
|

Re: How to connect more than 2 hetrogenous Streams!!

vino yang
Hi Puneet,

Hequn gave you two good solutions. If you have a good knowledge of Flink DataStream API. 
You can also customize it to connect more than two streams, you must know :
  • DataStream#connect API
  • ConnectedStreams
  • CoMapFunction, CoFlatMapFunction...
Referring to them, you can got connect(DataStream1, DataStream2...). 

Really, some scenario need more than two streams connecting such as CEP.

Thanks, vino.


2018-07-27 19:48 GMT+08:00 Hequn Cheng <[hidden email]>:
Hi Puneet,

Flink doesn't support connecting more than 2 streams with different schema. There are ways I think might help you.
1. unify the schema and use union.
2. use multi join to join different streams.

Hope this helps.
Hequn 

On Thu, Jul 26, 2018 at 2:29 PM, Puneet Kinra <[hidden email]> wrote:
Hi 

Is there a way to connect more than 2 streams with different stream schema

--
Cheers 

Puneet Kinra

Mobile:+918800167808 | Skype : [hidden email]

e-mail :[hidden email]