Connect more than two streams

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Connect more than two streams

Govindarajan Srinivasaraghavan

Hi,


I have two streams reading from kafka, one for data and other for control. The data stream is split by type and there are around six types. Each type has its own processing logic and finally everything has to be merged to get the collective state per device. I was thinking I could connect multiple streams, process and maintain state but connect only supports two streams. Is there some way to achieve my desired functionality?


By the way after split and some processing all of them are keyed streams.


Reply | Threaded
Open this post in threaded view
|

Re: Connect more than two streams

Jonas Gröger
Hello Govindarajan,

one way to merge multiple streams is to union them. You can do that with
the union operator described at [1]. Is that what you are looking for?


[1]
https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/datastream_api.html

-- Jonas

Am Mo, 24. Jul 2017, um 23:18, schrieb Govindarajan Srinivasaraghavan
[via Apache Flink User Mailing List archive.]:

>
>
> Hi,
>
>
> I have two streams reading from kafka, one for data and other for
> control.
> The data stream is split by type and there are around six types. Each
> type
> has its own processing logic and finally everything has to be merged to
> get
> the collective state per device. I was thinking I could connect multiple
> streams, process and maintain state but connect only supports two
> streams.
> Is there some way to achieve my desired functionality?
>
>
> By the way after split and some processing all of them are keyed streams.
>
>
>
>
> _______________________________________________
> If you reply to this email, your message will be added to the discussion
> below:
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Connect-more-than-two-streams-tp14419.html
>
> To unsubscribe from Apache Flink User Mailing List archive., visit
>