Shared DataStream

[nragon]

Hi,

Can we share the end point of on job (datastream) with another job?
The arquitecture I'm working on should provide abstraction and dynamism between processing jobs, which transform and load data into hbase or other sink, and cep jobs, which will be used to detect anomalies. But because the datastream of one processing job can be used to N cep jobs i wonder if sharing datastreams between jobs would be possible? Or if not, do I need to send them back to kafka.

Thanks

[Fabian Hueske-2]

Flink does not provide any tooling to link two jobs together.

I would use Kafka to connect Flink jobs with each other.

Best, Fabian

2017-04-06 17:28 GMT+02:00 nragon <[hidden email]>:

Hi,

Can we share the end point of on job (datastream) with another job?
The arquitecture I'm working on should provide abstraction and dynamism
between processing jobs, which transform and load data into hbase or other
sink, and cep jobs, which will be used to detect anomalies. But because the
datastream of one processing job can be used to N cep jobs i wonder if
sharing datastreams between jobs would be possible? Or if not, do I need to
send them back to kafka.

Thanks



--
View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Shared-DataStream-tp12545.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.

[nragon]

Yeah, hoping you'd say that :)
Would be nice to share them just because the network overhead between flink and kafka.

Thanks