Stream Iterations

[Govindarajan Srinivasaraghavan]

Hi All,

I have a use case for which I need some suggestions. It's a streaming application with kafka source and then groupBy, keyBy and perform some calculations. The output of each calculation has to be a side input for the next calculation and also it needs to be sent to a sink.

Right now I'm achieving this by storing the result state in memory and also save it in redis cache. I was looking at delta iterations in flink documentation. It would great if someone can help me understand if I can achieve this using iterations or any other api. Thanks in advance.

Regards,

Govind

[Ufuk Celebi]

The delta iterations are a batch-only feature.

Did you try the DataStream#iterate?

https://ci.apache.org/projects/flink/flink-docs-release-1.1/apis/streaming/index.html#iterations

On Mon, Dec 19, 2016 at 5:35 AM, Govindarajan Srinivasaraghavan
<[hidden email]> wrote:

> Hi All,
>
> I have a use case for which I need some suggestions. It's a streaming
> application with kafka source and then groupBy, keyBy and perform some
> calculations. The output of each calculation has to be a side input for the
> next calculation and also it needs to be sent to a sink.
>
> Right now I'm achieving this by storing the result state in memory and also
> save it in redis cache. I was looking at delta iterations in flink
> documentation. It would great if someone can help me understand if I can
> achieve this using iterations or any other api. Thanks in advance.
>
> Regards,
> Govind