Can AsyncFunction be applied to connected streams

[PedroMrChaves]

Hello,

Is there a way to apply the AsyncFunction to connected streams like in a CoFlatMap?
I would like to connect streams from different types and process one of them based on the state
created by the other in an asynchronous fashion.

Regards,
Pedro Chaves

[Aljoscha Krettek]

Hi,

While it’s not possible to directly apply an Async I/O operator to a connected stream you can use a two-stage approach:
 - Have a CoFlatMap that enriches the elements from the first stream based on the second stream
 - Use an Async I/O operator right after that, on the enriched stream

Would that be an option for you?

Best,
Aljoscha

> On 6. Jul 2017, at 16:17, PedroMrChaves <[hidden email]> wrote:
>
> Hello,
>
> Is there a way to apply the AsyncFunction to connected streams like in a
> CoFlatMap?
> I would like to connect streams from different types and process one of them
> based on the state
> created by the other in an asynchronous fashion.
>
> Regards,
> Pedro Chaves
>
>
>
> -----
> Best Regards,
> Pedro Chaves
> --
> View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Can-AsyncFunction-be-applied-to-connected-streams-tp14137.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.

[PedroMrChaves]

Hello,

I wanted to keep the data locally, if I associate the fetched metadata with each event (in an enrichment phase) it would considerably increase their size since the metadata that I need to process the event in the async I/O is to large.

Regards,
Pedro.

[Aljoscha Krettek]

I think this would not necessarily be a problem. If the async operation is directly after the enrichment operation the enriched operations will be directly forwarded to the async operation. (With a copy step, that can be disabled by enabling object reuse at the StreamExecutionEnvironment)

Best,
Aljoscha

> On 7. Jul 2017, at 15:59, PedroMrChaves <[hidden email]> wrote:
>
> Hello,
>
> I wanted to keep the data locally, if I associate the fetched metadata with
> eachevent (in an enrichment phase) it would considerably increase their size
> since the metadata that I need to process the event in the async I/O is to
> large.
>
> Regards,
> Pedro.
>
>
>
> -----
> Best Regards,
> Pedro Chaves
> --
> View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Can-AsyncFunction-be-applied-to-connected-streams-tp14137p14148.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.