Hi Vijay,
Generally, for asynchronous operations to enrich (or in your case, fetching the algorithm for the actual transformation of the data), you’ll want to look at Flink’s Async I/O [1].
For your second question, I can see it as a stateful `FlatMapFunction` that keeps the seen results as managed state. Once all results are seen (i.e. the last result arrives at the operator), you join them and emit your final result further downstream.
Does this help with the use case you have in mind?
Cheers,
Gordon
On 27 April 2017 at 5:29:05 PM, G.S.Vijay Raajaa ([hidden email]) wrote:
HI,
I have just started to explore Flink
and have couple of questions. I am wondering if its possible
to call a rest endpoint asynchronously and pipe the response to the
next state of my transformation on the stream. The idea is such
that after charging my data in a predefined time window, I would
like to apply some algorithm/transformation on the window of data
external to Flink. The algos have been exposed as REST
endpoints.
My seconds question is an extension
to the previous one, if i need to transform my window data by
applying three different algorithms in parallel exposed via
subsequent rest endpoints, how do wait unless the individual rest
endpoints respond back . Potentially I need to join the results of
the three algorithms before I call the sink.
Regards,
Vijay Raajaa
Locked
