Re: Operations dependencies between values with different key in a ConnectedStreams

Posted by Chao Wang on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Operations-dependencies-between-values-with-different-key-in-a-ConnectedStreams-tp14518p14521.html

Hi Gabriele,

I think CEP may be able to deal with this kind of expressions, in
general, although I am not sure about how to deal with different time
windows (5s and 3s, in your case). Take a look at the available patterns
in the CEP library doc:
https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/libs/cep.html#the-pattern-api

Chao


On 07/28/2017 10:08 AM, Gabriele Di Bernardo wrote:

> Hi guys,
>
> I have a question for you. I have an application with two keyed data streams: one for control and the other one for the data. Each control message represents an operation to be performed on the data values marked with a certain identifier. I connected the two streams and I process the data with a CoProcessFunction.
>
> The operations I do are really simple like collecting the MAX or the MEAN value of the last n seconds. Now, I would like to create more complex operations where the result value of a key might depend by the result value of another key. To be more clear, I would like to evaluate expressions like: if {ALL the values of data marked with id 22 in the last 5s} are BIGGER THAN {The MEAN value of data marked with id 17 in the last 3s}. In this example, I should defer the evaluation of the expression until I have the MEAN value of the right part of the expression and check it against ALL the data keyed with key 22 from the last 5 seconds. I’d like to ask you if something like this in Flink is doable and what is the best way to do that in your opinion. I am also checking how the CEP library works (can it be helpful?).
>
> Thank you so much in advance.
>
> Best,
>
>
> Gabriele