How do I compute the average and keep track of a state over a window in DataStream?

[Felipe Gutierrez]

Hi all,

I want to compute the average of two stream data sources and also keep track of a ValueState variable which is a CountMinSketch class that I implemented. For this, I tried to use RichAggregateFunction however it throws an exception saying that  - Exception in thread "main" java.lang.UnsupportedOperationException: This aggregation function cannot be a RichFunction.

Here is the code that I am implementing together with the line that is compiling but says that Flink does not support it - https://github.com/felipegutierrez/explore-flink/blob/master/src/main/java/org/sense/flink/examples/stream/MultiSensorMultiStationsReadingMqtt2.java#L70

How would I implement this in Flink?

Thanks for your ideas

Felipe

[Congxian Qiu]

Hi Felipe

Maybe you could use process function[1]

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.7/dev/stream/operators/process_function.html

Best, Congxian

On Feb 28, 2019, 22:47 +0800, Felipe Gutierrez <[hidden email]>, wrote:

Hi all,

I want to compute the average of two stream data sources and also keep track of a ValueState variable which is a CountMinSketch class that I implemented. For this, I tried to use RichAggregateFunction however it throws an exception saying that  - Exception in thread "main" java.lang.UnsupportedOperationException: This aggregation function cannot be a RichFunction.

Here is the code that I am implementing together with the line that is compiling but says that Flink does not support it - https://github.com/felipegutierrez/explore-flink/blob/master/src/main/java/org/sense/flink/examples/stream/MultiSensorMultiStationsReadingMqtt2.java#L70

How would I implement this in Flink?

Thanks for your ideas

Felipe

[Felipe Gutierrez]

thanks Congxian. I will check Process Function over windows.

On Fri, Mar 1, 2019 at 8:16 AM Congxian Qiu <[hidden email]> wrote:

Hi Felipe

Maybe you could use process function[1]

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.7/dev/stream/operators/process_function.html

Best, Congxian

On Feb 28, 2019, 22:47 +0800, Felipe Gutierrez <[hidden email]>, wrote:

Hi all,

I want to compute the average of two stream data sources and also keep track of a ValueState variable which is a CountMinSketch class that I implemented. For this, I tried to use RichAggregateFunction however it throws an exception saying that  - Exception in thread "main" java.lang.UnsupportedOperationException: This aggregation function cannot be a RichFunction.

Here is the code that I am implementing together with the line that is compiling but says that Flink does not support it - https://github.com/felipegutierrez/explore-flink/blob/master/src/main/java/org/sense/flink/examples/stream/MultiSensorMultiStationsReadingMqtt2.java#L70

How would I implement this in Flink?

Thanks for your ideas

Felipe