Use state problem

[ゞ野蠻遊戲χ]

Deal all

Keyed state (ValueState, ReducingState, ListState, AggregatingState, MapState)

Supported in Keyed Stream, meaning only in KeyedProcessFunction? But in practice, I can also use these states in ProcessAllWindowFunction and ProcessWindowFunction. Why?

thank you

jiazhi

[Yun Tang]

Hi  Jiazhi

Keyed state is only supported in keyed stream as it needs key selector and key serializer to select specific key from the input element, this is correct.

If you dig into Flink code, the keyed state backend would only be created when the operator has its own serializer[1].

After 'keyBy' to transform data stream to keyed stream, you can use the keyed state in the following operator. And the following operator is not limited to only operator with "keyed" prefix.

For ProcessAllWindowFunction, we need to use it with windowAll transformation [2] and windowAll actually includes keyBy actions if takeing a look at the code[3].

For ProcessWindowFunction, we must use keyBy first and then use next window transformation.

Hope this explanation could help you.

[1] https://github.com/apache/flink/blob/c23787093a9da0e12561fbe22dd6da6164ffe951/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/operators/StreamTaskStateInitializerImpl.java#L281-L283

[2] https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/operators/#datastream-transformations

[3] https://github.com/apache/flink/blob/c23787093a9da0e12561fbe22dd6da6164ffe951/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/datastream/AllWindowedStream.java#L118

Best

Yun Tang

---

From: ゞ野蠻遊戲χ <[hidden email]>
Sent: Thursday, July 9, 2020 9:58
To: user <[hidden email]>
Subject: Use state problem

 

Deal all

Keyed state (ValueState, ReducingState, ListState, AggregatingState, MapState)

Supported in Keyed Stream, meaning only in KeyedProcessFunction? But in practice, I can also use these states in ProcessAllWindowFunction and ProcessWindowFunction. Why?

thank you

jiazhi