Re: State Access Beyond RichCoFlatMapFunction

Posted by Kezhu Wang on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/State-Access-Beyond-RichCoFlatMapFunction-tp41373p41380.html

(a) It is by design. For keyed state, you can only access state for that key, not others. If you want one value per key, ValueState fits more appropriate that MapState.

(b) state-processor-api aims to access/create/modify/upgrade offline savepoint but not running state. Queryable state may meet your requirement, but it is not actively developed for a while according to my observation and still beta. 

Queryable state: https://ci.apache.org/projects/flink/flink-docs-release-1.12/dev/stream/state/queryable_state.html

On February 9, 2021 at 22:09:29, Sandeep khanzode ([hidden email]) wrote:

Hello,

I am creating a class that extends RichCoFlatMapFunction. I need to connect() two streams to basically share the state of one stream in another. 

This is what I do:

private transient MapState<KeyClass, ValueClass> state;

@Override

public void open(Configuration parameters) throws Exception {
    MapStateDescriptor<KeyClass, ValueClass> stateDescriptor =
            new MapStateDescriptor<>(“abc-saved-state",
                    Types.POJO(KeyClass.class), Types.POJO(ValueClass.class));
    state = getRuntimeContext().getMapState(stateDescriptor);

This works correctly.

I have two questions:

(a) Whenever I debug, I can only see the current key in the MapState, not all the possible keys that were created before and saved. Next time, I get a hit for another key, I will only see the other key and not the rest of previous keys. Is it by design or am I missing something?

(b) Can I somehow access this state beyond the class that holds the state? I.e. can I access the state in some other class? If not, can I use the https://ci.apache.org/projects/flink/flink-docs-release-1.9/dev/libs/state_processor_api.html to do this? Is that the correct way to access the running state of one stream elsewhere in the program without corrupting it?

Your response will be greatly appreciated. I will be happy to add more details if required.

Thanks,

Sandeep Ramesh Khanzode