Re: Is MapState tied to Operator Input & Output type?

Posted by Yun Tang on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Is-MapState-tied-to-Operator-Input-Output-type-tp38683p38684.html

Hi

The type of map state is not directly related with input & output type, this is only related with how you define the state descriptor.

• Have you ever changed the state descriptor after changing the type of input/output type?
• Have you assigned the id [1] to the operator which using the 'TestProcess'? The state might not be restored if you change your code without id assigned.

[1] https://ci.apache.org/projects/flink/flink-docs-stable/ops/state/savepoints.html#assigning-operator-ids

Best

Yun Tang

---

From: Arpith P <[hidden email]>
Sent: Tuesday, October 13, 2020 19:26
To: user <[hidden email]>
Subject: Is MapState tied to Operator Input & Output type?

 

Hi,

I’ve a ProcessFunction which initially was receiving input & output type of String ⁽¹⁾ & inside processElement I was updating MapState. Now I have changed the Input & Output type to be Map, String ⁽²⁾, but if I restore from the last checkpoint folder MapState is coming in as empty. I’ve checked that checkpoint folder actually saves data (i.e. Files size > 1GB). Does map state tied with ProcessFunction input & output type, if not why doesn't mapstate get restored.

 

(1)    

       public class TestProcess extends ProcessFunction<String, String> implements CheckpointedFunction

(2)    

    public class TestProcess extends ProcessFunction<Map<String, String>, String> implements CheckpointedFunction