Re: Flink CheckPoint/Savepoint Behavior Question

Posted by raghav280392 on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Flink-CheckPoint-Savepoint-Behavior-Question-tp41176p41179.html

Flink is aware of all the tasks running in the cluster. If any of the tasks fails, the failed task is restored using the checkpoint (only If the task uses Flink Operator State). This scenario will not use savepoints. Savepoints are same as checkpoints and the difference is that the savepoints are created manually or when we manually cancel/stop a job. We can then start the same job again by pointing to the savepoint. If we start a job without a savepoint, the job will start with an empty operator state.

Correct me If I am wrong.

Other references:

https://stackoverflow.com/questions/62935269/apache-flink-how-checkpoint-savepoint-works-if-we-run-duplicate-jobs-multi-te

https://stackoverflow.com/questions/64605940/apache-flink-fsstatebackend-how-state-is-recovered-in-case-of-ta+sk-manager-f

https://stackoverflow.com/questions/55613112/is-it-possible-to-recover-after-losing-the-checkpoint-coordinator/55615858#55615858

https://ci.apache.org/projects/flink/flink-docs-stable/ops/state/checkpoints.html#retained-checkpoints

Thank you

Virus-free. www.avast.com

On Tue, Feb 2, 2021 at 4:07 AM Jason Liu <[hidden email]> wrote:

We currently have some logic to load data from S3 into memory in our Flink/Kinesis Analytics app. This happens before the RichFunction.open() function.

We have two questions here and I can't find too much information in the apache.org website:

1. (More of a clarification) When Flink does checkpointing/savepointing only the state and the stream positions are saved right? The memory content won't be saved and restored.
   

2. When Flink restores from checkpoint/savepoint, does it still go through the application initialization phase? Basically will the code before the RichFunction' open() be run? If not, would the operators.open() functions run, when Flink restore from checkpoint/savepoint?
   

Thanks,

Jason

--

Raghavendar T S

www.teknosrc.com

Virus-free. www.avast.com