[Flink 1.6.1] _metadata file in retained checkpoint

[Rinat]

Hi mates, I’m trying to configure my job to retain checkpoints on it’s cancellation and got some troubles.

I got the following args why not to use savepoints:

 1. we already have all the job state on DFS in checkpoints directory

 2. I can multiply size of the state on 2, when stopping the job, because the state is already stored in checkpoints dir, and I’ll save it one more time into savepoint dir

 3. creation of checkpoints is incremental and savepoints - is not, so in my case (10 Tb state) the process of savepoint creation will took too long time

 4. As I know, we can rescale or job from retained checkpoints

I've configured my job to retain checkpoints on job cancelation and found an interesting issue - _metadata file is removed, when job is cancelled.

So, I couldn’t restore my job from the retained checkpoint, is it an expected behaviour ? If so, what is wrong ?

Sincerely yours,

Rinat Sharipov

Software Engineer at 1DMP CORE Team

email: [hidden email]

mobile: [phone]

CleverDATA

make your data clever

[Vasyl Bervetskyi]

Hi Rinat,

 

Savepoint need to be triggered when you want to create point in time which you want to use in future to revert back your state, also you could cancel job with savepoint which makes sure that you won’t lose any data during canceling job.

 

About your question that flink delete metadata file: I am using Flink 1.8.0 and Flink deletes metadata file when I cancel job with savepoint, It seems Flink doesn’t see any reason to keep checkpoint cause you created savepoint. When you just cancel job without savepoint flink shouldn’t delete metadata file from checkpoint.

From: Rinat <[hidden email]>
Sent: Wednesday, June 5, 2019 4:36 PM
To: user <[hidden email]>
Subject: [Flink 1.6.1] _metadata file in retained checkpoint

 

Hi mates, I’m trying to configure my job to retain checkpoints on it’s cancellation and got some troubles.

 

I got the following args why not to use savepoints:

 

 1. we already have all the job state on DFS in checkpoints directory

 2. I can multiply size of the state on 2, when stopping the job, because the state is already stored in checkpoints dir, and I’ll save it one more time into savepoint dir

 3. creation of checkpoints is incremental and savepoints - is not, so in my case (10 Tb state) the process of savepoint creation will took too long time

 4. As I know, we can rescale or job from retained checkpoints

 

I've configured my job to retain checkpoints on job cancelation and found an interesting issue - _metadata file is removed, when job is cancelled.

 

So, I couldn’t restore my job from the retained checkpoint, is it an expected behaviour ? If so, what is wrong ?

 

 

Sincerely yours,

Rinat Sharipov

Software Engineer at 1DMP CORE Team

 

email: [hidden email]

mobile: [phone]

 

CleverDATA

make your data clever

 

[Rinat]

Hi Vasyl, thx for your reply, I’ll check

On 10 Jun 2019, at 14:22, Vasyl Bervetskyi <[hidden email]> wrote:

Hi Rinat,

 

Savepoint need to be triggered when you want to create point in time which you want to use in future to revert back your state, also you could cancel job with savepoint which makes sure that you won’t lose any data during canceling job.

 

About your question that flink delete metadata file: I am using Flink 1.8.0 and Flink deletes metadata file when I cancel job with savepoint, It seems Flink doesn’t see any reason to keep checkpoint cause you created savepoint. When you just cancel job without savepoint flink shouldn’t delete metadata file from checkpoint.

From: Rinat <[hidden email]> 
Sent: Wednesday, June 5, 2019 4:36 PM
To: user <[hidden email]>
Subject: [Flink 1.6.1] _metadata file in retained checkpoint

 

Hi mates, I’m trying to configure my job to retain checkpoints on it’s cancellation and got some troubles.

 

I got the following args why not to use savepoints:

 

 1. we already have all the job state on DFS in checkpoints directory

 2. I can multiply size of the state on 2, when stopping the job, because the state is already stored in checkpoints dir, and I’ll save it one more time into savepoint dir

 3. creation of checkpoints is incremental and savepoints - is not, so in my case (10 Tb state) the process of savepoint creation will took too long time

 4. As I know, we can rescale or job from retained checkpoints

 

I've configured my job to retain checkpoints on job cancelation and found an interesting issue - _metadata file is removed, when job is cancelled.

 

So, I couldn’t restore my job from the retained checkpoint, is it an expected behaviour ? If so, what is wrong ?

 

 

Sincerely yours,

Rinat Sharipov

Software Engineer at 1DMP CORE Team

 

email: [hidden email]

mobile: [phone]

 

CleverDATA

make your data clever

Sincerely yours,

Rinat Sharipov

Software Engineer at 1DMP CORE Team

email: [hidden email]

mobile: [phone]

CleverDATA

make your data clever