Are files in savepoint still needed after restoring if turning on incremental checkpointing

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Are files in savepoint still needed after restoring if turning on incremental checkpointing

Lu Niu
Hi, Flink Users

Assuming one flink job turns incremental checkpointing and restores from a savepoint. It runs fine for a while and commits one checkpoint and then it fully restarts because of one error. At this time, is it possible that the job still needs files in the original savepoint for recovery? If so, how to determine whether a savepoint is safe to delete? 

Best
Lu
Reply | Threaded
Open this post in threaded view
|

Re: Are files in savepoint still needed after restoring if turning on incremental checkpointing

Yun Tang
Hi Lu

Once a new checkpoint is completed when restoring from a savepoint, the previous savepoint would be useless if you decide to restore from new checkpoint.
In other words, new incremental checkpoint has no relationship with older savepoint from which restored.

Best
Yun Tang

From: Lu Niu <[hidden email]>
Sent: Saturday, July 18, 2020 5:48
To: user <[hidden email]>
Subject: Are files in savepoint still needed after restoring if turning on incremental checkpointing
 
Hi, Flink Users

Assuming one flink job turns incremental checkpointing and restores from a savepoint. It runs fine for a while and commits one checkpoint and then it fully restarts because of one error. At this time, is it possible that the job still needs files in the original savepoint for recovery? If so, how to determine whether a savepoint is safe to delete? 

Best
Lu
Reply | Threaded
Open this post in threaded view
|

Re: Are files in savepoint still needed after restoring if turning on incremental checkpointing

Lu Niu
Thanks Yun! That's what I thought :)

Best
Lu

On Sun, Jul 19, 2020 at 7:57 PM Yun Tang <[hidden email]> wrote:
Hi Lu

Once a new checkpoint is completed when restoring from a savepoint, the previous savepoint would be useless if you decide to restore from new checkpoint.
In other words, new incremental checkpoint has no relationship with older savepoint from which restored.

Best
Yun Tang

From: Lu Niu <[hidden email]>
Sent: Saturday, July 18, 2020 5:48
To: user <[hidden email]>
Subject: Are files in savepoint still needed after restoring if turning on incremental checkpointing
 
Hi, Flink Users

Assuming one flink job turns incremental checkpointing and restores from a savepoint. It runs fine for a while and commits one checkpoint and then it fully restarts because of one error. At this time, is it possible that the job still needs files in the original savepoint for recovery? If so, how to determine whether a savepoint is safe to delete? 

Best
Lu