Delete save point when using incremental checkpoint

[Kien Truong]

Hi,

When using increment checkpoint mode, can I delete the save point that the job recovered from after sometime ? Or do I have to keep that checkpoint forever because it's a part of the snapshot chain ?

Best regards,

Kien

[Chesnay Schepler]

Hi,

There is an important distinction between checkpoints (triggered by
Flink, may be incremental) and savepoints (manually triggered, always
self-contained).

Your question is unfortunately mixing both terms, please expand which
you're referring to.

Regards,
Chesnay

On 11.10.2017 10:31, Kien Truong wrote:
> Hi,
>
> When using increment checkpoint mode, can I delete the save point that
> the job recovered from after sometime ? Or do I have to keep that
> checkpoint forever because it's a part of the snapshot chain ?
>
> Best regards,
> Kien

[Stephan Ewen]

Kien,

I think what you are asking is: Do incremental checkpoints refer to chunks of state in a previous savepoint. Meaning that if the savepoint is deleted, will the checkpoint be missing some chunks.

The answer is: No, savepoints are always full snapshots and incremental checkpoints do not reference and savepoint state chunks.

Stephan

On Wed, Oct 11, 2017 at 10:54 AM, Chesnay Schepler <[hidden email]> wrote:

Hi,

There is an important distinction between checkpoints (triggered by Flink, may be incremental) and savepoints (manually triggered, always self-contained).

Your question is unfortunately mixing both terms, please expand which you're referring to.

Regards,
Chesnay

On 11.10.2017 10:31, Kien Truong wrote:

Hi,

When using increment checkpoint mode, can I delete the save point that the job recovered from after sometime ? Or do I have to keep that checkpoint forever because it's a part of the snapshot chain ?

Best regards,
Kien

[Kien Truong]

Thanks Stephan. That's exactly what I was asking about.

Best regards,

Kien

On Oct 11, 2017, at 16:59, Stephan Ewen <[hidden email]> wrote:

Kien,

I think what you are asking is: Do incremental checkpoints refer to chunks of state in a previous savepoint. Meaning that if the savepoint is deleted, will the checkpoint be missing some chunks.

The answer is: No, savepoints are always full snapshots and incremental checkpoints do not reference and savepoint state chunks.

Stephan

On Wed, Oct 11, 2017 at 10:54 AM, Chesnay Schepler <[hidden email]> wrote:

Hi,

There is an important distinction between checkpoints (triggered by Flink, may be incremental) and savepoints (manually triggered, always self-contained).

Your question is unfortunately mixing both terms, please expand which you're referring to.

Regards,
Chesnay

On 11.10.2017 10:31, Kien Truong wrote:

Hi,

When using increment checkpoint mode, can I delete the save point that the job recovered from after sometime ? Or do I have to keep that checkpoint forever because it's a part of the snapshot chain ?

Best regards,
Kien