Restoring from checkpoint with different parallism

[Rex Fenley]

Hello,

When using the TableAPI, is it safe to run a flink job with a different `-p` parallelism while restoring from a checkpoint (not a savepoint) using `-s`, without any rescaling of actual machines? I don't seem to find this documented anywhere.

Thanks!

--

[Yun Tang]

 Hi Rex,

I think doc [1] should have given some descriptions. Rescaling from previous checkpoint is still supported in current Flink version.

[1] https://ci.apache.org/projects/flink/flink-docs-stable/ops/state/checkpoints.html#resuming-from-a-retained-checkpoint

Best

Yun Tang

---

From: Rex Fenley <[hidden email]>
Sent: Tuesday, January 12, 2021 11:01
To: user <[hidden email]>
Cc: Brad Davis <[hidden email]>
Subject: Restoring from checkpoint with different parallism

 

Hello,

When using the TableAPI, is it safe to run a flink job with a different `-p` parallelism while restoring from a checkpoint (not a savepoint) using `-s`, without any rescaling of actual machines? I don't seem to find this documented anywhere.

Thanks!

--

[Rex Fenley]

Thanks! Looks like you can't with unaligned checkpoints, which seems to imply that you can with normal checkpointing mechanism.

On Mon, Jan 11, 2021 at 7:56 PM Yun Tang <[hidden email]> wrote:

 Hi Rex,

I think doc [1] should have given some descriptions. Rescaling from previous checkpoint is still supported in current Flink version.

[1] https://ci.apache.org/projects/flink/flink-docs-stable/ops/state/checkpoints.html#resuming-from-a-retained-checkpoint

Best

Yun Tang

---

From: Rex Fenley <[hidden email]>
Sent: Tuesday, January 12, 2021 11:01
To: user <[hidden email]>
Cc: Brad Davis <[hidden email]>
Subject: Restoring from checkpoint with different parallism

 

Hello,

When using the TableAPI, is it safe to run a flink job with a different `-p` parallelism while restoring from a checkpoint (not a savepoint) using `-s`, without any rescaling of actual machines? I don't seem to find this documented anywhere.

Thanks!

--

Rex Fenley  |  Software Engineer - Mobile and Backend

Remind.com |  BLOG  |  FOLLOW US  |  LIKE US

--

[Arvid Heise-3]

Hi Rex,

that's exactly right and we are also working on bringing this feature to unaligned checkpoints.

On Tue, Jan 12, 2021 at 5:18 AM Rex Fenley <[hidden email]> wrote:

Thanks! Looks like you can't with unaligned checkpoints, which seems to imply that you can with normal checkpointing mechanism.

On Mon, Jan 11, 2021 at 7:56 PM Yun Tang <[hidden email]> wrote:

 Hi Rex,

I think doc [1] should have given some descriptions. Rescaling from previous checkpoint is still supported in current Flink version.

[1] https://ci.apache.org/projects/flink/flink-docs-stable/ops/state/checkpoints.html#resuming-from-a-retained-checkpoint

Best

Yun Tang

---

From: Rex Fenley <[hidden email]>
Sent: Tuesday, January 12, 2021 11:01
To: user <[hidden email]>
Cc: Brad Davis <[hidden email]>
Subject: Restoring from checkpoint with different parallism

 

Hello,

When using the TableAPI, is it safe to run a flink job with a different `-p` parallelism while restoring from a checkpoint (not a savepoint) using `-s`, without any rescaling of actual machines? I don't seem to find this documented anywhere.

Thanks!

--

Rex Fenley  |  Software Engineer - Mobile and Backend

Remind.com |  BLOG  |  FOLLOW US  |  LIKE US

--

Rex Fenley  |  Software Engineer - Mobile and Backend

Remind.com |  BLOG  |  FOLLOW US  |  LIKE US

--