the savepoint problem of upgrading job from blink-1.5 to flink-1.10

[Roc Marshal]

Hi, all.

When using savepoint to upgrade a Flink job from blink-1.5 to flink-1.10, 

the system prompts that blink savepointV3 is incompatible with the version in Flink. Is there any solution? 

Thank you so much.

Sincerely,

Roc Marshal

 

[Congxian Qiu]

Hi,

Could you please share the stack or the log message?
If I understand correctly, savepoint V3 is not contained in 1.10, 

Roc Marshal <[hidden email]> 于2020年5月15日周五 下午4:33写道：

Hi, all.

When using savepoint to upgrade a Flink job from blink-1.5 to flink-1.10, 

the system prompts that blink savepointV3 is incompatible with the version in Flink. Is there any solution? 

Thank you so much.

Sincerely,

Roc Marshal

 

[Yun Tang]

Hi Roc

Blink-1.5 should never make the promise that it could be compatible with Flink-1.10.

Moreover, the SavepointV3Serializer in Blink is totally no the same thing as Flink, and the reason why we introduce SavepointV3Serializer is because we use different state handle when we open source blink. As you can see, we use "org.apache.flink.runtime.state.KeyGroupsStateSnapshot" instead of "org.apache.flink.runtime.state.KeyGroupsStateHandle", and thus the savepoint generated by Blink cannot be easily consumed by Flink.

Best

Yun Tang

---

From: Congxian Qiu <[hidden email]>
Sent: Friday, May 15, 2020 17:20
To: Roc Marshal <[hidden email]>
Cc: user <[hidden email]>
Subject: Re: the savepoint problem of upgrading job from blink-1.5 to flink-1.10

 

Hi,

Could you please share the stack or the log message?
If I understand correctly, savepoint V3 is not contained in 1.10, 

Roc Marshal <[hidden email]> 于2020年5月15日周五 下午4:33写道：

Hi, all.

When using savepoint to upgrade a Flink job from blink-1.5 to flink-1.10, 

the system prompts that blink savepointV3 is incompatible with the version in Flink. Is there any solution? 

Thank you so much.

Sincerely,

Roc Marshal

 

[Congxian Qiu]

This email wants to correct my previous mail, Yun's answer is correct.

Yun Tang <[hidden email]> 于2020年5月15日周五 下午6:01写道：

Hi Roc

Blink-1.5 should never make the promise that it could be compatible with Flink-1.10.

Moreover, the SavepointV3Serializer in Blink is totally no the same thing as Flink, and the reason why we introduce SavepointV3Serializer is because we use different state handle when we open source blink. As you can see, we use "org.apache.flink.runtime.state.KeyGroupsStateSnapshot" instead of "org.apache.flink.runtime.state.KeyGroupsStateHandle", and thus the savepoint generated by Blink cannot be easily consumed by Flink.

Best

Yun Tang

---

From: Congxian Qiu <[hidden email]>
Sent: Friday, May 15, 2020 17:20
To: Roc Marshal <[hidden email]>
Cc: user <[hidden email]>
Subject: Re: the savepoint problem of upgrading job from blink-1.5 to flink-1.10

 

Hi,

Could you please share the stack or the log message?
If I understand correctly, savepoint V3 is not contained in 1.10, 

Best,

Congxian

Roc Marshal <[hidden email]> 于2020年5月15日周五 下午4:33写道：

Hi, all.

When using savepoint to upgrade a Flink job from blink-1.5 to flink-1.10, 

the system prompts that blink savepointV3 is incompatible with the version in Flink. Is there any solution? 

Thank you so much.

Sincerely,

Roc Marshal

 

[Arvid Heise-3]

Hi Roc,

just as an addition to Congxian, Flink 1.11 will be released soonish. We just started the release process (created release branch to freeze the features), which should include 2-4 weeks of testing/bug fixing. Of course, if you are interested, you could use the current master/release branch and try it for yourself. We could also inform you of the first release candidate RC0, which I suspect at the end of this week/beginning of next week.

On Sun, May 17, 2020 at 11:56 AM Congxian Qiu <[hidden email]> wrote:

This email wants to correct my previous mail, Yun's answer is correct.

I mistook "blink-1.5" for "flink-1.5" when replying to the previous email. and the savepoint V3  is not in flink-1.10(will be in 1.11)

Best,

Congxian

Yun Tang <[hidden email]> 于2020年5月15日周五 下午6:01写道：

Hi Roc

Blink-1.5 should never make the promise that it could be compatible with Flink-1.10.

Moreover, the SavepointV3Serializer in Blink is totally no the same thing as Flink, and the reason why we introduce SavepointV3Serializer is because we use different state handle when we open source blink. As you can see, we use "org.apache.flink.runtime.state.KeyGroupsStateSnapshot" instead of "org.apache.flink.runtime.state.KeyGroupsStateHandle", and thus the savepoint generated by Blink cannot be easily consumed by Flink.

Best

Yun Tang

---

From: Congxian Qiu <[hidden email]>
Sent: Friday, May 15, 2020 17:20
To: Roc Marshal <[hidden email]>
Cc: user <[hidden email]>
Subject: Re: the savepoint problem of upgrading job from blink-1.5 to flink-1.10

 

Hi,

Could you please share the stack or the log message?
If I understand correctly, savepoint V3 is not contained in 1.10, 

Best,

Congxian

Roc Marshal <[hidden email]> 于2020年5月15日周五 下午4:33写道：

Hi, all.

When using savepoint to upgrade a Flink job from blink-1.5 to flink-1.10, 

the system prompts that blink savepointV3 is incompatible with the version in Flink. Is there any solution? 

Thank you so much.

Sincerely,

Roc Marshal

 

--