Changing the topology while upgrading a job submitted by SQL

[Yi Tang]

Hi folks,

A question about changing the topology while upgrading a job submitted by SQL.

Is it possible for now?

Looks like if we want to recover a job from a savepoint, it requires the uid of the operator matches the corresponding one in the state. The automatically generated uid depends largely on the topology, right? So we would better assign the uid manually which can not be achieved using SQL?

Thanks

[Timo Walther]

Hi,

unfortunately, we currently don't provide any upgrading guarantees for
SQL. In theory we could add a possibility to add operator uids, however,
this will not help much because the underlying SQL operators or better
optimization rules that create a smarter pipeline could change the
entire topology with every major Flink version upgrade.

Regards,
Timo

On 10.11.20 10:15, Yi Tang wrote:

> Hi folks,
>
> A question about changing the topology while upgrading a job submitted
> by SQL.
> Is it possible for now?
>
> Looks like if we want to recover a job from a savepoint, it requires the
> uid of the operator matches the corresponding one in the state. The
> automatically generated uid depends largely on the topology, right? So
> we would better assign the uid manually which can not be achieved using SQL?
>
> Thanks

[Yi Tang]

Hi Timo,

Got it, thanks

Timo Walther <[hidden email]> 于 2020年11月10日周二 22:39写道：

Hi,

unfortunately, we currently don't provide any upgrading guarantees for
SQL. In theory we could add a possibility to add operator uids, however,
this will not help much because the underlying SQL operators or better
optimization rules that create a smarter pipeline could change the
entire topology with every major Flink version upgrade.

Regards,
Timo

On 10.11.20 10:15, Yi Tang wrote:
> Hi folks,
>
> A question about changing the topology while upgrading a job submitted
> by SQL.
> Is it possible for now?
>
> Looks like if we want to recover a job from a savepoint, it requires the
> uid of the operator matches the corresponding one in the state. The
> automatically generated uid depends largely on the topology, right? So
> we would better assign the uid manually which can not be achieved using SQL?
>
> Thanks