Clarity on Flink 1.5 Rescale mechanism

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Clarity on Flink 1.5 Rescale mechanism

Sampath Bhat
Hello
In flink 1.5 release notes - https://flink.apache.org/news/2018/05/25/release-1.5.0.html#release-notes 

Various Other Features and Improvements:

Applications can be rescaled without manually triggering a savepoint. Under the hood, Flink will still take a savepoint, stop the application, and rescale it to the new parallelism.

What exactly does this statement mean?
How to rescale the application ?
If i'm running my flink job application with parallelism of 5 and then I execute the following command
flink run <jar_path> -p 10 then will the application rescale to 10 parallelism?

Adding on what will be the behavior in flink 1.5 if I increase the number of task manager in a flink cluster?

So if i have a flink job running on flink cluster having 1 task manager and i increase the number of task manager to 2 will the flink rescale the flink job also or will the flink job be unaffected?

Reply | Threaded
Open this post in threaded view
|

Re: Clarity on Flink 1.5 Rescale mechanism

Stefan Richter
Hi,

it means that you can now modify the parallelism of a running job with a new „modify“ command in the CLI, see [1]. Adding task manager will add their offered slots to the pool of available slots, it will not automatically change the parallelism.

Best,
Stefan

[1] https://ci.apache.org/projects/flink/flink-docs-master/ops/cli.html

Am 12.06.2018 um 11:50 schrieb Sampath Bhat <[hidden email]>:

Hello
In flink 1.5 release notes -
https://flink.apache.org/news/2018/05/25/release-1.5.0.html#release-notes

Various Other Features and Improvements:
Applications can be rescaled without manually triggering a savepoint. Under
the hood, Flink will still take a savepoint, stop the application, and
rescale it to the new parallelism.

What exactly does this statement mean?
How to rescale the application ?
If i'm running my flink job application with parallelism of 5 and then I
execute the following command
flink run <jar_path> -p 10 then will the application rescale to 10
parallelism?

Adding on what will be the behavior in flink 1.5 if I increase the number
of task manager in a flink cluster?

So if i have a flink job running on flink cluster having 1 task manager and
i increase the number of task manager to 2 will the flink rescale the flink
job also or will the flink job be unaffected?