Hey Gyula,
as far as I can tell, there are no docs for this yet. Good news is
that the docs are getting a lot of love for the release and I guess
this is simply something that is still missing.
The setMaxParallelism call should be available in all the places where
you can also call setParallelism. You have the option to set it on the
environment and per operator.
If you don't configure anything, we compute somewhat sane defaults in
order to allow scaling up later:
Parallelism 1-64 => default max parallelism: 128
Parallelism 65+ => parallelism + parallelism/2 (where I parallelism is
rounded up to the next power of two)
The maximum allowed max parallelism is 32768.
The computation happens here:
https://github.com/apache/flink/blob/master/flink-runtime/src/main/java/org/apache/flink/runtime/state/KeyGroupRangeAssignment.java#L126.
– Ufuk
On Wed, Jan 25, 2017 at 9:05 AM, Gyula Fóra <
[hidden email]> wrote:
> Hi all,
>
> I can't seem to find in the documentation how to set the maximum parallelism
> for rescaling keyed state.
>
> Can anyone help me out here?
>
> Thanks!
> Gyula
Locked
