How to verify what maxParallelism is set to?

[Sean Bollin]

Hi all,

How do you verify what max parallelism is set to on the job level? I do not see it in the 1.6 UI, for example.

I’m setting maxParallelism to 4096 on the StreamExecutionEnvironment before execution but printing out the maxParallelism in an operator still displays -1. Since this is such an important setting I’d like some sanity check to verify it is the value I expect.

Any API or way I can verify?

[Guowei Ma]

Hi,

StreamExecutionEnvironment is used to set a default maxParallelism for global. If a "operator"'s maxParallelism is -1 the operator will be set the maxParallelism which is set by StreamExecutionEnvironment.

>>>Any API or way I can verify?

I can't find any easy way to do that. But you could use get the StreamGraph from api StreamExecutionEnvironment.getStreamGraph and try to print the StreamNode::maxParallelism.

Sean Bollin <[hidden email]> 于2019年4月30日周二 上午8:19写道：

Hi all,

How do you verify what max parallelism is set to on the job level? I do not see it in the 1.6 UI, for example.

I’m setting maxParallelism to 4096 on the StreamExecutionEnvironment before execution but printing out the maxParallelism in an operator still displays -1. Since this is such an important setting I’d like some sanity check to verify it is the value I expect.

Any API or way I can verify?

[Sean Bollin]

Thanks! Do you know if it's possible somehow to verify the global
maxParallelism other than calling .getMaxParallelism? Either through
an API call or the UI?

On Mon, Apr 29, 2019 at 8:12 PM Guowei Ma <[hidden email]> wrote:

>
> Hi,
> StreamExecutionEnvironment is used to set a default maxParallelism for global. If a "operator"'s maxParallelism is -1 the operator will be set the maxParallelism which is set by StreamExecutionEnvironment.
>
> >>>Any API or way I can verify?
> I can't find any easy way to do that. But you could use get the StreamGraph from api StreamExecutionEnvironment.getStreamGraph and try to print the StreamNode::maxParallelism.
> Best,
> Guowei
>
>
> Sean Bollin <[hidden email]> 于2019年4月30日周二 上午8:19写道：
>>
>> Hi all,
>>
>> How do you verify what max parallelism is set to on the job level? I do not see it in the 1.6 UI, for example.
>>
>> I’m setting maxParallelism to 4096 on the StreamExecutionEnvironment before execution but printing out the maxParallelism in an operator still displays -1. Since this is such an important setting I’d like some sanity check to verify it is the value I expect.
>>
>> Any API or way I can verify?

[Fabian Hueske-2]

Hi Sean,

I was looking for the max-parallelism value in the UI, but couldn't find it. Also the REST API does not seem to provide it.

Would you mind opening a Jira issue for adding it to the REST API and the Web UI?

Thank you,

Fabian

Am Di., 30. Apr. 2019 um 06:36 Uhr schrieb Sean Bollin <[hidden email]>:

Thanks! Do you know if it's possible somehow to verify the global
maxParallelism other than calling .getMaxParallelism? Either through
an API call or the UI?

On Mon, Apr 29, 2019 at 8:12 PM Guowei Ma <[hidden email]> wrote:
>
> Hi,
> StreamExecutionEnvironment is used to set a default maxParallelism for global. If a "operator"'s maxParallelism is -1 the operator will be set the maxParallelism which is set by StreamExecutionEnvironment.
>
> >>>Any API or way I can verify?
> I can't find any easy way to do that. But you could use get the StreamGraph from api StreamExecutionEnvironment.getStreamGraph and try to print the StreamNode::maxParallelism.
> Best,
> Guowei
>
>
> Sean Bollin <[hidden email]> 于2019年4月30日周二 上午8:19写道：
>>
>> Hi all,
>>
>> How do you verify what max parallelism is set to on the job level? I do not see it in the 1.6 UI, for example.
>>
>> I’m setting maxParallelism to 4096 on the StreamExecutionEnvironment before execution but printing out the maxParallelism in an operator still displays -1. Since this is such an important setting I’d like some sanity check to verify it is the value I expect.
>>
>> Any API or way I can verify?

[Bob Tiernay]

I agree that a way to introspect the effective current value would be a great
observability tool for sanity checking.

Fabian, do you know if a ticket was ever created?



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/