Task Slots allocation

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Task Slots allocation

vijayakumarpl
I am using flink 1.4.0 in standalone cluster mode

I have a job with a graph like a Source(parallelism 3) -> Filter(parallelism 3) -> Map(parallelism 1) -> sink(parallelism 3)

As per the understanding, max number of tasks slots should be bounded by the max parallelism, which in this case is 3.

The behavior i observe is that, when more than required number of slots(say 10) are available in cluster, this job uses 5 slots. I attempted setting the same slotsharinggroup to all streamnodes, but didn't help.

Is this a change in behavior in 1.4.0?. If so are there any workarounds to force the old behavior



--
Thanks,
-Vijay
Reply | Threaded
Open this post in threaded view
|

Re: Task Slots allocation

Aljoscha Krettek
Hi,

Could you maybe send a screenshot from the dashboard that shows how tasks are being assigned to slots?

Aljoscha

> On 4. Mar 2018, at 18:06, vijayakumar palaniappan <[hidden email]> wrote:
>
> I am using flink 1.4.0 in standalone cluster mode
>
> I have a job with a graph like a Source(parallelism 3) -> Filter(parallelism 3) -> Map(parallelism 1) -> sink(parallelism 3)
>
> As per the understanding, max number of tasks slots should be bounded by the max parallelism, which in this case is 3.
>
> The behavior i observe is that, when more than required number of slots(say 10) are available in cluster, this job uses 5 slots. I attempted setting the same slotsharinggroup to all streamnodes, but didn't help.
>
> Is this a change in behavior in 1.4.0?. If so are there any workarounds to force the old behavior
>
>
>
> --
> Thanks,
> -Vijay