Flink Task Slots

[Vijayendra Yadav]

Hi Team,

https://ci.apache.org/projects/flink/flink-docs-release-1.9/ops/config.html#configuring-taskmanager-processing-slots 

 In my Flinkkafkaconsumer streaming process. I have kafka with 3 partitions. 

I am getting parallelism = 3  (-p 3). Here 3 taskmangers are launched but each taskmanager gets only one slot. 

When I try to provide property in flink yaml. as : 

taskmanager.numberOfTaskSlots: 6 

Flink on a yarn cluster launches only when taskmanager with specified slots. 

While I was expecting it to still launch 3 taskmanagers and just allocate more slots to each of them.

Could you please suggest how to allocate more slots to each task manager without reducing the number of taskmangers ?

Regards,

Vijay

[Vijayendra Yadav]

Please ignore, last Email, Its working now by adding more parallelism.

On Fri, Aug 28, 2020 at 12:39 PM Vijayendra Yadav <[hidden email]> wrote:

Hi Team,

https://ci.apache.org/projects/flink/flink-docs-release-1.9/ops/config.html#configuring-taskmanager-processing-slots 

 In my Flinkkafkaconsumer streaming process. I have kafka with 3 partitions. 

I am getting parallelism = 3  (-p 3). Here 3 taskmangers are launched but each taskmanager gets only one slot. 

When I try to provide property in flink yaml. as : 

taskmanager.numberOfTaskSlots: 6 

Flink on a yarn cluster launches only when taskmanager with specified slots. 

While I was expecting it to still launch 3 taskmanagers and just allocate more slots to each of them.

Could you please suggest how to allocate more slots to each task manager without reducing the number of taskmangers ?

Regards,

Vijay