Task manager count goes the expand then converge process when running flink on YARN

[徐涛]

Hi experts

I am running flink job on YARN in job cluster mode, the job is divided into 2 tasks, the following are some configs of the job:

parallelism.default => 16

taskmanager.numberOfTaskSlots => 8

-yn => 2

when the program starts, I found that the count of task managers is not set immediately, but first expand then converge, I record the number during the process:

Task Managers         Task Slots         Available Task Slots

1. 14  104                    88

        2. 15                             120                    104

        3. 16                             128                    112

        4. 6                               48                      32

        5. 3                               24                      8

        6. 2                               16                      0

The final state is correct. There are 2 tasks, 32 subtask in total, due to slot sharing, only 16 slots are enough, the number of task slots per TM are 8, so 2 TMs are needed.

I have the following question:

Because I specify yn=2, why does not directly allocate 2 TMs, but goes the expand then converge process?  Why does it apply 16 task managers at most? If it is not a must, how to avoid it?

Thanks a lot!

Best

Henry