Re: How many task managers to launch for a job?

Posted by Xintong Song on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/How-many-task-managers-to-launch-for-a-job-tp27983p27988.html

Hi black,

If you are running Flink on Yarn or Mesos, Flink will automatically allocate resource and launch new TaskManagers as needed.

If you are using Flink standalone mode, then the easiest way is to enable slot sharing and set all the vertices into the same group (which is by default). In that way, the total slots (or number of TaskManagers if you config on slot for each TaskManager) needed for running the job would be the maximum parallelism of the job graph vertices. Further information on slot sharing could be found here.

On Thu, May 23, 2019 at 11:49 PM black chase <[hidden email]> wrote:

Hi,

I am redesigning the scheduler of the JobManager to place tasks of a job across TaskManagers accroding to a scheduling policy.

I am reading the Flip-6 proposal and found that the common case is "one TaskManager launchs one slot", and "one Flink cluster serves one job". But I did not find how many TaskManagers to launch in a computing node. Is there any common practice for this ?

--