Limit number of jobs or Job Managers

[Pankaj Chand]

Hi everyone,

Is there any way (parameter or function) I can limit the number of concurrent jobs executing in my Flink cluster? Or alternatively, limit the number of concurrent Job Managers (since there has to be one Job Manager for every job)?

Thanks!

Pankaj

[Haibo Sun]

Hi,  Pankaj Chand 

If you're running Flink on YARN, you can do this by limiting the number of applications in the cluster or in the queue. As far as I know, Flink does not limit that.

The following are the configuration items for  YARN :

    yarn.scheduler.capacity.maximum-applications

    yarn.scheduler.capacity.<queue-path>.maximum-applications

Best,

Haibo

At 2019-06-27 20:55:48, "Pankaj Chand" <[hidden email]> wrote:

Hi everyone,

Is there any way (parameter or function) I can limit the number of concurrent jobs executing in my Flink cluster? Or alternatively, limit the number of concurrent Job Managers (since there has to be one Job Manager for every job)?

Thanks!

Pankaj

[Pankaj Chand]

Hi Haibo Sun,

It's exactly what I needed. Thank you so much! 

Best,

Pankaj

On Thu, Jun 27, 2019 at 7:45 AM Haibo Sun <[hidden email]> wrote:

Hi,  Pankaj Chand 

If you're running Flink on YARN, you can do this by limiting the number of applications in the cluster or in the queue. As far as I know, Flink does not limit that.

The following are the configuration items for  YARN :

    yarn.scheduler.capacity.maximum-applications

    yarn.scheduler.capacity.<queue-path>.maximum-applications

Best,

Haibo

At 2019-06-27 20:55:48, "Pankaj Chand" <[hidden email]> wrote:

Hi everyone,

Is there any way (parameter or function) I can limit the number of concurrent jobs executing in my Flink cluster? Or alternatively, limit the number of concurrent Job Managers (since there has to be one Job Manager for every job)?

Thanks!

Pankaj