Flink Yarn Cluster specifying Job Manager IP

[arpit8622]

Hi,

I have yarn cluster with 7 nodes.

Five nodes  -16gb ram

One node - 8gb ram

One yarn resourcemanager - 8gb

I want to specifically use 8gb machine (not resourcemanager) to act as job manager and other five nodes as task managers.

Is there a way to do that ?

[Alexis Gendronneau]

Hi Arpit,

I'm not sure of what you tries to do, but if you want yarn to execute a type of job on particular nodes, you may find a way using node labels : 

https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/NodeLabel.html

I'm not sure you'll be able to specify that the jobmanager has to be run on a specific node, but your flink cluster would run on the 5 or 6 nodes you'll provide for your dedicated queue. 

If you manage to do so could you send some feedback about how you did it ? 

Regards

2016-05-30 21:44 GMT+02:00 arpit srivastava <[hidden email]>:

Hi,

I have yarn cluster with 7 nodes.

Five nodes  -16gb ram

One node - 8gb ram

One yarn resourcemanager - 8gb

I want to specifically use 8gb machine (not resourcemanager) to act as job manager and other five nodes as task managers.

Is there a way to do that ?

--

[arpit8622]

Will try and come back with feedback if its possible.

Thanks,

Arpit

On Tue, May 31, 2016 at 12:00 PM, Alexis Gendronneau <[hidden email]> wrote:

Hi Arpit,

I'm not sure of what you tries to do, but if you want yarn to execute a type of job on particular nodes, you may find a way using node labels : 

https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/NodeLabel.html

I'm not sure you'll be able to specify that the jobmanager has to be run on a specific node, but your flink cluster would run on the 5 or 6 nodes you'll provide for your dedicated queue. 

If you manage to do so could you send some feedback about how you did it ? 

Regards

2016-05-30 21:44 GMT+02:00 arpit srivastava <[hidden email]>:

Hi,

I have yarn cluster with 7 nodes.

Five nodes  -16gb ram

One node - 8gb ram

One yarn resourcemanager - 8gb

I want to specifically use 8gb machine (not resourcemanager) to act as job manager and other five nodes as task managers.

Is there a way to do that ?

--