Yarn run single job

[Garrett Barton]

Greetings all,

 The docs say that I can skip creating a cluster and let the jobs create their own clusters on yarn.  The example given is:

./bin/flink run -m yarn-cluster -yn 2 ./examples/batch/WordCount.jar

What I cannot figure out is what the -m option is meant for.  In my opinion there is no jobmanager to specify, I expect flink to start one.  Skipping the option doesn't work as it defaults to the conf one which has a comment saying flink manages it for yarn deployments.

I tried pointing it at my yarn resource manager, it didn't like any of the ports.

Any ideas?

[Chesnay Schepler]

-m yarn-cluster switches the client into yarn mode.

yarn-cluster is not a placeholder or anything, you have to literally type that in.

On 10.07.2018 17:02, Garrett Barton wrote:

Greetings all,

 The docs say that I can skip creating a cluster and let the jobs create their own clusters on yarn.  The example given is:

./bin/flink run -m yarn-cluster -yn 2 ./examples/batch/WordCount.jar

What I cannot figure out is what the -m option is meant for.  In my opinion there is no jobmanager to specify, I expect flink to start one.  Skipping the option doesn't work as it defaults to the conf one which has a comment saying flink manages it for yarn deployments.

I tried pointing it at my yarn resource manager, it didn't like any of the ports.

Any ideas?

[Garrett Barton]

AHH it works!  Never occurred to me that it meant literally type in yarn-cluster.

Thank you!

On Tue, Jul 10, 2018 at 11:17 AM Chesnay Schepler <[hidden email]> wrote:

-m yarn-cluster switches the client into yarn mode.

yarn-cluster is not a placeholder or anything, you have to literally type that in.

On 10.07.2018 17:02, Garrett Barton wrote:

Greetings all,

 The docs say that I can skip creating a cluster and let the jobs create their own clusters on yarn.  The example given is:

./bin/flink run -m yarn-cluster -yn 2 ./examples/batch/WordCount.jar

What I cannot figure out is what the -m option is meant for.  In my opinion there is no jobmanager to specify, I expect flink to start one.  Skipping the option doesn't work as it defaults to the conf one which has a comment saying flink manages it for yarn deployments.

I tried pointing it at my yarn resource manager, it didn't like any of the ports.

Any ideas?