override jvm params

[Cussac, Franck]

Hi,

 

Following the documentation I want to use –yD option to override some params in my conf like this :

 

flink run -m $HOSTPORT -yD "env.java.opts.taskmanager=-Drules.consumer.topic=test" -yD "env.java.opts.jobmanager=-Drules.consumer.topic=test" myjar mymain

 

but it is just ignored. Nothing happend. But if I run with java on my IDE and :

-Drules.consumer.topic=test

in JVM’s parameter it works eprfectly.

 

What do I have to do to override my params with yarn and flink ?

 

 

Best regards,

Franck Cussac.

 

[Hequn Cheng]

Hi Cussac,

If I understand correctly, you want to pass rules.consumer.topic=test and rules.consumer.topic=test to flink jvm.

I think you can try: 

flink run -m $HOSTPORT -yD rules.consumer.topic=test -yD rules.consumer.topic=test 

Hope this helps.

Hequn

On Wed, Jul 25, 2018 at 3:26 PM, Cussac, Franck <[hidden email]> wrote:

Hi,

 

Following the documentation I want to use –yD option to override some params in my conf like this :

 

flink run -m $HOSTPORT -yD "env.java.opts.taskmanager=-Drules.consumer.topic=test" -yD "env.java.opts.jobmanager=-Drules.consumer.topic=test" myjar mymain

 

but it is just ignored. Nothing happend. But if I run with java on my IDE and :

-Drules.consumer.topic=test

in JVM’s parameter it works eprfectly.

 

What do I have to do to override my params with yarn and flink ?

 

 

Best regards,

Franck Cussac.

 

[Cussac, Franck]

Hi Hequn,

 

Thanks for your answer. I just tested and it doesn’t work.

 

I’m using PureConfig to parse my conf files. With java I can override any argument using –D<param>=<value> syntax. How can I do same with flink in yarn mode ?

 

Franck.

 

 

De : Hequn Cheng [mailto:[hidden email]]
Envoyé : mercredi 25 juillet 2018 14:04
À : Cussac, Franck <[hidden email]>
Cc : user <[hidden email]>
Objet : Re: override jvm params

 

Hi Cussac,

If I understand correctly, you want to pass rules.consumer.topic=test and rules.consumer.topic=test to flink jvm.

I think you can try: 

flink run -m $HOSTPORT -yD rules.consumer.topic=test -yD rules.consumer.topic=test 

 

Hope this helps.

Hequn

 

On Wed, Jul 25, 2018 at 3:26 PM, Cussac, Franck <[hidden email]> wrote:

Hi,

 

Following the documentation I want to use –yD option to override some params in my conf like this :

 

flink run -m $HOSTPORT -yD "env.java.opts.taskmanager=-Drules.consumer.topic=test" -yD "env.java.opts.jobmanager=-Drules.consumer.topic=test" myjar mymain

 

but it is just ignored. Nothing happend. But if I run with java on my IDE and :

-Drules.consumer.topic=test

in JVM’s parameter it works eprfectly.

 

What do I have to do to override my params with yarn and flink ?

 

 

Best regards,

Franck Cussac.

 

 

[vino yang]

Hi Cussac,

Flink on Yarn support dynamic properties. Can you try this : -yD<key>=<value>?

The implementation is here[1][2].

[1]: https://github.com/apache/flink/blob/master/flink-yarn/src/main/java/org/apache/flink/yarn/cli/FlinkYarnSessionCli.java#L151

[2]: https://github.com/apache/flink/blob/master/flink-yarn/src/main/java/org/apache/flink/yarn/cli/FlinkYarnSessionCli.java#L198

Thanks, vino.

2018-07-25 20:27 GMT+08:00 Cussac, Franck <[hidden email]>:

Hi Hequn,

 

Thanks for your answer. I just tested and it doesn’t work.

 

I’m using PureConfig to parse my conf files. With java I can override any argument using –D<param>=<value> syntax. How can I do same with flink in yarn mode ?

 

Franck.

 

 

De : Hequn Cheng [mailto:[hidden email]]
Envoyé : mercredi 25 juillet 2018 14:04
À : Cussac, Franck <[hidden email]>
Cc : user <[hidden email]>
Objet : Re: override jvm params

 

Hi Cussac,

If I understand correctly, you want to pass rules.consumer.topic=test and rules.consumer.topic=test to flink jvm.

I think you can try: 

flink run -m $HOSTPORT -yD rules.consumer.topic=test -yD rules.consumer.topic=test 

 

Hope this helps.

Hequn

 

On Wed, Jul 25, 2018 at 3:26 PM, Cussac, Franck <[hidden email]> wrote:

Hi,

 

Following the documentation I want to use –yD option to override some params in my conf like this :

 

flink run -m $HOSTPORT -yD "env.java.opts.taskmanager=-Drules.consumer.topic=test" -yD "env.java.opts.jobmanager=-Drules.consumer.topic=test" myjar mymain

 

but it is just ignored. Nothing happend. But if I run with java on my IDE and :

-Drules.consumer.topic=test

in JVM’s parameter it works eprfectly.

 

What do I have to do to override my params with yarn and flink ?

 

 

Best regards,

Franck Cussac.