Custom java home on YARN

[Paul Lam]

Hi,

I’m trying to run Flink on a YARN cluster that’s running on JDK 7, and I think it’s a quite common scenario, but it seems that currently there’s no way to pass the JAVA_HOME environment variable to YARN.

Am I missing something? And should I create an issue requesting for that?

Best,
Paul Lam

[vino yang]

Hi Paul,

You can try: -yD yarn.taskmanager.env.JAVA_HOME=xx in the command line.

Thanks, vino.

Paul Lam <[hidden email]> 于2018年8月29日周三 下午3:27写道：

Hi,

I’m trying to run Flink on a YARN cluster that’s running on JDK 7, and I think it’s a quite common scenario, but it seems that currently there’s no way to pass the JAVA_HOME environment variable to YARN.

Am I missing something? And should I create an issue requesting for that?

Best,
Paul Lam

[Paul Lam]

Hi vino,

Thanks for your suggestion. I believe it’s working. But I still get this exception in jobmanager.err:

> java.lang.UnsupportedClassVersionError: org/apache/flink/yarn/entrypoint/YarnSessionClusterEntrypoint : Unsupported major.minor version 52.0

I tried setting Java home for the jobmanager too, but it doesn’t help.

Do you have any idea on this? Thanks a lot!

Best,

Paul Lam

在 2018年8月29日，16:08，vino yang <[hidden email]> 写道：

Hi Paul,

You can try: -yD yarn.taskmanager.env.JAVA_HOME=xx in the command line.

Thanks, vino.

Paul Lam <[hidden email]> 于2018年8月29日周三 下午3:27写道：

Hi,

I’m trying to run Flink on a YARN cluster that’s running on JDK 7, and I think it’s a quite common scenario, but it seems that currently there’s no way to pass the JAVA_HOME environment variable to YARN.

Am I missing something? And should I create an issue requesting for that?

Best,
Paul Lam

[vino yang]

Hi Paul,

This exception means that the jdk version of the execution code is lower than the compiled jdk version. 

52 means that the JDK version that compiles it is 1.8. 

There is information that Flink 1.4 has removed support for Java 7, so your current best choice is to use JDK 1.8.[1]

[1]: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-1-4-0-can-not-override-JAVA-HOME-for-single-job-deployment-on-YARN-td17265.html

Thanks, vino.

Paul Lam <[hidden email]> 于2018年8月30日周四 上午9:58写道：

Hi vino,

Thanks for your suggestion. I believe it’s working. But I still get this exception in jobmanager.err:

> java.lang.UnsupportedClassVersionError: org/apache/flink/yarn/entrypoint/YarnSessionClusterEntrypoint : Unsupported major.minor version 52.0

I tried setting Java home for the jobmanager too, but it doesn’t help.

Do you have any idea on this? Thanks a lot!

Best,

Paul Lam

在 2018年8月29日，16:08，vino yang <[hidden email]> 写道：

Hi Paul,

You can try: -yD yarn.taskmanager.env.JAVA_HOME=xx in the command line.

Thanks, vino.

Paul Lam <[hidden email]> 于2018年8月29日周三 下午3:27写道：

Hi,

I’m trying to run Flink on a YARN cluster that’s running on JDK 7, and I think it’s a quite common scenario, but it seems that currently there’s no way to pass the JAVA_HOME environment variable to YARN.

Am I missing something? And should I create an issue requesting for that?

Best,
Paul Lam

[Paul Lam]

Hi vino,

Yes, I’m trying to set Java 8 environment for Flink job, but YARN is running on Java 7 which I can’t change.

I find the solution in the mail list you referred to. It turns out that I was using a wrong key to set Java home for the jobmanager, and the right key should be “containerized.master.env.JAVA_HOME”.

Thank you very much!

Best,

Paul Lam

在 2018年8月30日，10:09，vino yang <[hidden email]> 写道：

Hi Paul,

This exception means that the jdk version of the execution code is lower than the compiled jdk version. 

52 means that the JDK version that compiles it is 1.8. 

There is information that Flink 1.4 has removed support for Java 7, so your current best choice is to use JDK 1.8.[1]

[1]: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-1-4-0-can-not-override-JAVA-HOME-for-single-job-deployment-on-YARN-td17265.html

Thanks, vino.

Paul Lam <[hidden email]> 于2018年8月30日周四 上午9:58写道：

Hi vino,

Thanks for your suggestion. I believe it’s working. But I still get this exception in jobmanager.err:

> java.lang.UnsupportedClassVersionError: org/apache/flink/yarn/entrypoint/YarnSessionClusterEntrypoint : Unsupported major.minor version 52.0

I tried setting Java home for the jobmanager too, but it doesn’t help.

Do you have any idea on this? Thanks a lot!

Best,

Paul Lam

在 2018年8月29日，16:08，vino yang <[hidden email]> 写道：

Hi Paul,

You can try: -yD yarn.taskmanager.env.JAVA_HOME=xx in the command line.

Thanks, vino.

Paul Lam <[hidden email]> 于2018年8月29日周三 下午3:27写道：

Hi,

I’m trying to run Flink on a YARN cluster that’s running on JDK 7, and I think it’s a quite common scenario, but it seems that currently there’s no way to pass the JAVA_HOME environment variable to YARN.

Am I missing something? And should I create an issue requesting for that?

Best,
Paul Lam

[vino yang]

Glad to hear this message!

Paul Lam <[hidden email]> 于2018年8月30日周四 上午10:35写道：

Hi vino,

Yes, I’m trying to set Java 8 environment for Flink job, but YARN is running on Java 7 which I can’t change.

I find the solution in the mail list you referred to. It turns out that I was using a wrong key to set Java home for the jobmanager, and the right key should be “containerized.master.env.JAVA_HOME”.

Thank you very much!

Best,

Paul Lam

在 2018年8月30日，10:09，vino yang <[hidden email]> 写道：

Hi Paul,

This exception means that the jdk version of the execution code is lower than the compiled jdk version. 

52 means that the JDK version that compiles it is 1.8. 

There is information that Flink 1.4 has removed support for Java 7, so your current best choice is to use JDK 1.8.[1]

[1]: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-1-4-0-can-not-override-JAVA-HOME-for-single-job-deployment-on-YARN-td17265.html

Thanks, vino.

Paul Lam <[hidden email]> 于2018年8月30日周四 上午9:58写道：

Hi vino,

Thanks for your suggestion. I believe it’s working. But I still get this exception in jobmanager.err:

> java.lang.UnsupportedClassVersionError: org/apache/flink/yarn/entrypoint/YarnSessionClusterEntrypoint : Unsupported major.minor version 52.0

I tried setting Java home for the jobmanager too, but it doesn’t help.

Do you have any idea on this? Thanks a lot!

Best,

Paul Lam

在 2018年8月29日，16:08，vino yang <[hidden email]> 写道：

Hi Paul,

You can try: -yD yarn.taskmanager.env.JAVA_HOME=xx in the command line.

Thanks, vino.

Paul Lam <[hidden email]> 于2018年8月29日周三 下午3:27写道：

Hi,

I’m trying to run Flink on a YARN cluster that’s running on JDK 7, and I think it’s a quite common scenario, but it seems that currently there’s no way to pass the JAVA_HOME environment variable to YARN.

Am I missing something? And should I create an issue requesting for that?

Best,
Paul Lam

[Paul Lam]

Hi tison,

This information is very helpful. Thank you!

Best,

Paul Lam

在 2018年8月30日，10:41，陈梓立 <[hidden email]> 写道：

Hi Paul,

For your information,  `yarn.taskmanager.env.JAVA_HOME` is deprecated, you can set env of taskmanager using the key similar to jobmanager, that is `containerized.taskmanager.env.JAVA_HOME`

All args with prefix `containerized.taskmanager.env.` would be passed to taskmanager, similar to jobmanager.

Paul Lam <[hidden email]> 于2018年8月30日周四 上午10:35写道：

Hi vino,

Yes, I’m trying to set Java 8 environment for Flink job, but YARN is running on Java 7 which I can’t change.

I find the solution in the mail list you referred to. It turns out that I was using a wrong key to set Java home for the jobmanager, and the right key should be “containerized.master.env.JAVA_HOME”.

Thank you very much!

Best,

Paul Lam

在 2018年8月30日，10:09，vino yang <[hidden email]> 写道：

Hi Paul,

This exception means that the jdk version of the execution code is lower than the compiled jdk version. 

52 means that the JDK version that compiles it is 1.8. 

There is information that Flink 1.4 has removed support for Java 7, so your current best choice is to use JDK 1.8.[1]

[1]: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-1-4-0-can-not-override-JAVA-HOME-for-single-job-deployment-on-YARN-td17265.html

Thanks, vino.

Paul Lam <[hidden email]> 于2018年8月30日周四 上午9:58写道：

Hi vino,

Thanks for your suggestion. I believe it’s working. But I still get this exception in jobmanager.err:

> java.lang.UnsupportedClassVersionError: org/apache/flink/yarn/entrypoint/YarnSessionClusterEntrypoint : Unsupported major.minor version 52.0

I tried setting Java home for the jobmanager too, but it doesn’t help.

Do you have any idea on this? Thanks a lot!

Best,

Paul Lam

在 2018年8月29日，16:08，vino yang <[hidden email]> 写道：

Hi Paul,

You can try: -yD yarn.taskmanager.env.JAVA_HOME=xx in the command line.

Thanks, vino.

Paul Lam <[hidden email]> 于2018年8月29日周三 下午3:27写道：

Hi,

I’m trying to run Flink on a YARN cluster that’s running on JDK 7, and I think it’s a quite common scenario, but it seems that currently there’s no way to pass the JAVA_HOME environment variable to YARN.

Am I missing something? And should I create an issue requesting for that?

Best,
Paul Lam