Flink 1.4.0 can not override JAVA_HOME for single-job deployment on YARN

[sanmutongzi]

Hi,
I am usring flink single-job mode on YARN. After i upgrade flink
verson from 1.3.2 to  1.4.0, the parameter
"yarn.taskmanager.env.JAVA_HOME" doesn’t work  as before.
I can only found error log on yarn like this:

Exception in thread "main" java.lang.UnsupportedClassVersionError:
org/apache/flink/yarn/YarnApplicationMasterRunner : Unsupported
major.minor version 52.0
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:482)

Is there something different i should know  to avoid this problem ?
Thanks!

[Nico Kruber]

Hi,
are you running Flink in an JRE >= 8? We dropped Java 7 support for
Flink 1.4.


Nico

On 14/12/17 12:35, 杨光 wrote:

> Hi,
> I am usring flink single-job mode on YARN. After i upgrade flink
> verson from 1.3.2 to  1.4.0, the parameter
> "yarn.taskmanager.env.JAVA_HOME" doesn’t work  as before.
> I can only found error log on yarn like this:
>
> Exception in thread "main" java.lang.UnsupportedClassVersionError:
> org/apache/flink/yarn/YarnApplicationMasterRunner : Unsupported
> major.minor version 52.0
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
> at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
> at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:482)
>
> Is there something different i should know  to avoid this problem ?
> Thanks!
>

[sanmutongzi]

Yes , i'm using Java8 , and i found the 1.4  version  provided  new
parameters : "containerized.master.env.ENV_VAR1" and
"containerized.taskmanager.env".
I change my start command from "-yD yarn.taskmanager.env.JAVA_HOME" to
" -yD containerized.taskmanager.env.JAVA_HOME=/opt/jdk1.8.0_121 -yD
containerized.master.env.JAVA_HOME=/opt/jdk1.8.0_121 " and it  works
.
Thanks a lot  .

2017-12-14 20:52 GMT+08:00 Nico Kruber <[hidden email]>:

> Hi,
> are you running Flink in an JRE >= 8? We dropped Java 7 support for
> Flink 1.4.
>
>
> Nico
>
> On 14/12/17 12:35, 杨光 wrote:
>> Hi,
>> I am usring flink single-job mode on YARN. After i upgrade flink
>> verson from 1.3.2 to  1.4.0, the parameter
>> "yarn.taskmanager.env.JAVA_HOME" doesn’t work  as before.
>> I can only found error log on yarn like this:
>>
>> Exception in thread "main" java.lang.UnsupportedClassVersionError:
>> org/apache/flink/yarn/YarnApplicationMasterRunner : Unsupported
>> major.minor version 52.0
>> at java.lang.ClassLoader.defineClass1(Native Method)
>> at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
>> at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
>> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
>> at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>> at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:482)
>>
>> Is there something different i should know  to avoid this problem ?
>> Thanks!
>>
>

[Fabian Hueske-2]

Thanks for reporting back!

2017-12-15 4:52 GMT+01:00 杨光 <[hidden email]>:

Yes , i'm using Java8 , and i found the 1.4  version  provided  new
parameters : "containerized.master.env.ENV_VAR1" and
"containerized.taskmanager.env".
I change my start command from "-yD yarn.taskmanager.env.JAVA_HOME" to
" -yD containerized.taskmanager.env.JAVA_HOME=/opt/jdk1.8.0_121 -yD
containerized.master.env.JAVA_HOME=/opt/jdk1.8.0_121 " and it  works
.
Thanks a lot  .

2017-12-14 20:52 GMT+08:00 Nico Kruber <[hidden email]>:
> Hi,
> are you running Flink in an JRE >= 8? We dropped Java 7 support for
> Flink 1.4.
>
>
> Nico
>
> On 14/12/17 12:35, 杨光 wrote:
>> Hi,
>> I am usring flink single-job mode on YARN. After i upgrade flink
>> verson from 1.3.2 to  1.4.0, the parameter
>> "yarn.taskmanager.env.JAVA_HOME" doesn’t work  as before.
>> I can only found error log on yarn like this:
>>
>> Exception in thread "main" java.lang.UnsupportedClassVersionError:
>> org/apache/flink/yarn/YarnApplicationMasterRunner : Unsupported
>> major.minor version 52.0
>> at java.lang.ClassLoader.defineClass1(Native Method)
>> at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
>> at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
>> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
>> at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>> at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:482)
>>
>> Is there something different i should know  to avoid this problem ?
>> Thanks!
>>
>

[Stephan Ewen]

Could you open an issue to add the old config keys as backwards supported "deprecated keys"? That should help making the transition smoother.

On Fri, Dec 15, 2017 at 9:29 AM, Fabian Hueske <[hidden email]> wrote:

Thanks for reporting back!

2017-12-15 4:52 GMT+01:00 杨光 <[hidden email]>:

Yes , i'm using Java8 , and i found the 1.4  version  provided  new
parameters : "containerized.master.env.ENV_VAR1" and
"containerized.taskmanager.env".
I change my start command from "-yD yarn.taskmanager.env.JAVA_HOME" to
" -yD containerized.taskmanager.env.JAVA_HOME=/opt/jdk1.8.0_121 -yD
containerized.master.env.JAVA_HOME=/opt/jdk1.8.0_121 " and it  works
.
Thanks a lot  .

2017-12-14 20:52 GMT+08:00 Nico Kruber <[hidden email]>:
> Hi,
> are you running Flink in an JRE >= 8? We dropped Java 7 support for
> Flink 1.4.
>
>
> Nico
>
> On 14/12/17 12:35, 杨光 wrote:
>> Hi,
>> I am usring flink single-job mode on YARN. After i upgrade flink
>> verson from 1.3.2 to  1.4.0, the parameter
>> "yarn.taskmanager.env.JAVA_HOME" doesn’t work  as before.
>> I can only found error log on yarn like this:
>>
>> Exception in thread "main" java.lang.UnsupportedClassVersionError:
>> org/apache/flink/yarn/YarnApplicationMasterRunner : Unsupported
>> major.minor version 52.0
>> at java.lang.ClassLoader.defineClass1(Native Method)
>> at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
>> at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
>> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
>> at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>> at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:482)
>>
>> Is there something different i should know  to avoid this problem ?
>> Thanks!
>>
>

[sanmutongzi]

Hi  Stephan
I tested my case  again and  found the  parameters "-yD
containerized.taskmanager.env.JAVA_HOME=/opt/jdk1.8.0_121   -yD
containerized.master.env.JAVA_HOME=/opt/jdk1.8.0_121" and "-yD
yarn.taskmanager.env.JAVA_HOME=/opt/jdk1.8.0_121  -yD
containerized.master.env.JAVA_HOME=/opt/jdk1.8.0_121 "  can both solve
the JRE8 error  . Looks like the "yarn.taskmanager.env." is already
backwards . So my early problem is caused by the low version java for
the jobmanager process only . Do you  think  the issue is still
necessary?
Thanks .

2017-12-15 19:32 GMT+08:00 Stephan Ewen <[hidden email]>:

> Could you open an issue to add the old config keys as backwards supported
> "deprecated keys"? That should help making the transition smoother.
>
> On Fri, Dec 15, 2017 at 9:29 AM, Fabian Hueske <[hidden email]> wrote:
>>
>> Thanks for reporting back!
>>
>> 2017-12-15 4:52 GMT+01:00 杨光 <[hidden email]>:
>>>
>>> Yes , i'm using Java8 , and i found the 1.4  version  provided  new
>>> parameters : "containerized.master.env.ENV_VAR1" and
>>> "containerized.taskmanager.env".
>>> I change my start command from "-yD yarn.taskmanager.env.JAVA_HOME" to
>>> " -yD containerized.taskmanager.env.JAVA_HOME=/opt/jdk1.8.0_121 -yD
>>> containerized.master.env.JAVA_HOME=/opt/jdk1.8.0_121 " and it  works
>>> .
>>> Thanks a lot  .
>>>
>>> 2017-12-14 20:52 GMT+08:00 Nico Kruber <[hidden email]>:
>>> > Hi,
>>> > are you running Flink in an JRE >= 8? We dropped Java 7 support for
>>> > Flink 1.4.
>>> >
>>> >
>>> > Nico
>>> >
>>> > On 14/12/17 12:35, 杨光 wrote:
>>> >> Hi,
>>> >> I am usring flink single-job mode on YARN. After i upgrade flink
>>> >> verson from 1.3.2 to  1.4.0, the parameter
>>> >> "yarn.taskmanager.env.JAVA_HOME" doesn’t work  as before.
>>> >> I can only found error log on yarn like this:
>>> >>
>>> >> Exception in thread "main" java.lang.UnsupportedClassVersionError:
>>> >> org/apache/flink/yarn/YarnApplicationMasterRunner : Unsupported
>>> >> major.minor version 52.0
>>> >> at java.lang.ClassLoader.defineClass1(Native Method)
>>> >> at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
>>> >> at
>>> >> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
>>> >> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
>>> >> at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
>>> >> at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
>>> >> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>>> >> at java.security.AccessController.doPrivileged(Native Method)
>>> >> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>>> >> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>>> >> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
>>> >> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>>> >> at
>>> >> sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:482)
>>> >>
>>> >> Is there something different i should know  to avoid this problem ?
>>> >> Thanks!
>>> >>
>>> >
>>
>>
>