(DEPRECATED) Apache Flink User Mailing List archive.

flink 1.9 conflict jackson version

classic Classic list List threaded Threaded
♦
♦
 4 messages Options
Fanbin Bu
Reply | Threaded
Open this post in threaded view
♦
♦
|

flink 1.9 conflict jackson version

Fanbin Bu
Hi,

After I upgrade flink 1.9, I got the following error message on EMR, it works locally on IntelliJ.

I'm explicitly declaring the dependency as 
implementation 'com.fasterxml.jackson.module:jackson-module-scala_2.11:2.10.1'
and I have
implementation group: 'com.amazonaws', name: 'aws-java-sdk-emr', version: '1.11.595'



java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.ObjectMapper.enable([Lcom/fasterxml/jackson/core/JsonParser$Feature;)Lcom/fasterxml/jackson/databind/ObjectMapper;
	at com.amazonaws.partitions.PartitionsLoader.<clinit>(PartitionsLoader.java:54)
	at com.amazonaws.regions.RegionMetadataFactory.create(RegionMetadataFactory.java:30)
	at com.amazonaws.regions.RegionUtils.initialize(RegionUtils.java:65)
	at com.amazonaws.regions.RegionUtils.getRegionMetadata(RegionUtils.java:53)
	at com.amazonaws.regions.RegionUtils.getRegion(RegionUtils.java:107)
	at com.amazonaws.client.builder.AwsClientBuilder.getRegionObject(AwsClientBuilder.java:256)
	at com.amazonaws.client.builder.AwsClientBuilder.setRegion(AwsClientBuilder.java:460)
	at com.amazonaws.client.builder.AwsClientBuilder.configureMutableProperties(AwsClientBuilder.java:424)
	at com.amazonaws.client.builder.AwsAsyncClientBuilder.build(AwsAsyncClientBuilder.java:80)
	at com.coinbase.util.KmsClient$.getSnowflakeUsernamePassword(KmsClient.scala:21)
	at com.coinbase.ml.RunFlinkJob$.runBatch(RunFlinkJob.scala:94)
	at com.coinbase.ml.RunFlinkJob$.runFlinkJob(RunFlinkJob.scala:38)
	at com.coinbase.ml.FlinkFeatureProcessingJobEntryPoint$.main(CmdLineParser.scala:76)
	at com.coinbase.ml.FlinkFeatureProcessingJobEntryPoint.main(CmdLineParser.scala)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:576)
	at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:438)
	at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:274)
	at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:746)
	at org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:273)
	at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:205)
	at org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1010)
	at org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1083)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1844)
	at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
	at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1083)
ouywl
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re:flink 1.9 conflict jackson version

ouywl
Hi Bu 
   I think It can use mvn-shade-plugin to resolve your problem,  It seem flink-client conflict with your owner jar?

On 12/17/2019 08:10，[hidden email] wrote：
Hi,

After I upgrade flink 1.9, I got the following error message on EMR, it works locally on IntelliJ.

I'm explicitly declaring the dependency as 
implementation 'com.fasterxml.jackson.module:jackson-module-scala_2.11:2.10.1'
and I have
implementation group: 'com.amazonaws', name: 'aws-java-sdk-emr', version: '1.11.595'



java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.ObjectMapper.enable([Lcom/fasterxml/jackson/core/JsonParser$Feature;)Lcom/fasterxml/jackson/databind/ObjectMapper;
	at com.amazonaws.partitions.PartitionsLoader.<clinit>(PartitionsLoader.java:54)
	at com.amazonaws.regions.RegionMetadataFactory.create(RegionMetadataFactory.java:30)
	at com.amazonaws.regions.RegionUtils.initialize(RegionUtils.java:65)
	at com.amazonaws.regions.RegionUtils.getRegionMetadata(RegionUtils.java:53)
	at com.amazonaws.regions.RegionUtils.getRegion(RegionUtils.java:107)
	at com.amazonaws.client.builder.AwsClientBuilder.getRegionObject(AwsClientBuilder.java:256)
	at com.amazonaws.client.builder.AwsClientBuilder.setRegion(AwsClientBuilder.java:460)
	at com.amazonaws.client.builder.AwsClientBuilder.configureMutableProperties(AwsClientBuilder.java:424)
	at com.amazonaws.client.builder.AwsAsyncClientBuilder.build(AwsAsyncClientBuilder.java:80)
	at com.coinbase.util.KmsClient$.getSnowflakeUsernamePassword(KmsClient.scala:21)
	at com.coinbase.ml.RunFlinkJob$.runBatch(RunFlinkJob.scala:94)
	at com.coinbase.ml.RunFlinkJob$.runFlinkJob(RunFlinkJob.scala:38)
	at com.coinbase.ml.FlinkFeatureProcessingJobEntryPoint$.main(CmdLineParser.scala:76)
	at com.coinbase.ml.FlinkFeatureProcessingJobEntryPoint.main(CmdLineParser.scala)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:576)
	at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:438)
	at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:274)
	at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:746)
	at org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:273)
	at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:205)
	at org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1010)
	at org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1083)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1844)
	at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
	at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1083)
anuj.aj07
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: flink 1.9 conflict jackson version

anuj.aj07
Hi Fanbin,

I am facing a similar kind of issue. Let me know if you are able to resolve this issue then please help me also


On Tue, Dec 17, 2019 at 7:50 AM ouywl <[hidden email]> wrote:
Hi Bu 
   I think It can use mvn-shade-plugin to resolve your problem,  It seem flink-client conflict with your owner jar?

On 12/17/2019 08:10，[hidden email] wrote：
Hi,

After I upgrade flink 1.9, I got the following error message on EMR, it works locally on IntelliJ.

I'm explicitly declaring the dependency as 
implementation 'com.fasterxml.jackson.module:jackson-module-scala_2.11:2.10.1'
and I have
implementation group: 'com.amazonaws', name: 'aws-java-sdk-emr', version: '1.11.595'



java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.ObjectMapper.enable([Lcom/fasterxml/jackson/core/JsonParser$Feature;)Lcom/fasterxml/jackson/databind/ObjectMapper;
	at com.amazonaws.partitions.PartitionsLoader.<clinit>(PartitionsLoader.java:54)
	at com.amazonaws.regions.RegionMetadataFactory.create(RegionMetadataFactory.java:30)
	at com.amazonaws.regions.RegionUtils.initialize(RegionUtils.java:65)
	at com.amazonaws.regions.RegionUtils.getRegionMetadata(RegionUtils.java:53)
	at com.amazonaws.regions.RegionUtils.getRegion(RegionUtils.java:107)
	at com.amazonaws.client.builder.AwsClientBuilder.getRegionObject(AwsClientBuilder.java:256)
	at com.amazonaws.client.builder.AwsClientBuilder.setRegion(AwsClientBuilder.java:460)
	at com.amazonaws.client.builder.AwsClientBuilder.configureMutableProperties(AwsClientBuilder.java:424)
	at com.amazonaws.client.builder.AwsAsyncClientBuilder.build(AwsAsyncClientBuilder.java:80)
	at com.coinbase.util.KmsClient$.getSnowflakeUsernamePassword(KmsClient.scala:21)
	at com.coinbase.ml.RunFlinkJob$.runBatch(RunFlinkJob.scala:94)
	at com.coinbase.ml.RunFlinkJob$.runFlinkJob(RunFlinkJob.scala:38)
	at com.coinbase.ml.FlinkFeatureProcessingJobEntryPoint$.main(CmdLineParser.scala:76)
	at com.coinbase.ml.FlinkFeatureProcessingJobEntryPoint.main(CmdLineParser.scala)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:576)
	at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:438)
	at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:274)
	at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:746)
	at org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:273)
	at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:205)
	at org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1010)
	at org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1083)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1844)
	at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
	at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1083)


--
Thanks & Regards,
Anuj Jain
Mob. : +91- 8588817877
Skype : anuj.jain07
Fanbin Bu
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: flink 1.9 conflict jackson version

Fanbin Bu
Hi Aj,

I got a work around to put my app jar inside /usr/lib/flink/lib directory. 

On Mon, Apr 6, 2020 at 11:27 PM aj <[hidden email]> wrote:
Hi Fanbin,

I am facing a similar kind of issue. Let me know if you are able to resolve this issue then please help me also


On Tue, Dec 17, 2019 at 7:50 AM ouywl <[hidden email]> wrote:
Hi Bu 
   I think It can use mvn-shade-plugin to resolve your problem,  It seem flink-client conflict with your owner jar?
签名由 网易邮箱大师 定制

On 12/17/2019 08:10，[hidden email] wrote：
Hi,

After I upgrade flink 1.9, I got the following error message on EMR, it works locally on IntelliJ.

I'm explicitly declaring the dependency as 
implementation 'com.fasterxml.jackson.module:jackson-module-scala_2.11:2.10.1'
and I have
implementation group: 'com.amazonaws', name: 'aws-java-sdk-emr', version: '1.11.595'



java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.ObjectMapper.enable([Lcom/fasterxml/jackson/core/JsonParser$Feature;)Lcom/fasterxml/jackson/databind/ObjectMapper;
	at com.amazonaws.partitions.PartitionsLoader.<clinit>(PartitionsLoader.java:54)
	at com.amazonaws.regions.RegionMetadataFactory.create(RegionMetadataFactory.java:30)
	at com.amazonaws.regions.RegionUtils.initialize(RegionUtils.java:65)
	at com.amazonaws.regions.RegionUtils.getRegionMetadata(RegionUtils.java:53)
	at com.amazonaws.regions.RegionUtils.getRegion(RegionUtils.java:107)
	at com.amazonaws.client.builder.AwsClientBuilder.getRegionObject(AwsClientBuilder.java:256)
	at com.amazonaws.client.builder.AwsClientBuilder.setRegion(AwsClientBuilder.java:460)
	at com.amazonaws.client.builder.AwsClientBuilder.configureMutableProperties(AwsClientBuilder.java:424)
	at com.amazonaws.client.builder.AwsAsyncClientBuilder.build(AwsAsyncClientBuilder.java:80)
	at com.coinbase.util.KmsClient$.getSnowflakeUsernamePassword(KmsClient.scala:21)
	at com.coinbase.ml.RunFlinkJob$.runBatch(RunFlinkJob.scala:94)
	at com.coinbase.ml.RunFlinkJob$.runFlinkJob(RunFlinkJob.scala:38)
	at com.coinbase.ml.FlinkFeatureProcessingJobEntryPoint$.main(CmdLineParser.scala:76)
	at com.coinbase.ml.FlinkFeatureProcessingJobEntryPoint.main(CmdLineParser.scala)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:576)
	at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:438)
	at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:274)
	at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:746)
	at org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:273)
	at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:205)
	at org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1010)
	at org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1083)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1844)
	at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
	at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1083)


--
Thanks & Regards,
Anuj Jain
Mob. : +91- 8588817877
Skype : anuj.jain07
Free forum by Nabble Edit this page