(DEPRECATED) Apache Flink User Mailing List archive.

getting an exception

Classic

List

Threaded

4 messages Options

avilevi

getting an exception

Hi,

I'm using Flink 1.8.1. our code is mostly using Scala.

When I try to submit my job (on my local machine ) it crashes with the error below (BTW on the IDE it runs perfectly).

Any assistance would be appreciated.

Thanks

Avi

2019-08-05 12:58:03.783 [Flink-DispatcherRestEndpoint-thread-3] ERROR org.apache.flink.runtime.webmonitor.handlers.JarRunHandler  - Unhandled exception.
org.apache.flink.client.program.ProgramInvocationException: The program caused an error: 
	at org.apache.flink.client.program.OptimizerPlanEnvironment.getOptimizedPlan(OptimizerPlanEnvironment.java:93)
	at org.apache.flink.client.program.PackagedProgramUtils.createJobGraph(PackagedProgramUtils.java:80)
	at org.apache.flink.runtime.webmonitor.handlers.utils.JarHandlerUtils$JarHandlerContext.toJobGraph(JarHandlerUtils.java:126)
	at org.apache.flink.runtime.webmonitor.handlers.JarRunHandler.lambda$getJobGraphAsync$6(JarRunHandler.java:142)
	at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NoSuchMethodError: org.apache.flink.api.java.ClosureCleaner.clean(Ljava/lang/Object;Lorg/apache/flink/api/common/ExecutionConfig$ClosureCleanerLevel;Z)V
	at org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer011.<init>(FlinkKafkaProducer011.java:494)
	at org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer011.<init>(FlinkKafkaProducer011.java:448)
	at org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer011.<init>(FlinkKafkaProducer011.java:383)
	at com.bluevoyant.commons.queueHandlers.KeyedKafkaProducerImpl.producer(KafkaImpl.scala:18)
	at com.bluevoyant.commons.queueHandlers.KeyedKafkaProducerImpl.producer$(KafkaImpl.scala:18)
	at com.bluevoyant.lookalike.analytic.queueHandlers.QueueHandlerImpl$.producer(QueueHandlerImpl.scala:13)
	at com.bluevoyant.lookalike.analytic.StreamingJob$.delayedEndpoint$com$bluevoyant$lookalike$analytic$StreamingJob$1(StreamingJob.scala:42)
	at com.bluevoyant.lookalike.analytic.StreamingJob$delayedInit$body.apply(StreamingJob.scala:14)
	at scala.Function0.apply$mcV$sp(Function0.scala:34)
	at scala.Function0.apply$mcV$sp$(Function0.scala:34)
	at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
	at scala.App.$anonfun$main$1$adapted(App.scala:76)
	at scala.collection.immutable.List.foreach(List.scala:388)
	at scala.App.main(App.scala:76)
	at scala.App.main$(App.scala:74)
	at com.bluevoyant.lookalike.analytic.StreamingJob$.main(StreamingJob.scala:14)
	at com.bluevoyant.lookalike.analytic.StreamingJob.main(StreamingJob.scala)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:529)
	at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:421)
	at org.apache.flink.client.program.OptimizerPlanEnvironment.getOptimizedPlan(OptimizerPlanEnvironment.java:83)
	... 7 common frames omitted

Victor Wong

Re: getting an exception

Hi Avi:

It seems you are submitting your job with an older Flink version (< 1.8), please check your flink-dist version.

Regards,

Victor

From: Avi Levi <[hidden email]>
Date: Monday, August 5, 2019 at 9:11 PM
To: user <[hidden email]>
Subject: getting an exception

Hi,

I'm using Flink 1.8.1. our code is mostly using Scala.

When I try to submit my job (on my local machine ) it crashes with the error below (BTW on the IDE it runs perfectly).

Any assistance would be appreciated.

Thanks

Avi

2019-08-05 12:58:03.783 [Flink-DispatcherRestEndpoint-thread-3] ERROR org.apache.flink.runtime.webmonitor.handlers.JarRunHandler  - Unhandled exception.

org.apache.flink.client.program.ProgramInvocationException: The program caused an error:

        at org.apache.flink.client.program.OptimizerPlanEnvironment.getOptimizedPlan(OptimizerPlanEnvironment.java:93)

        at org.apache.flink.client.program.PackagedProgramUtils.createJobGraph(PackagedProgramUtils.java:80)

        at org.apache.flink.runtime.webmonitor.handlers.utils.JarHandlerUtils$JarHandlerContext.toJobGraph(JarHandlerUtils.java:126)

        at org.apache.flink.runtime.webmonitor.handlers.JarRunHandler.lambda$getJobGraphAsync$6(JarRunHandler.java:142)

        at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

        at java.lang.Thread.run(Thread.java:748)

Caused by: java.lang.NoSuchMethodError: org.apache.flink.api.java.ClosureCleaner.clean(Ljava/lang/Object;Lorg/apache/flink/api/common/ExecutionConfig$ClosureCleanerLevel;Z)V

        at org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer011.<init>(FlinkKafkaProducer011.java:494)

        at org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer011.<init>(FlinkKafkaProducer011.java:448)

        at org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer011.<init>(FlinkKafkaProducer011.java:383)

        at com.bluevoyant.commons.queueHandlers.KeyedKafkaProducerImpl.producer(KafkaImpl.scala:18)

        at com.bluevoyant.commons.queueHandlers.KeyedKafkaProducerImpl.producer$(KafkaImpl.scala:18)

        at com.bluevoyant.lookalike.analytic.queueHandlers.QueueHandlerImpl$.producer(QueueHandlerImpl.scala:13)

        at com.bluevoyant.lookalike.analytic.StreamingJob$.delayedEndpoint$com$bluevoyant$lookalike$analytic$StreamingJob$1(StreamingJob.scala:42)

        at com.bluevoyant.lookalike.analytic.StreamingJob$delayedInit$body.apply(StreamingJob.scala:14)

        at scala.Function0.apply$mcV$sp(Function0.scala:34)

        at scala.Function0.apply$mcV$sp$(Function0.scala:34)

        at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)

        at scala.App.$anonfun$main$1$adapted(App.scala:76)

        at scala.collection.immutable.List.foreach(List.scala:388)

        at scala.App.main(App.scala:76)

        at scala.App.main$(App.scala:74)

        at com.bluevoyant.lookalike.analytic.StreamingJob$.main(StreamingJob.scala:14)

        at com.bluevoyant.lookalike.analytic.StreamingJob.main(StreamingJob.scala)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:498)

        at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:529)

        at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:421)

        at org.apache.flink.client.program.OptimizerPlanEnvironment.getOptimizedPlan(OptimizerPlanEnvironment.java:83)

        ... 7 common frames omitted

Gaël Renoux

Re: getting an exception

Hi Avi and Victor,

I just opened this ticket on JIRA: https://issues.apache.org/jira/browse/FLINK-13586 (I hadn't seen these e-mails). Backward compatibility is broken between 1.8.0 and 1.8.1 if you use Kafka connectors.

Can you upgrade your flink-connector-kafka dependency to 1.8.1 ? It won't deploy on a 1.8.0 server any more, if that's a concern for you.

Gaël

On Mon, Aug 5, 2019 at 4:37 PM Wong Victor <[hidden email]> wrote:

Hi Avi:

It seems you are submitting your job with an older Flink version (< 1.8), please check your flink-dist version.

Regards,

Victor

From: Avi Levi <[hidden email]>
Date: Monday, August 5, 2019 at 9:11 PM
To: user <[hidden email]>
Subject: getting an exception

Hi,

I'm using Flink 1.8.1. our code is mostly using Scala.

When I try to submit my job (on my local machine ) it crashes with the error below (BTW on the IDE it runs perfectly).

Any assistance would be appreciated.

Thanks

Avi

2019-08-05 12:58:03.783 [Flink-DispatcherRestEndpoint-thread-3] ERROR org.apache.flink.runtime.webmonitor.handlers.JarRunHandler  - Unhandled exception.

org.apache.flink.client.program.ProgramInvocationException: The program caused an error:

        at org.apache.flink.client.program.OptimizerPlanEnvironment.getOptimizedPlan(OptimizerPlanEnvironment.java:93)

        at org.apache.flink.client.program.PackagedProgramUtils.createJobGraph(PackagedProgramUtils.java:80)

        at org.apache.flink.runtime.webmonitor.handlers.utils.JarHandlerUtils$JarHandlerContext.toJobGraph(JarHandlerUtils.java:126)

        at org.apache.flink.runtime.webmonitor.handlers.JarRunHandler.lambda$getJobGraphAsync$6(JarRunHandler.java:142)

        at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

        at java.lang.Thread.run(Thread.java:748)

Caused by: java.lang.NoSuchMethodError: org.apache.flink.api.java.ClosureCleaner.clean(Ljava/lang/Object;Lorg/apache/flink/api/common/ExecutionConfig$ClosureCleanerLevel;Z)V

        at org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer011.<init>(FlinkKafkaProducer011.java:494)

        at org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer011.<init>(FlinkKafkaProducer011.java:448)

        at org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer011.<init>(FlinkKafkaProducer011.java:383)

        at com.bluevoyant.commons.queueHandlers.KeyedKafkaProducerImpl.producer(KafkaImpl.scala:18)

        at com.bluevoyant.commons.queueHandlers.KeyedKafkaProducerImpl.producer$(KafkaImpl.scala:18)

        at com.bluevoyant.lookalike.analytic.queueHandlers.QueueHandlerImpl$.producer(QueueHandlerImpl.scala:13)

        at com.bluevoyant.lookalike.analytic.StreamingJob$.delayedEndpoint$com$bluevoyant$lookalike$analytic$StreamingJob$1(StreamingJob.scala:42)

        at com.bluevoyant.lookalike.analytic.StreamingJob$delayedInit$body.apply(StreamingJob.scala:14)

        at scala.Function0.apply$mcV$sp(Function0.scala:34)

        at scala.Function0.apply$mcV$sp$(Function0.scala:34)

        at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)

        at scala.App.$anonfun$main$1$adapted(App.scala:76)

        at scala.collection.immutable.List.foreach(List.scala:388)

        at scala.App.main(App.scala:76)

        at scala.App.main$(App.scala:74)

        at com.bluevoyant.lookalike.analytic.StreamingJob$.main(StreamingJob.scala:14)

        at com.bluevoyant.lookalike.analytic.StreamingJob.main(StreamingJob.scala)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:498)

        at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:529)

        at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:421)

        at org.apache.flink.client.program.OptimizerPlanEnvironment.getOptimizedPlan(OptimizerPlanEnvironment.java:83)

        ... 7 common frames omitted

Gaël Renoux
Senior R&D Engineer, DataDome

M <a href="tel:+33+6+76+89+16+52" style="text-decoration:none;color:rgb(68,68,68);font-family:Arial,Helvetica,sans-serif" target="_blank"> +33 6 76 89 16 52

E [hidden email]

W www.datadome.co

avilevi

Re: getting an exception

Yeap that was it (deploying 1.8.1 over 1.8.0 ) thanks !!!

On Mon, Aug 5, 2019 at 5:53 PM Gaël Renoux <[hidden email]> wrote:

This Message originated outside your organization.

Hi Avi and Victor,

Can you upgrade your flink-connector-kafka dependency to 1.8.1 ? It won't deploy on a 1.8.0 server any more, if that's a concern for you.

Gaël

On Mon, Aug 5, 2019 at 4:37 PM Wong Victor <[hidden email]> wrote:

Hi Avi:

It seems you are submitting your job with an older Flink version (< 1.8), please check your flink-dist version.

Regards,

Victor

From: Avi Levi <[hidden email]>
Date: Monday, August 5, 2019 at 9:11 PM
To: user <[hidden email]>
Subject: getting an exception

Hi,

I'm using Flink 1.8.1. our code is mostly using Scala.

When I try to submit my job (on my local machine ) it crashes with the error below (BTW on the IDE it runs perfectly).

Any assistance would be appreciated.

Thanks

Avi

2019-08-05 12:58:03.783 [Flink-DispatcherRestEndpoint-thread-3] ERROR org.apache.flink.runtime.webmonitor.handlers.JarRunHandler  - Unhandled exception.

org.apache.flink.client.program.ProgramInvocationException: The program caused an error:

        at org.apache.flink.client.program.OptimizerPlanEnvironment.getOptimizedPlan(OptimizerPlanEnvironment.java:93)

        at org.apache.flink.client.program.PackagedProgramUtils.createJobGraph(PackagedProgramUtils.java:80)

        at org.apache.flink.runtime.webmonitor.handlers.utils.JarHandlerUtils$JarHandlerContext.toJobGraph(JarHandlerUtils.java:126)

        at org.apache.flink.runtime.webmonitor.handlers.JarRunHandler.lambda$getJobGraphAsync$6(JarRunHandler.java:142)

        at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

        at java.lang.Thread.run(Thread.java:748)

Caused by: java.lang.NoSuchMethodError: org.apache.flink.api.java.ClosureCleaner.clean(Ljava/lang/Object;Lorg/apache/flink/api/common/ExecutionConfig$ClosureCleanerLevel;Z)V

        at org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer011.<init>(http://FlinkKafkaProducer011.java:494)

        at org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer011.<init>(http://FlinkKafkaProducer011.java:448)

        at org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer011.<init>(http://FlinkKafkaProducer011.java:383)

        at com.bluevoyant.commons.queueHandlers.KeyedKafkaProducerImpl.producer(KafkaImpl.scala:18)

        at com.bluevoyant.commons.queueHandlers.KeyedKafkaProducerImpl.producer$(KafkaImpl.scala:18)

        at com.bluevoyant.lookalike.analytic.queueHandlers.QueueHandlerImpl$.producer(QueueHandlerImpl.scala:13)

        at com.bluevoyant.lookalike.analytic.StreamingJob$.delayedEndpoint$com$bluevoyant$lookalike$analytic$StreamingJob$1(StreamingJob.scala:42)

        at com.bluevoyant.lookalike.analytic.StreamingJob$delayedInit$body.apply(StreamingJob.scala:14)

        at scala.Function0.apply$mcV$sp(Function0.scala:34)

        at scala.Function0.apply$mcV$sp$(Function0.scala:34)

        at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)

        at scala.App.$anonfun$main$1$adapted(App.scala:76)

        at scala.collection.immutable.List.foreach(List.scala:388)

        at scala.App.main(App.scala:76)

        at scala.App.main$(App.scala:74)

        at com.bluevoyant.lookalike.analytic.StreamingJob$.main(StreamingJob.scala:14)

        at com.bluevoyant.lookalike.analytic.StreamingJob.main(StreamingJob.scala)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:498)

        at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:529)

        at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:421)

        at org.apache.flink.client.program.OptimizerPlanEnvironment.getOptimizedPlan(OptimizerPlanEnvironment.java:83)

        ... 7 common frames omitted

Gaël Renoux
Senior R&D Engineer, DataDome

M <a href="tel:+33+6+76+89+16+52" style="text-decoration:none;color:rgb(68,68,68);font-family:Arial,Helvetica,sans-serif" target="_blank"> +33 6 76 89 16 52

E [hidden email]

W www.datadome.co