|
|
Hi Avi,
I actually wanted to check the result of gradle dependencies. But okay I want ahead and checked your sources and as suspected, your build is using scala 2.12.12 org.scala-lang:scala-library:2.12.7 -> 2.12.12.
Please use strict version [1].
Hi, did all that including upgrading the gradle version and got the same result. please find attached a sample project here
Thank you Avi
Hi Avi,
could you run
gradle dependencies and report back to me?
Also did you ensure to run
gradle clean before? The gradle version you are using is ancient, so I'm not sure if it's picking up the change correctly.
No, I don't. I actually tried also with 2.12.7 and got the same result
Hi Avi,
apparently the maximum version that Flink supports for scala is 2.12.7 [1]. Do you have a specific reason to use a higher version?
Hi Arvid, Please find attached full build.gradle file
Can you post the full dependencies of sbt/maven/gradle whatever?
Hi Arvid, using : flinkVersion = '1.12.0'
scalaBinaryVersion = '2.12' I simplified the example to (same exception) : object Flinktest extends App {
private val env = StreamExecutionEnvironment.getExecutionEnvironment
env.fromElements("A", "B","c")
.windowAll(TumblingEventTimeWindows.of(Time.seconds(5)))
.process{new ProcessAllWindowFunction[String, List[String], TimeWindow] {
override def process(context: Context, elements: Iterable[String], out: Collector[List[String]]): Unit = {
out.collect(elements.toList)
}
}
}
.print()
env.execute("Sample")
}
Hi Avi,
without being a scala-guy, I'm guessing that you are mixing scala versions. Could you check that your user code uses the same scala version as Flink (1.11 or 1.12)? I have also heard of issues with different minor versions of scala, so make sure to use the exact same version (e.g. 2.11.12).
I am trying to aggregate all records in a time window. This is my ProcessAllWindowFunction :
case class SimpleAggregate(elms: List[String])
class AggregateLogs extends ProcessAllWindowFunction[String, SimpleAggregate, TimeWindow ] {
override def process(context: Context, elements: Iterable[String], out: Collector[SimpleAggregate]): Unit = {
val es: List[String] = elements.toList
val record = SimpleAggregate(es)
out.collect(record)
}
}But I am getting this exception why ?
Exception in thread "main" java.util.concurrent.ExecutionException: scala.tools.reflect.ToolBoxError: reflective compilation has failed: cannot initialize the compiler due to java.lang.BootstrapMethodError: java.lang.NoSuchMethodError: scala.collection.immutable.List.$anonfun$flatMap$1$adapted(Lscala/runtime/BooleanRef;Lscala/runtime/ObjectRef;Lscala/runtime/ObjectRef;Ljava/lang/Object;)Ljava/lang/Object;
at org.apache.flink.shaded.guava18.com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:299)
at org.apache.flink.shaded.guava18.com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:286)
at org.apache.flink.shaded.guava18.com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:116)
at org.apache.flink.shaded.guava18.com.google.common.util.concurrent.Uninterruptibles.getUninterruptibly(Uninterruptibles.java:137)
at org.apache.flink.shaded.guava18.com.google.common.cache.LocalCache$Segment.getAndRecordStats(LocalCache.java:2348)
at org.apache.flink.shaded.guava18.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2320)
at org.apache.flink.shaded.guava18.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2282)
at org.apache.flink.shaded.guava18.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2197)
at org.apache.flink.shaded.guava18.com.google.common.cache.LocalCache.get(LocalCache.java:3937)
at org.apache.flink.shaded.guava18.com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4739)
at org.apache.flink.api.scala.typeutils.TraversableSerializer$.compileCbf(TraversableSerializer.scala:184)
at org.apache.flink.api.scala.typeutils.TraversableSerializer.compileCbf(TraversableSerializer.scala:51)
at org.apache.flink.api.scala.typeutils.TraversableSerializer.<init>(TraversableSerializer.scala:41)
at com.neosec.handlefinancial.HandleFinancialJob$$anon$1$$anon$2$$anon$3.<init>(HandleFinancialJob.scala:52)
at com.neosec.handlefinancial.HandleFinancialJob$$anon$1$$anon$2.createSerializer(HandleFinancialJob.scala:52)
at com.neosec.handlefinancial.HandleFinancialJob$$anon$1$$anon$2.createSerializer(HandleFinancialJob.scala:52)
at com.neosec.handlefinancial.HandleFinancialJob$$anon$1.$anonfun$createSerializer$1(HandleFinancialJob.scala:52)
at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:158)
at com.neosec.handlefinancial.HandleFinancialJob$$anon$1.createSerializer(HandleFinancialJob.scala:52)
at org.apache.flink.streaming.api.graph.StreamGraph.createSerializer(StreamGraph.java:864)
at org.apache.flink.streaming.api.graph.StreamGraph.addOperator(StreamGraph.java:308)
at org.apache.flink.streaming.api.graph.StreamGraph.addOperator(StreamGraph.java:293)
at org.apache.flink.streaming.api.graph.StreamGraphGenerator.transformOneInputTransform(StreamGraphGenerator.java:680)
at org.apache.flink.streaming.api.graph.StreamGraphGenerator.transform(StreamGraphGenerator.java:253)
at org.apache.flink.streaming.api.graph.StreamGraphGenerator.generate(StreamGraphGenerator.java:212)
at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.getStreamGraph(StreamExecutionEnvironment.java:1863)
at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.getStreamGraph(StreamExecutionEnvironment.java:1848)
at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.execute(StreamExecutionEnvironment.java:1699)
at org.apache.flink.streaming.api.scala.StreamExecutionEnvironment.execute(StreamExecutionEnvironment.scala:699)
at com.neosec.handlefinancial.HandleFinancialJob$.delayedEndpoint$com$neosec$handlefinancial$HandleFinancialJob$1(HandleFinancialJob.scala:60)
at com.neosec.handlefinancial.HandleFinancialJob$delayedInit$body.apply(HandleFinancialJob.scala:20)
at scala.Function0.apply$mcV$sp(Function0.scala:39)
at scala.Function0.apply$mcV$sp$(Function0.scala:39)
at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:17)
at scala.App.$anonfun$main$1$adapted(App.scala:80)
at scala.collection.immutable.List.foreach(List.scala:431)
at scala.App.main(App.scala:80)
at scala.App.main$(App.scala:78)
at com.neosec.handlefinancial.HandleFinancialJob$.main(HandleFinancialJob.scala:20)
at com.neosec.handlefinancial.HandleFinancialJob.main(HandleFinancialJob.scala)
Caused by: scala.tools.reflect.ToolBoxError: reflective compilation has failed: cannot initialize the compiler due to java.lang.BootstrapMethodError: java.lang.NoSuchMethodError: scala.collection.immutable.List.$anonfun$flatMap$1$adapted(Lscala/runtime/BooleanRef;Lscala/runtime/ObjectRef;Lscala/runtime/ObjectRef;Ljava/lang/Object;)Ljava/lang/Object;
Caused by: scala.tools.reflect.ToolBoxError: reflective compilation has failed: cannot initialize the compiler due to java.lang.BootstrapMethodError: java.lang.NoSuchMethodError: scala.collection.immutable.List.$anonfun$flatMap$1$adapted(Lscala/runtime/BooleanRef;Lscala/runtime/ObjectRef;Lscala/runtime/ObjectRef;Ljava/lang/Object;)Ljava/lang/Object;
at scala.tools.reflect.ToolBoxFactory$ToolBoxImpl$withCompilerApi$api$.liftedTree1$1(ToolBoxFactory.scala:360)
at scala.tools.reflect.ToolBoxFactory$ToolBoxImpl$withCompilerApi$api$.compiler$lzycompute(ToolBoxFactory.scala:346)
at scala.tools.reflect.ToolBoxFactory$ToolBoxImpl$withCompilerApi$api$.compiler(ToolBoxFactory.scala:345)
at scala.tools.reflect.ToolBoxFactory$ToolBoxImpl$withCompilerApi$.apply(ToolBoxFactory.scala:372)
at scala.tools.reflect.ToolBoxFactory$ToolBoxImpl.parse(ToolBoxFactory.scala:429)
at org.apache.flink.api.scala.typeutils.TraversableSerializer$LazyRuntimeCompiler.compileCbfInternal(TraversableSerializer.scala:229)
at org.apache.flink.api.scala.typeutils.TraversableSerializer$LazyRuntimeCompiler.call(TraversableSerializer.scala:220)
at org.apache.flink.api.scala.typeutils.TraversableSerializer$LazyRuntimeCompiler.call(TraversableSerializer.scala:216)
at org.apache.flink.shaded.guava18.com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4742)
at org.apache.flink.shaded.guava18.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3527)
at org.apache.flink.shaded.guava18.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2319)
... 34 more
Caused by: java.lang.BootstrapMethodError: java.lang.NoSuchMethodError: scala.collection.immutable.List.$anonfun$flatMap$1$adapted(Lscala/runtime/BooleanRef;Lscala/runtime/ObjectRef;Lscala/runtime/ObjectRef;Ljava/lang/Object;)Ljava/lang/Object;
at scala.tools.nsc.PhaseAssembly$DependencyGraph.compilerPhaseList(PhaseAssembly.scala:102)
at scala.tools.nsc.PhaseAssembly.computePhaseAssembly(PhaseAssembly.scala:244)
at scala.tools.nsc.PhaseAssembly.computePhaseAssembly$(PhaseAssembly.scala:216)
Caused by: java.lang.BootstrapMethodError: java.lang.NoSuchMethodError: scala.collection.immutable.List.$anonfun$flatMap$1$adapted(Lscala/runtime/BooleanRef;Lscala/runtime/ObjectRef;Lscala/runtime/ObjectRef;Ljava/lang/Object;)Ljava/lang/Object;
at scala.tools.nsc.Global.computePhaseAssembly(Global.scala:44)
at scala.tools.nsc.Global.computePhaseDescriptors(Global.scala:713)
at scala.tools.nsc.Global.phaseDescriptors$lzycompute(Global.scala:717)
at scala.tools.nsc.Global.phaseDescriptors(Global.scala:717)
at scala.tools.nsc.Global$Run.<init>(Global.scala:1219)
at scala.tools.reflect.ReflectSetup.$init$(ReflectSetup.scala:21)
at scala.tools.reflect.ReflectGlobal.<init>(ReflectGlobal.scala:26)
at scala.tools.reflect.ToolBoxFactory$ToolBoxImpl$ToolBoxGlobal.<init>(ToolBoxFactory.scala:52)
at scala.tools.reflect.ToolBoxFactory$ToolBoxImpl$withCompilerApi$api$.liftedTree1$1(ToolBoxFactory.scala:350)
... 44 more
Caused by: java.lang.NoSuchMethodError: scala.collection.immutable.List.$anonfun$flatMap$1$adapted(Lscala/runtime/BooleanRef;Lscala/runtime/ObjectRef;Lscala/runtime/ObjectRef;Ljava/lang/Object;)Ljava/lang/Object;
... 56 more
Caused by: java.lang.NoSuchMethodError: scala.collection.immutable.List.$anonfun$flatMap$1$adapted(Lscala/runtime/BooleanRef;Lscala/runtime/ObjectRef;Lscala/runtime/ObjectRef;Ljava/lang/Object;)Ljava/lang/Object;
--
Arvid Heise | Senior Java Developer 
Follow us @VervericaData -- Join Flink Forward - The Apache Flink Conference Stream Processing | Event Driven | Real Time -- Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany -- Ververica GmbH
Registered at Amtsgericht Charlottenburg: HRB 158244 B
Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji (Toni) Cheng
--
Arvid Heise | Senior Java Developer
Follow us @VervericaData -- Join Flink Forward - The Apache Flink Conference Stream Processing | Event Driven | Real Time -- Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany -- Ververica GmbH
Registered at Amtsgericht Charlottenburg: HRB 158244 B
Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji (Toni) Cheng
--
Arvid Heise | Senior Java Developer
Follow us @VervericaData -- Join Flink Forward - The Apache Flink Conference Stream Processing | Event Driven | Real Time -- Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany -- Ververica GmbH
Registered at Amtsgericht Charlottenburg: HRB 158244 B
Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji (Toni) Cheng
--
Arvid Heise | Senior Java Developer
Follow us @VervericaData -- Join Flink Forward - The Apache Flink Conference Stream Processing | Event Driven | Real Time -- Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany -- Ververica GmbH
Registered at Amtsgericht Charlottenburg: HRB 158244 B
Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji (Toni) Cheng
-- Arvid Heise | Senior Java Developer
Follow us @VervericaData -- Join Flink Forward - The Apache Flink Conference Stream Processing | Event Driven | Real Time -- Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany -- Ververica GmbH
Registered at Amtsgericht Charlottenburg: HRB 158244 B
Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji (Toni) Cheng
|
Locked
