asm IllegalArgumentException with 1.0.0

[Zach Cox]

Hi - after upgrading to 1.0.0, I'm getting this exception now in a unit test:

   IllegalArgumentException:   (null:-1)

org.apache.flink.shaded.org.objectweb.asm.ClassVisitor.<init>(Unknown Source)

org.apache.flink.shaded.org.objectweb.asm.ClassVisitor.<init>(Unknown Source)

org.apache.flink.api.scala.InnerClosureFinder.<init>(ClosureCleaner.scala:279)

org.apache.flink.api.scala.ClosureCleaner$.getInnerClasses(ClosureCleaner.scala:95)

org.apache.flink.api.scala.ClosureCleaner$.clean(ClosureCleaner.scala:115)

org.apache.flink.streaming.api.scala.StreamExecutionEnvironment.scalaClean(StreamExecutionEnvironment.scala:568)

org.apache.flink.streaming.api.scala.StreamExecutionEnvironment.addSource(StreamExecutionEnvironment.scala:498)

The line that causes that exception is just adding a FlinkKafkaConsumer08 source.

ClassVisitor [1] seems to throw that IllegalArgumentException when it is not given a valid api version number, but InnerClosureFinder [2] looks fine to me.

Any idea what might be causing this? This unit test worked fine with 1.0.0-rc0 jars.

Thanks,

Zach

[1] http://websvn.ow2.org/filedetails.php?repname=asm&path=%2Ftrunk%2Fasm%2Fsrc%2Forg%2Fobjectweb%2Fasm%2FClassVisitor.java

[2] https://github.com/apache/flink/blob/master/flink-scala/src/main/scala/org/apache/flink/api/scala/ClosureCleaner.scala#L279

[Zach Cox]

I also noticed when I try to run this application in a local environment, I get the same IllegalArgumentException.

When I assemble this application into a fat jar and run it on a Flink cluster using the CLI tools, it seems to run fine. 

Maybe my local classpath is missing something that is provided on the Flink task managers?

-Zach

On Wed, Mar 9, 2016 at 5:16 PM Zach Cox <[hidden email]> wrote:

Hi - after upgrading to 1.0.0, I'm getting this exception now in a unit test:

   IllegalArgumentException:   (null:-1)

org.apache.flink.shaded.org.objectweb.asm.ClassVisitor.<init>(Unknown Source)

org.apache.flink.shaded.org.objectweb.asm.ClassVisitor.<init>(Unknown Source)

org.apache.flink.api.scala.InnerClosureFinder.<init>(ClosureCleaner.scala:279)

org.apache.flink.api.scala.ClosureCleaner$.getInnerClasses(ClosureCleaner.scala:95)

org.apache.flink.api.scala.ClosureCleaner$.clean(ClosureCleaner.scala:115)

org.apache.flink.streaming.api.scala.StreamExecutionEnvironment.scalaClean(StreamExecutionEnvironment.scala:568)

org.apache.flink.streaming.api.scala.StreamExecutionEnvironment.addSource(StreamExecutionEnvironment.scala:498)

The line that causes that exception is just adding a FlinkKafkaConsumer08 source.

ClassVisitor [1] seems to throw that IllegalArgumentException when it is not given a valid api version number, but InnerClosureFinder [2] looks fine to me.

Any idea what might be causing this? This unit test worked fine with 1.0.0-rc0 jars.

Thanks,

Zach

[1] http://websvn.ow2.org/filedetails.php?repname=asm&path=%2Ftrunk%2Fasm%2Fsrc%2Forg%2Fobjectweb%2Fasm%2FClassVisitor.java

[2] https://github.com/apache/flink/blob/master/flink-scala/src/main/scala/org/apache/flink/api/scala/ClosureCleaner.scala#L279

[Stephan Ewen]

Dependency shading changed a bit between RC4 and RC5 - maybe a different minor ASM version is now included in the "test" scope.

Can you share the dependencies of the problematic project?

On Thu, Mar 10, 2016 at 12:26 AM, Zach Cox <[hidden email]> wrote:

I also noticed when I try to run this application in a local environment, I get the same IllegalArgumentException.

When I assemble this application into a fat jar and run it on a Flink cluster using the CLI tools, it seems to run fine. 

Maybe my local classpath is missing something that is provided on the Flink task managers?

-Zach

On Wed, Mar 9, 2016 at 5:16 PM Zach Cox <[hidden email]> wrote:

Hi - after upgrading to 1.0.0, I'm getting this exception now in a unit test:

   IllegalArgumentException:   (null:-1)

org.apache.flink.shaded.org.objectweb.asm.ClassVisitor.<init>(Unknown Source)

org.apache.flink.shaded.org.objectweb.asm.ClassVisitor.<init>(Unknown Source)

org.apache.flink.api.scala.InnerClosureFinder.<init>(ClosureCleaner.scala:279)

org.apache.flink.api.scala.ClosureCleaner$.getInnerClasses(ClosureCleaner.scala:95)

org.apache.flink.api.scala.ClosureCleaner$.clean(ClosureCleaner.scala:115)

org.apache.flink.streaming.api.scala.StreamExecutionEnvironment.scalaClean(StreamExecutionEnvironment.scala:568)

org.apache.flink.streaming.api.scala.StreamExecutionEnvironment.addSource(StreamExecutionEnvironment.scala:498)

The line that causes that exception is just adding a FlinkKafkaConsumer08 source.

ClassVisitor [1] seems to throw that IllegalArgumentException when it is not given a valid api version number, but InnerClosureFinder [2] looks fine to me.

Any idea what might be causing this? This unit test worked fine with 1.0.0-rc0 jars.

Thanks,

Zach

[1] http://websvn.ow2.org/filedetails.php?repname=asm&path=%2Ftrunk%2Fasm%2Fsrc%2Forg%2Fobjectweb%2Fasm%2FClassVisitor.java

[2] https://github.com/apache/flink/blob/master/flink-scala/src/main/scala/org/apache/flink/api/scala/ClosureCleaner.scala#L279

[Zach Cox]

Here are the jars on the classpath when I try to run our Flink job in a local environment (via `sbt run`): 

https://gist.githubusercontent.com/zcox/0992aba1c517b51dc879/raw/7136ec034c2beef04bd65de9f125ce3796db511f/gistfile1.txt

There are many transitive dependencies pulled in from internal library projects that probably need to be cleaned out. Maybe we are including something that conflicts? Or maybe something important is being excluded?

Are the asm classes included in Flink jars in some shaded form?

Thanks,

Zach

On Thu, Mar 10, 2016 at 5:06 AM Stephan Ewen <[hidden email]> wrote:

Dependency shading changed a bit between RC4 and RC5 - maybe a different minor ASM version is now included in the "test" scope.

Can you share the dependencies of the problematic project?

On Thu, Mar 10, 2016 at 12:26 AM, Zach Cox <[hidden email]> wrote:

I also noticed when I try to run this application in a local environment, I get the same IllegalArgumentException.

When I assemble this application into a fat jar and run it on a Flink cluster using the CLI tools, it seems to run fine. 

Maybe my local classpath is missing something that is provided on the Flink task managers?

-Zach

On Wed, Mar 9, 2016 at 5:16 PM Zach Cox <[hidden email]> wrote:

Hi - after upgrading to 1.0.0, I'm getting this exception now in a unit test:

   IllegalArgumentException:   (null:-1)

org.apache.flink.shaded.org.objectweb.asm.ClassVisitor.<init>(Unknown Source)

org.apache.flink.shaded.org.objectweb.asm.ClassVisitor.<init>(Unknown Source)

org.apache.flink.api.scala.InnerClosureFinder.<init>(ClosureCleaner.scala:279)

org.apache.flink.api.scala.ClosureCleaner$.getInnerClasses(ClosureCleaner.scala:95)

org.apache.flink.api.scala.ClosureCleaner$.clean(ClosureCleaner.scala:115)

org.apache.flink.streaming.api.scala.StreamExecutionEnvironment.scalaClean(StreamExecutionEnvironment.scala:568)

org.apache.flink.streaming.api.scala.StreamExecutionEnvironment.addSource(StreamExecutionEnvironment.scala:498)

The line that causes that exception is just adding a FlinkKafkaConsumer08 source.

ClassVisitor [1] seems to throw that IllegalArgumentException when it is not given a valid api version number, but InnerClosureFinder [2] looks fine to me.

Any idea what might be causing this? This unit test worked fine with 1.0.0-rc0 jars.

Thanks,

Zach

[1] http://websvn.ow2.org/filedetails.php?repname=asm&path=%2Ftrunk%2Fasm%2Fsrc%2Forg%2Fobjectweb%2Fasm%2FClassVisitor.java

[2] https://github.com/apache/flink/blob/master/flink-scala/src/main/scala/org/apache/flink/api/scala/ClosureCleaner.scala#L279

[Zach Cox]

After some poking around I noticed that flink-connector-elasticsearch_2.10-1.0.0.jar contains shaded asm classes. If I remove that dependency from my project then I do not get the IllegalArgumentException.

On Thu, Mar 10, 2016 at 11:51 AM Zach Cox <[hidden email]> wrote:

Here are the jars on the classpath when I try to run our Flink job in a local environment (via `sbt run`): 

https://gist.githubusercontent.com/zcox/0992aba1c517b51dc879/raw/7136ec034c2beef04bd65de9f125ce3796db511f/gistfile1.txt

There are many transitive dependencies pulled in from internal library projects that probably need to be cleaned out. Maybe we are including something that conflicts? Or maybe something important is being excluded?

Are the asm classes included in Flink jars in some shaded form?

Thanks,

Zach

On Thu, Mar 10, 2016 at 5:06 AM Stephan Ewen <[hidden email]> wrote:

Dependency shading changed a bit between RC4 and RC5 - maybe a different minor ASM version is now included in the "test" scope.

Can you share the dependencies of the problematic project?

On Thu, Mar 10, 2016 at 12:26 AM, Zach Cox <[hidden email]> wrote:

I also noticed when I try to run this application in a local environment, I get the same IllegalArgumentException.

When I assemble this application into a fat jar and run it on a Flink cluster using the CLI tools, it seems to run fine. 

Maybe my local classpath is missing something that is provided on the Flink task managers?

-Zach

On Wed, Mar 9, 2016 at 5:16 PM Zach Cox <[hidden email]> wrote:

Hi - after upgrading to 1.0.0, I'm getting this exception now in a unit test:

   IllegalArgumentException:   (null:-1)

org.apache.flink.shaded.org.objectweb.asm.ClassVisitor.<init>(Unknown Source)

org.apache.flink.shaded.org.objectweb.asm.ClassVisitor.<init>(Unknown Source)

org.apache.flink.api.scala.InnerClosureFinder.<init>(ClosureCleaner.scala:279)

org.apache.flink.api.scala.ClosureCleaner$.getInnerClasses(ClosureCleaner.scala:95)

org.apache.flink.api.scala.ClosureCleaner$.clean(ClosureCleaner.scala:115)

org.apache.flink.streaming.api.scala.StreamExecutionEnvironment.scalaClean(StreamExecutionEnvironment.scala:568)

org.apache.flink.streaming.api.scala.StreamExecutionEnvironment.addSource(StreamExecutionEnvironment.scala:498)

The line that causes that exception is just adding a FlinkKafkaConsumer08 source.

ClassVisitor [1] seems to throw that IllegalArgumentException when it is not given a valid api version number, but InnerClosureFinder [2] looks fine to me.

Any idea what might be causing this? This unit test worked fine with 1.0.0-rc0 jars.

Thanks,

Zach

[1] http://websvn.ow2.org/filedetails.php?repname=asm&path=%2Ftrunk%2Fasm%2Fsrc%2Forg%2Fobjectweb%2Fasm%2FClassVisitor.java

[2] https://github.com/apache/flink/blob/master/flink-scala/src/main/scala/org/apache/flink/api/scala/ClosureCleaner.scala#L279

[Andrew Whitaker]

We're having the same issue (we also have a dependency on flink-connector-elasticsearch). It's only happening to us in IntelliJ though. Is this the case for you as well?

On Thu, Mar 10, 2016 at 3:20 PM, Zach Cox <[hidden email]> wrote:

After some poking around I noticed that flink-connector-elasticsearch_2.10-1.0.0.jar contains shaded asm classes. If I remove that dependency from my project then I do not get the IllegalArgumentException.

On Thu, Mar 10, 2016 at 11:51 AM Zach Cox <[hidden email]> wrote:

Here are the jars on the classpath when I try to run our Flink job in a local environment (via `sbt run`): 

https://gist.githubusercontent.com/zcox/0992aba1c517b51dc879/raw/7136ec034c2beef04bd65de9f125ce3796db511f/gistfile1.txt

There are many transitive dependencies pulled in from internal library projects that probably need to be cleaned out. Maybe we are including something that conflicts? Or maybe something important is being excluded?

Are the asm classes included in Flink jars in some shaded form?

Thanks,

Zach

On Thu, Mar 10, 2016 at 5:06 AM Stephan Ewen <[hidden email]> wrote:

Dependency shading changed a bit between RC4 and RC5 - maybe a different minor ASM version is now included in the "test" scope.

Can you share the dependencies of the problematic project?

On Thu, Mar 10, 2016 at 12:26 AM, Zach Cox <[hidden email]> wrote:

I also noticed when I try to run this application in a local environment, I get the same IllegalArgumentException.

When I assemble this application into a fat jar and run it on a Flink cluster using the CLI tools, it seems to run fine. 

Maybe my local classpath is missing something that is provided on the Flink task managers?

-Zach

On Wed, Mar 9, 2016 at 5:16 PM Zach Cox <[hidden email]> wrote:

Hi - after upgrading to 1.0.0, I'm getting this exception now in a unit test:

   IllegalArgumentException:   (null:-1)

org.apache.flink.shaded.org.objectweb.asm.ClassVisitor.<init>(Unknown Source)

org.apache.flink.shaded.org.objectweb.asm.ClassVisitor.<init>(Unknown Source)

org.apache.flink.api.scala.InnerClosureFinder.<init>(ClosureCleaner.scala:279)

org.apache.flink.api.scala.ClosureCleaner$.getInnerClasses(ClosureCleaner.scala:95)

org.apache.flink.api.scala.ClosureCleaner$.clean(ClosureCleaner.scala:115)

org.apache.flink.streaming.api.scala.StreamExecutionEnvironment.scalaClean(StreamExecutionEnvironment.scala:568)

org.apache.flink.streaming.api.scala.StreamExecutionEnvironment.addSource(StreamExecutionEnvironment.scala:498)

The line that causes that exception is just adding a FlinkKafkaConsumer08 source.

ClassVisitor [1] seems to throw that IllegalArgumentException when it is not given a valid api version number, but InnerClosureFinder [2] looks fine to me.

Any idea what might be causing this? This unit test worked fine with 1.0.0-rc0 jars.

Thanks,

Zach

[1] http://websvn.ow2.org/filedetails.php?repname=asm&path=%2Ftrunk%2Fasm%2Fsrc%2Forg%2Fobjectweb%2Fasm%2FClassVisitor.java

[2] https://github.com/apache/flink/blob/master/flink-scala/src/main/scala/org/apache/flink/api/scala/ClosureCleaner.scala#L279

--

[Zach Cox]

Yes pretty much - we use sbt to run the job in a local environment, not Intellij, but should be the same thing. We were also seeing that exception running unit tests locally. We did not see the exception when assembling a fat jar and submitting to a remote Flink cluster.

It seems like the flink-connector-elasticsearch jar should not have shaded classes in it. Maybe that jar in maven central was built incorrectly?

We worked around this by just not depending on that elasticsearch connector at all, since we wrote our own connector for Elasticsearch 2.x.

-Zach

On Mon, Mar 14, 2016 at 2:03 PM Andrew Whitaker <[hidden email]> wrote:

We're having the same issue (we also have a dependency on flink-connector-elasticsearch). It's only happening to us in IntelliJ though. Is this the case for you as well?

On Thu, Mar 10, 2016 at 3:20 PM, Zach Cox <[hidden email]> wrote:

After some poking around I noticed that flink-connector-elasticsearch_2.10-1.0.0.jar contains shaded asm classes. If I remove that dependency from my project then I do not get the IllegalArgumentException.

On Thu, Mar 10, 2016 at 11:51 AM Zach Cox <[hidden email]> wrote:

Here are the jars on the classpath when I try to run our Flink job in a local environment (via `sbt run`): 

https://gist.githubusercontent.com/zcox/0992aba1c517b51dc879/raw/7136ec034c2beef04bd65de9f125ce3796db511f/gistfile1.txt

There are many transitive dependencies pulled in from internal library projects that probably need to be cleaned out. Maybe we are including something that conflicts? Or maybe something important is being excluded?

Are the asm classes included in Flink jars in some shaded form?

Thanks,

Zach

On Thu, Mar 10, 2016 at 5:06 AM Stephan Ewen <[hidden email]> wrote:

Dependency shading changed a bit between RC4 and RC5 - maybe a different minor ASM version is now included in the "test" scope.

Can you share the dependencies of the problematic project?

On Thu, Mar 10, 2016 at 12:26 AM, Zach Cox <[hidden email]> wrote:

I also noticed when I try to run this application in a local environment, I get the same IllegalArgumentException.

When I assemble this application into a fat jar and run it on a Flink cluster using the CLI tools, it seems to run fine. 

Maybe my local classpath is missing something that is provided on the Flink task managers?

-Zach

On Wed, Mar 9, 2016 at 5:16 PM Zach Cox <[hidden email]> wrote:

Hi - after upgrading to 1.0.0, I'm getting this exception now in a unit test:

   IllegalArgumentException:   (null:-1)

org.apache.flink.shaded.org.objectweb.asm.ClassVisitor.<init>(Unknown Source)

org.apache.flink.shaded.org.objectweb.asm.ClassVisitor.<init>(Unknown Source)

org.apache.flink.api.scala.InnerClosureFinder.<init>(ClosureCleaner.scala:279)

org.apache.flink.api.scala.ClosureCleaner$.getInnerClasses(ClosureCleaner.scala:95)

org.apache.flink.api.scala.ClosureCleaner$.clean(ClosureCleaner.scala:115)

org.apache.flink.streaming.api.scala.StreamExecutionEnvironment.scalaClean(StreamExecutionEnvironment.scala:568)

org.apache.flink.streaming.api.scala.StreamExecutionEnvironment.addSource(StreamExecutionEnvironment.scala:498)

The line that causes that exception is just adding a FlinkKafkaConsumer08 source.

ClassVisitor [1] seems to throw that IllegalArgumentException when it is not given a valid api version number, but InnerClosureFinder [2] looks fine to me.

Any idea what might be causing this? This unit test worked fine with 1.0.0-rc0 jars.

Thanks,

Zach

[1] http://websvn.ow2.org/filedetails.php?repname=asm&path=%2Ftrunk%2Fasm%2Fsrc%2Forg%2Fobjectweb%2Fasm%2FClassVisitor.java

[2] https://github.com/apache/flink/blob/master/flink-scala/src/main/scala/org/apache/flink/api/scala/ClosureCleaner.scala#L279

--

Andrew Whitaker | [hidden email]

--

Note: this information is confidential. It is prohibited to share, post online or otherwise publicize without Braintree's prior written consent.

[Stephan Ewen]

Does this problem persist? (It might have been caused by maven caches with bad artifacts).

The many transitive dependencies you see often come from the connectors - that is also why we do not put the connectors into the lib folder directly, so that these libraries are not always part of every Flink program.

On Mon, Mar 14, 2016 at 9:16 PM, Zach Cox <[hidden email]> wrote:

Yes pretty much - we use sbt to run the job in a local environment, not Intellij, but should be the same thing. We were also seeing that exception running unit tests locally. We did not see the exception when assembling a fat jar and submitting to a remote Flink cluster.

It seems like the flink-connector-elasticsearch jar should not have shaded classes in it. Maybe that jar in maven central was built incorrectly?

We worked around this by just not depending on that elasticsearch connector at all, since we wrote our own connector for Elasticsearch 2.x.

-Zach

On Mon, Mar 14, 2016 at 2:03 PM Andrew Whitaker <[hidden email]> wrote:

We're having the same issue (we also have a dependency on flink-connector-elasticsearch). It's only happening to us in IntelliJ though. Is this the case for you as well?

On Thu, Mar 10, 2016 at 3:20 PM, Zach Cox <[hidden email]> wrote:

After some poking around I noticed that flink-connector-elasticsearch_2.10-1.0.0.jar contains shaded asm classes. If I remove that dependency from my project then I do not get the IllegalArgumentException.

On Thu, Mar 10, 2016 at 11:51 AM Zach Cox <[hidden email]> wrote:

Here are the jars on the classpath when I try to run our Flink job in a local environment (via `sbt run`): 

https://gist.githubusercontent.com/zcox/0992aba1c517b51dc879/raw/7136ec034c2beef04bd65de9f125ce3796db511f/gistfile1.txt

There are many transitive dependencies pulled in from internal library projects that probably need to be cleaned out. Maybe we are including something that conflicts? Or maybe something important is being excluded?

Are the asm classes included in Flink jars in some shaded form?

Thanks,

Zach

On Thu, Mar 10, 2016 at 5:06 AM Stephan Ewen <[hidden email]> wrote:

Dependency shading changed a bit between RC4 and RC5 - maybe a different minor ASM version is now included in the "test" scope.

Can you share the dependencies of the problematic project?

On Thu, Mar 10, 2016 at 12:26 AM, Zach Cox <[hidden email]> wrote:

I also noticed when I try to run this application in a local environment, I get the same IllegalArgumentException.

When I assemble this application into a fat jar and run it on a Flink cluster using the CLI tools, it seems to run fine. 

Maybe my local classpath is missing something that is provided on the Flink task managers?

-Zach

On Wed, Mar 9, 2016 at 5:16 PM Zach Cox <[hidden email]> wrote:

Hi - after upgrading to 1.0.0, I'm getting this exception now in a unit test:

   IllegalArgumentException:   (null:-1)

org.apache.flink.shaded.org.objectweb.asm.ClassVisitor.<init>(Unknown Source)

org.apache.flink.shaded.org.objectweb.asm.ClassVisitor.<init>(Unknown Source)

org.apache.flink.api.scala.InnerClosureFinder.<init>(ClosureCleaner.scala:279)

org.apache.flink.api.scala.ClosureCleaner$.getInnerClasses(ClosureCleaner.scala:95)

org.apache.flink.api.scala.ClosureCleaner$.clean(ClosureCleaner.scala:115)

org.apache.flink.streaming.api.scala.StreamExecutionEnvironment.scalaClean(StreamExecutionEnvironment.scala:568)

org.apache.flink.streaming.api.scala.StreamExecutionEnvironment.addSource(StreamExecutionEnvironment.scala:498)

The line that causes that exception is just adding a FlinkKafkaConsumer08 source.

ClassVisitor [1] seems to throw that IllegalArgumentException when it is not given a valid api version number, but InnerClosureFinder [2] looks fine to me.

Any idea what might be causing this? This unit test worked fine with 1.0.0-rc0 jars.

Thanks,

Zach

[1] http://websvn.ow2.org/filedetails.php?repname=asm&path=%2Ftrunk%2Fasm%2Fsrc%2Forg%2Fobjectweb%2Fasm%2FClassVisitor.java

[2] https://github.com/apache/flink/blob/master/flink-scala/src/main/scala/org/apache/flink/api/scala/ClosureCleaner.scala#L279

--

Andrew Whitaker | [hidden email]

--

Note: this information is confidential. It is prohibited to share, post online or otherwise publicize without Braintree's prior written consent.

[David Kim]

Hi Stephan!

Following up on this issue, it seems the issue doesn't show itself when using version 1.0.1. I'm able to run our unit tests in IntelliJ now.

Thanks!

David

On Wed, Apr 13, 2016 at 1:59 PM Stephan Ewen <[hidden email]> wrote:

Does this problem persist? (It might have been caused by maven caches with bad artifacts).

The many transitive dependencies you see often come from the connectors - that is also why we do not put the connectors into the lib folder directly, so that these libraries are not always part of every Flink program.

On Mon, Mar 14, 2016 at 9:16 PM, Zach Cox <[hidden email]> wrote:

Yes pretty much - we use sbt to run the job in a local environment, not Intellij, but should be the same thing. We were also seeing that exception running unit tests locally. We did not see the exception when assembling a fat jar and submitting to a remote Flink cluster.

It seems like the flink-connector-elasticsearch jar should not have shaded classes in it. Maybe that jar in maven central was built incorrectly?

We worked around this by just not depending on that elasticsearch connector at all, since we wrote our own connector for Elasticsearch 2.x.

-Zach

On Mon, Mar 14, 2016 at 2:03 PM Andrew Whitaker <[hidden email]> wrote:

We're having the same issue (we also have a dependency on flink-connector-elasticsearch). It's only happening to us in IntelliJ though. Is this the case for you as well?

On Thu, Mar 10, 2016 at 3:20 PM, Zach Cox <[hidden email]> wrote:

After some poking around I noticed that flink-connector-elasticsearch_2.10-1.0.0.jar contains shaded asm classes. If I remove that dependency from my project then I do not get the IllegalArgumentException.

On Thu, Mar 10, 2016 at 11:51 AM Zach Cox <[hidden email]> wrote:

Here are the jars on the classpath when I try to run our Flink job in a local environment (via `sbt run`): 

https://gist.githubusercontent.com/zcox/0992aba1c517b51dc879/raw/7136ec034c2beef04bd65de9f125ce3796db511f/gistfile1.txt

There are many transitive dependencies pulled in from internal library projects that probably need to be cleaned out. Maybe we are including something that conflicts? Or maybe something important is being excluded?

Are the asm classes included in Flink jars in some shaded form?

Thanks,

Zach

On Thu, Mar 10, 2016 at 5:06 AM Stephan Ewen <[hidden email]> wrote:

Dependency shading changed a bit between RC4 and RC5 - maybe a different minor ASM version is now included in the "test" scope.

Can you share the dependencies of the problematic project?

On Thu, Mar 10, 2016 at 12:26 AM, Zach Cox <[hidden email]> wrote:

I also noticed when I try to run this application in a local environment, I get the same IllegalArgumentException.

When I assemble this application into a fat jar and run it on a Flink cluster using the CLI tools, it seems to run fine. 

Maybe my local classpath is missing something that is provided on the Flink task managers?

-Zach

On Wed, Mar 9, 2016 at 5:16 PM Zach Cox <[hidden email]> wrote:

Hi - after upgrading to 1.0.0, I'm getting this exception now in a unit test:

   IllegalArgumentException:   (null:-1)

org.apache.flink.shaded.org.objectweb.asm.ClassVisitor.<init>(Unknown Source)

org.apache.flink.shaded.org.objectweb.asm.ClassVisitor.<init>(Unknown Source)

org.apache.flink.api.scala.InnerClosureFinder.<init>(ClosureCleaner.scala:279)

org.apache.flink.api.scala.ClosureCleaner$.getInnerClasses(ClosureCleaner.scala:95)

org.apache.flink.api.scala.ClosureCleaner$.clean(ClosureCleaner.scala:115)

org.apache.flink.streaming.api.scala.StreamExecutionEnvironment.scalaClean(StreamExecutionEnvironment.scala:568)

org.apache.flink.streaming.api.scala.StreamExecutionEnvironment.addSource(StreamExecutionEnvironment.scala:498)

The line that causes that exception is just adding a FlinkKafkaConsumer08 source.

ClassVisitor [1] seems to throw that IllegalArgumentException when it is not given a valid api version number, but InnerClosureFinder [2] looks fine to me.

Any idea what might be causing this? This unit test worked fine with 1.0.0-rc0 jars.

Thanks,

Zach

[1] http://websvn.ow2.org/filedetails.php?repname=asm&path=%2Ftrunk%2Fasm%2Fsrc%2Forg%2Fobjectweb%2Fasm%2FClassVisitor.java

[2] https://github.com/apache/flink/blob/master/flink-scala/src/main/scala/org/apache/flink/api/scala/ClosureCleaner.scala#L279

--

Andrew Whitaker | [hidden email]

--

Note: this information is confidential. It is prohibited to share, post online or otherwise publicize without Braintree's prior written consent.