Kinesis Connector - NoClassDefFoundError

[Steve Bistline]

Hey all... upgrade from Flink 1.5.0 to 1.6.2 and for some reason cannot figure out what I missed in setting up the new environment. I am gettin this error:

java.lang.NoClassDefFoundError: Could not initialize class org.apache.flink.kinesis.shaded.com.amazonaws.partitions.PartitionsLoader
	at org.apache.flink.kinesis.shaded.com.amazonaws.regions.RegionMetadataFactory.create(RegionMetadataFactory.java:30)
	at org.apache.flink.kinesis.shaded.com.amazonaws.regions.RegionUtils.initialize(RegionUtils.java:64)
	at org.apache.flink.kinesis.shaded.com.amazonaws.regions.RegionUtils.getRegionMetadata(RegionUtils.java:52)
	at org.apache.flink.kinesis.shaded.com.amazonaws.regions.RegionUtils.getRegion(RegionUtils.java:105)
	at org.apache.flink.kinesis.shaded.com.amazonaws.client.builder.AwsClientBuilder.withRegion(AwsClientBuilder.java:239)
	at org.apache.flink.kinesis.shaded.com.amazonaws.client.builder.AwsClientBuilder.withRegion(AwsClientBuilder.java:226)
	at org.apache.flink.streaming.connectors.kinesis.util.AWSUtil.createKinesisClient(AWSUtil.java:93)
	at org.apache.flink.streaming.connectors.kinesis.proxy.KinesisProxy.createKinesisClient(KinesisProxy.java:203)
	at org.apache.flink.streaming.connectors.kinesis.proxy.KinesisProxy.<init>(KinesisProxy.java:138)
	at org.apache.flink.streaming.connectors.kinesis.proxy.KinesisProxy.create(KinesisProxy.java:213)
	at org.apache.flink.streaming.connectors.kinesis.internals.KinesisDataFetcher.<init>(KinesisDataFetcher.java:242)
	at org.apache.flink.streaming.connectors.kinesis.internals.KinesisDataFetcher.<init>(KinesisDataFetcher.java:207)
	at org.apache.flink.streaming.connectors.kinesis.FlinkKinesisConsumer.createFetcher(FlinkKinesisConsumer.java:417)
	at org.apache.flink.streaming.connectors.kinesis.FlinkKinesisConsumer.run(FlinkKinesisConsumer.java:233)
	at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:94)
	at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:58)
	at org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run(SourceStreamTask.java:99)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:300)
	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711)
	at java.lang.Thread.run(Thread.java:748)

[Dominik Wosiński]

Hey, 

Have you updated the versions both on the environment and the dependency on the job?
From my personal experience, 95 % of such issues is due to the mismatch between Flink versions on the cluster you are using and Your job. 

Best Regards,
Dom. 

wt., 20 lis 2018 o 07:41 Steve Bistline <[hidden email]> napisał(a):

Hey all... upgrade from Flink 1.5.0 to 1.6.2 and for some reason cannot figure out what I missed in setting up the new environment. I am gettin this error:

java.lang.NoClassDefFoundError: Could not initialize class org.apache.flink.kinesis.shaded.com.amazonaws.partitions.PartitionsLoader
	at org.apache.flink.kinesis.shaded.com.amazonaws.regions.RegionMetadataFactory.create(RegionMetadataFactory.java:30)
	at org.apache.flink.kinesis.shaded.com.amazonaws.regions.RegionUtils.initialize(RegionUtils.java:64)
	at org.apache.flink.kinesis.shaded.com.amazonaws.regions.RegionUtils.getRegionMetadata(RegionUtils.java:52)
	at org.apache.flink.kinesis.shaded.com.amazonaws.regions.RegionUtils.getRegion(RegionUtils.java:105)
	at org.apache.flink.kinesis.shaded.com.amazonaws.client.builder.AwsClientBuilder.withRegion(AwsClientBuilder.java:239)
	at org.apache.flink.kinesis.shaded.com.amazonaws.client.builder.AwsClientBuilder.withRegion(AwsClientBuilder.java:226)
	at org.apache.flink.streaming.connectors.kinesis.util.AWSUtil.createKinesisClient(AWSUtil.java:93)
	at org.apache.flink.streaming.connectors.kinesis.proxy.KinesisProxy.createKinesisClient(KinesisProxy.java:203)
	at org.apache.flink.streaming.connectors.kinesis.proxy.KinesisProxy.<init>(KinesisProxy.java:138)
	at org.apache.flink.streaming.connectors.kinesis.proxy.KinesisProxy.create(KinesisProxy.java:213)
	at org.apache.flink.streaming.connectors.kinesis.internals.KinesisDataFetcher.<init>(KinesisDataFetcher.java:242)
	at org.apache.flink.streaming.connectors.kinesis.internals.KinesisDataFetcher.<init>(KinesisDataFetcher.java:207)
	at org.apache.flink.streaming.connectors.kinesis.FlinkKinesisConsumer.createFetcher(FlinkKinesisConsumer.java:417)
	at org.apache.flink.streaming.connectors.kinesis.FlinkKinesisConsumer.run(FlinkKinesisConsumer.java:233)
	at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:94)
	at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:58)
	at org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run(SourceStreamTask.java:99)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:300)
	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711)
	at java.lang.Thread.run(Thread.java:748)