Failure running Flink locally with flink-s3-fs-hadoop + AWS SDK v2 as a dependency

[Yuval Itzchakov]

Hi,

I'm trying to run Flink with AWS SDK v2 and restore a Flink savepoint that's stored in S3.

When adding the dependency on flink-s3-fs-hadoop locally, I fail to run Flink at runtime with the following error:

Caused by: java.lang.invoke.LambdaConversionException: Invalid receiver type interface org.apache.http.Header; not a subtype of implementation type interface org.apache.http.NameValuePair

This happens because the flink-s3 JAR is a fat JAR containing an old version of org.apache.http.HttpMessage which does not implement the NameValuePair interface (there was a non backwards compatible change in the library, see https://github.com/aws/aws-sdk-java-v2/issues/652)

Since the library itself isn't shaded, it gets loaded first overriding the HttpClient newer version I have on the classpath and blows up when bootstrapping Flink. 

Any advice on how to work around this?

--