ElasticSearch RestClient throws NoSuchMethodError due to shade mechanism

[徐涛]

Hi All,

I use the following code try to build a RestClient

org.elasticsearch.client.RestClient.builder(  new HttpHost(xxx, xxx,"http")  ).build()

but when in running time, a NoSuchMethodError throws out, I think the reason is:

There are two RestClient classes, one is in the jar I include, the other one is in flink-connector-elasticsearch5, but the argument of build method in flink-connector-elasticsearch5 is org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.apache.http.HttpHost. So I want to know why org.elasticsearch.client.RestClientBuilder is not shaded, so runtime class conflict could be avoided?

        public static RestClientBuilder builder(org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.apache.http.HttpHost... hosts) {

            return new RestClientBuilder(hosts);

        }

Best

Henry

[Rong Rong]

Hi Henry,

I was not sure if this is the suggested way. but from what I understand of the pom file in elasticsearch5, you are allowed to change the sub version of the org.ealisticsearch.client via manually override using 

-Delasticsearch.version=5.x.x 

during maven build progress if you are using a different sub version. 

This way you don't need to include 2 jars of the elasticsearch client. Does this resolves your problem?

--

Rong

On Tue, Jan 15, 2019 at 2:39 AM 徐涛 <[hidden email]> wrote:

Hi All,

I use the following code try to build a RestClient

org.elasticsearch.client.RestClient.builder(  new HttpHost(xxx, xxx,"http")  ).build()

but when in running time, a NoSuchMethodError throws out, I think the reason is:

There are two RestClient classes, one is in the jar I include, the other one is in flink-connector-elasticsearch5, but the argument of build method in flink-connector-elasticsearch5 is org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.apache.http.HttpHost. So I want to know why org.elasticsearch.client.RestClientBuilder is not shaded, so runtime class conflict could be avoided?

        public static RestClientBuilder builder(org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.apache.http.HttpHost... hosts) {

            return new RestClientBuilder(hosts);

        }

Best

Henry

[Gary Yao-3]

Hi Henry,

Can you share your pom.xml and the full stacktrace with us? It is expected
behavior that org.elasticsearch.client.RestClientBuilder is not shaded. That
class comes from the elasticsearch Java client, and we only shade its
transitive dependencies. Could it be that you have a dependency in your
job's pom.xml to a different version of the elasticsearch client?

Best,
Gary

On Tue, Jan 15, 2019 at 11:39 AM 徐涛 <[hidden email]> wrote:

Hi All,

I use the following code try to build a RestClient

org.elasticsearch.client.RestClient.builder(  new HttpHost(xxx, xxx,"http")  ).build()

but when in running time, a NoSuchMethodError throws out, I think the reason is:

There are two RestClient classes, one is in the jar I include, the other one is in flink-connector-elasticsearch5, but the argument of build method in flink-connector-elasticsearch5 is org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.apache.http.HttpHost. So I want to know why org.elasticsearch.client.RestClientBuilder is not shaded, so runtime class conflict could be avoided?

        public static RestClientBuilder builder(org.apache.flink.streaming.connectors.elasticsearch5.shaded.org.apache.http.HttpHost... hosts) {

            return new RestClientBuilder(hosts);

        }

Best

Henry