Re: Flink cluster : Client is not connected to any Elasticsearch nodes!

[Tzu-Li (Gordon) Tai]

Hi Dhinesh,

Could it be that you didn’t configure the network binding address of the ES installation properly?

You need to make sure it isn’t binded to 127.0.0.1, which I think in some Elasticsearch versions is the default binding.

Also, just a reminder if you haven’t done so, please make sure that the ES dependencies is properly bundled for cluster execution. See [1] for details on this.

Cheers,

Gordon

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/linking.html

On 6 June 2017 at 12:01:54 AM, dhinesh raja ([hidden email]) wrote:

Dear Team,

I am running flink streaming job with Elasticsearch connector2. I am able to run in my eclipse but when I run in flink local cluster I got this error. could you please help me in this?I have attached my code. I am using flink 1.2.0 and elastic search 2.x

java.lang.RuntimeException: Client is not connected to any Elasticsearch nodes!
        at org.apache.flink.streaming.connectors.elasticsearch2.ElasticsearchSink.open(ElasticsearchSink.java:172)
        at org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36)
        at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:112)
        at org.apache.flink.streaming.runtime.tasks.StreamTask.openAllOperators(StreamTask.java:375)
        at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:251)
        at org.apache.flink.runtime.taskmanager.Task.run(Task.java:665)
        at java.lang.Thread.run(Thread.java:748)

--

[Tzu-Li (Gordon) Tai]

Hi Dhinesh,

One other thing that came to mind: the Elasticsearch 2 connector, by default, uses ES 2.3.5.

If you’re using an Elasticsearch 2 with major version higher than that, you need to build the connector with the matching version.

When running lower major version clients against a higher major version ES installation, this exception is very common.

Best,

Gordon

On 6 June 2017 at 6:39:52 PM, Tzu-Li (Gordon) Tai ([hidden email]) wrote:

Hi Dhinesh,

Could it be that you didn’t configure the network binding address of the ES installation properly?

You need to make sure it isn’t binded to 127.0.0.1, which I think in some Elasticsearch versions is the default binding.

Also, just a reminder if you haven’t done so, please make sure that the ES dependencies is properly bundled for cluster execution. See [1] for details on this.

Cheers,

Gordon

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/linking.html

On 6 June 2017 at 12:01:54 AM, dhinesh raja ([hidden email]) wrote:

Dear Team,

I am running flink streaming job with Elasticsearch connector2. I am able to run in my eclipse but when I run in flink local cluster I got this error. could you please help me in this?I have attached my code. I am using flink 1.2.0 and elastic search 2.x

java.lang.RuntimeException: Client is not connected to any Elasticsearch nodes!
        at org.apache.flink.streaming.connectors.elasticsearch2.ElasticsearchSink.open(ElasticsearchSink.java:172)
        at org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36)
        at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:112)
        at org.apache.flink.streaming.runtime.tasks.StreamTask.openAllOperators(StreamTask.java:375)
        at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:251)
        at org.apache.flink.runtime.taskmanager.Task.run(Task.java:665)
        at java.lang.Thread.run(Thread.java:748)

--

[Flavio Pompermaier]

When this connector was improved to be resilient to ES problems we used to use Logstash to index on ES and it was really cumbersome...this connector ease a lot the work of indexing to ES: it's much faster, it can index directly without persisting to file and it's a lot much easier to filter documents or remove attributes. Moreover it removes a somehow useless component from our architecture. We still have only the problem with the shading of Flink and conflict between guava versions but we solved the problem shading ES and using this version as dependency of the Flink ES connector.

Best,

Flavio

On Fri, Jun 16, 2017 at 11:14 AM, Tzu-Li (Gordon) Tai <[hidden email]> wrote:

Hi Dhinesh,

One other thing that came to mind: the Elasticsearch 2 connector, by default, uses ES 2.3.5.

If you’re using an Elasticsearch 2 with major version higher than that, you need to build the connector with the matching version.

When running lower major version clients against a higher major version ES installation, this exception is very common.

Best,

Gordon

On 6 June 2017 at 6:39:52 PM, Tzu-Li (Gordon) Tai ([hidden email]) wrote:

Hi Dhinesh,

Could it be that you didn’t configure the network binding address of the ES installation properly?

You need to make sure it isn’t binded to 127.0.0.1, which I think in some Elasticsearch versions is the default binding.

Also, just a reminder if you haven’t done so, please make sure that the ES dependencies is properly bundled for cluster execution. See [1] for details on this.

Cheers,

Gordon

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/linking.html

On 6 June 2017 at 12:01:54 AM, dhinesh raja ([hidden email]) wrote:

Dear Team,

I am running flink streaming job with Elasticsearch connector2. I am able to run in my eclipse but when I run in flink local cluster I got this error. could you please help me in this?I have attached my code. I am using flink 1.2.0 and elastic search 2.x

java.lang.RuntimeException: Client is not connected to any Elasticsearch nodes!
        at org.apache.flink.streaming.connectors.elasticsearch2.ElasticsearchSink.open(ElasticsearchSink.java:172)
        at org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36)
        at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:112)
        at org.apache.flink.streaming.runtime.tasks.StreamTask.openAllOperators(StreamTask.java:375)
        at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:251)
        at org.apache.flink.runtime.taskmanager.Task.run(Task.java:665)
        at java.lang.Thread.run(Thread.java:748)

--