ElasticSearch Connector for version 6.x and scala 2.11

[Rahul Raj]

Hi All,

Is there a Flink Elastic search connector for version 6.0 and scala 2.11? I couldn't find it listed here https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/connectors/elasticsearch.html .

Regards,

Rahul Raj

[Fabian Hueske-2]

Hi Rahul,

Flink does not provide a connector for ElasticSearch 6 yet.

There is this JIRA issue to track the development progress [1].

Best, Fabian

[1] https://issues.apache.org/jira/browse/FLINK-8101

2017-12-01 7:22 GMT+01:00 Rahul Raj <[hidden email]>:

Hi All,

Is there a Flink Elastic search connector for version 6.0 and scala 2.11? I couldn't find it listed here https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/connectors/elasticsearch.html .

Regards,

Rahul Raj

[Jens Oberender]

Hi

A workmate of mine tried to migrate the existing flink connector to
ElasticSearch 6 but we had problems with netty dependencies that clashed
(Flink uses 4.0.27 and ES is on 4.1).
You can change the flink-connector-elasticsearch5 connector to ES 5.6.4,
but then you have to do some adaptions to get it working, as they
changed the API within a major version!
But with that version you can write to ES 6.

He put his changes on Github:
https://github.com/cognitix/flink/tree/connector-elasticsearch6
It's called flink-connector-elasticsearch6, there.

We want to write a new and clean version for ElasticSearch 6, but
currently don't have the time.

Best regards,
        Jens Oberender


Am 01.12.2017 um 09:52 schrieb Fabian Hueske:

> Hi Rahul,
>
> Flink does not provide a connector for ElasticSearch 6 yet.
> There is this JIRA issue to track the development progress [1].
>
> Best, Fabian
>
> [1] https://issues.apache.org/jira/browse/FLINK-8101
>
> 2017-12-01 7:22 GMT+01:00 Rahul Raj <[hidden email]
> <mailto:[hidden email]>>:
>
>     Hi All,
>
>     Is there a Flink Elastic search connector for version 6.0 and scala
>     2.11? I couldn't find it listed
>     here https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/connectors/elasticsearch.html
>     <https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/connectors/elasticsearch.html>
>     .
>
>     Regards,
>     Rahul Raj
>
>

--
Jens Oberender
Lead Java Developer

cognitix GmbH
Neumarkt 29-33
DE-04109 Leipzig

Tel.: [phone]
Fax.: [phone]
E-Mail: [hidden email]
Internet: www.cognitix.de

Gesellschaft mit beschränkter Haftung
Geschäftsführer: Tilo Dinger
Handelsregister: Amtsgericht Leipzig, HRB 33904

PGP key fingerprint
5DC9 0F2B F2FE 6D74 F95D 1DC3 B926 6BB4 E2C5 C3A5

[Nico Kruber]

Actually, Flink's netty dependency (4.0.27) is shaded away into the
"org.apache.flink.shaded.netty4.io.netty" package now (since version
1.4) and should thus not clash anymore.
However, other netty versions may come into play from the job itself or
from the integration of Hadoop's classpath (if available).


Nico

On 01/12/17 14:42, Jens Oberender wrote:

> Hi
>
> A workmate of mine tried to migrate the existing flink connector to
> ElasticSearch 6 but we had problems with netty dependencies that clashed
> (Flink uses 4.0.27 and ES is on 4.1).
> You can change the flink-connector-elasticsearch5 connector to ES 5.6.4,
> but then you have to do some adaptions to get it working, as they
> changed the API within a major version!
> But with that version you can write to ES 6.
>
> He put his changes on Github:
> https://github.com/cognitix/flink/tree/connector-elasticsearch6
> It's called flink-connector-elasticsearch6, there.
>
> We want to write a new and clean version for ElasticSearch 6, but
> currently don't have the time.
>
> Best regards,
> Jens Oberender
>
>
> Am 01.12.2017 um 09:52 schrieb Fabian Hueske:
>> Hi Rahul,
>>
>> Flink does not provide a connector for ElasticSearch 6 yet.
>> There is this JIRA issue to track the development progress [1].
>>
>> Best, Fabian
>>
>> [1] https://issues.apache.org/jira/browse/FLINK-8101
>>
>> 2017-12-01 7:22 GMT+01:00 Rahul Raj <[hidden email]
>> <mailto:[hidden email]>>:
>>
>>     Hi All,
>>
>>     Is there a Flink Elastic search connector for version 6.0 and scala
>>     2.11? I couldn't find it listed
>>     here https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/connectors/elasticsearch.html
>>     <https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/connectors/elasticsearch.html>
>>     .
>>
>>     Regards,
>>     Rahul Raj
>>
>>
>