Accumulator with Elasticsearch Sink

[Hung]

Hi Flink users,

Did someone use accumulator with Elasticsearch Sink? So we can better
compare the last timestamps in the sink and the last timestamps in
Elasticsearch, in order to see how long does it take from the Elasticsearch
sink to Elasticsearch.

Best,

Sendoh



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[Tzu-Li (Gordon) Tai]

Hi Sendoh,

That sounds like a reasonable metric to add directly to the Elasticsearch connector.

Could you perhaps write a comment on that in https://issues.apache.org/jira/browse/FLINK-7697?

Cheers,

Gordon

On 19 October 2017 at 8:57:23 PM, Sendoh ([hidden email]) wrote:

Hi Flink users,

Did someone use accumulator with Elasticsearch Sink? So we can better
compare the last timestamps in the sink and the last timestamps in
Elasticsearch, in order to see how long does it take from the Elasticsearch
sink to Elasticsearch.

Best,

Sendoh



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[Hung]

Hi Gordon,

I commented the ticket. What would you think I start to implement it as
there is no progress recently?

I see it's only about adding metrics in the following class? and anyway I
probably would have to build it soon.

https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-elasticsearch-base/src/main/java/org/apache/flink/streaming/connectors/elasticsearch/ElasticsearchSinkBase.java

The other thing I find is my usecase needs accumulator than metric, as the global status is more useful for recognizing the latency of the job, the same as the following discussion:
http://mail-archives.apache.org/mod_mbox/flink-dev/201609.mbox/[e-mail]%3E

Cheers,

Sendoh



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/