Hi,
You should setup a metric reporter to collect Flink's metrics.
https://ci.apache.org/projects/flink/flink-docs-release-1.2/monitoring/metrics.htmlThere's a lot of useful information in the metrics, including the
consumer lags.
I'm using the Graphite reporter with InfluxDB for storage + Grafana to
display the metrics,
it's been working great so far.
Regards,
Kien
On 7/9/17 1:16 PM, Karthik Deivasigamani wrote:
> Hi,
> I'm using Flink 1.2.1 and FlinkKafkaConsumer09 to read data from
> Kafka Server (0.10.1.0). My consumer is able to read and everything
> works fine.
> But when I use the kafka_consumer_groups.sh command to find the lag
> metrics it does not return the lag. Upon reading online I found that
> Flink Kafka Consumer uses the low level API and manages the assignment
> of partition itself.
>
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-kafka-group-question-td8185.html>
> What is the best way to monitor my flink kafka consumer's lag? Does
> Flink expose the lag metrics per partition somewhere?
>
> ~
> Karthik
Locked
