Metrics not reported to graphite

[Jack Huang-2]

Hi all,

I followed the instruction for reporting metrics to a Graphite server on the official document (https://ci.apache.org/projects/flink/flink-docs-release-1.1/apis/metrics.html#metric-types). 

Specifically, I have the following config/code in my project

metrics.reporters: graphite
metrics.reporter.graphite.class: org.apache.flink.metrics.graphite.GraphiteReporter
metrics.reporter.graphite.host: node12
metrics.reporter.graphite.port: 2003

​

events.filter(new RichFilterFunction[Event]() {
    lazy val incomingCounter = getRuntimeContext.getMetricGroup.counter("event.incoming")
    lazy val downsampledCounter = getRuntimeContext.getMetricGroup.counter("event.downsampled")
    def filter(event:Event):Boolean = {
        incomingCounter.inc
        if(!event.bid_id.startsWith("0")) {
            return false;
        }
        downsampledCounter.inc
        return true;
    }
})

​

However I don't see anything on my graphite server. What am I missing?

Thanks,

Jack

[Greg Hogan]

Have you copied the required jar files into your lib/ directory? Only JMX support is provided in the distribution.

On Thu, Sep 1, 2016 at 5:07 PM, Jack Huang <[hidden email]> wrote:

Hi all,

I followed the instruction for reporting metrics to a Graphite server on the official document (https://ci.apache.org/projects/flink/flink-docs-release-1.1/apis/metrics.html#metric-types). 

Specifically, I have the following config/code in my project

metrics.reporters: graphite
metrics.reporter.graphite.class: org.apache.flink.metrics.graphite.GraphiteReporter
metrics.reporter.graphite.host: node12
metrics.reporter.graphite.port: 2003

​

events.filter(new RichFilterFunction[Event]() {
    lazy val incomingCounter = getRuntimeContext.getMetricGroup.counter("event.incoming")
    lazy val downsampledCounter = getRuntimeContext.getMetricGroup.counter("event.downsampled")
    def filter(event:Event):Boolean = {
        incomingCounter.inc
        if(!event.bid_id.startsWith("0")) {
            return false;
        }
        downsampledCounter.inc
        return true;
    }
})

​

However I don't see anything on my graphite server. What am I missing?

Thanks,

Jack

[Jack Huang-2]

Hi Greg,

Following your hint, I found the solution here (https://issues.apache.org/jira/browse/FLINK-4396) and resolved the issue. I had to put all three jars to the lib directory to get it to work.

A follow up questions: can I put a prefix (e.g. flink) to all flink metrics instead of having their names starting with the host ip address?

Thanks,
Jack

On Thu, Sep 1, 2016 at 3:04 PM, Greg Hogan <[hidden email]> wrote:

Have you copied the required jar files into your lib/ directory? Only JMX support is provided in the distribution.

On Thu, Sep 1, 2016 at 5:07 PM, Jack Huang <[hidden email]> wrote:

Hi all,

I followed the instruction for reporting metrics to a Graphite server on the official document (https://ci.apache.org/projects/flink/flink-docs-release-1.1/apis/metrics.html#metric-types). 

Specifically, I have the following config/code in my project

metrics.reporters: graphite
metrics.reporter.graphite.class: org.apache.flink.metrics.graphite.GraphiteReporter
metrics.reporter.graphite.host: node12
metrics.reporter.graphite.port: 2003

​

events.filter(new RichFilterFunction[Event]() {
    lazy val incomingCounter = getRuntimeContext.getMetricGroup.counter("event.incoming")
    lazy val downsampledCounter = getRuntimeContext.getMetricGroup.counter("event.downsampled")
    def filter(event:Event):Boolean = {
        incomingCounter.inc
        if(!event.bid_id.startsWith("0")) {
            return false;
        }
        downsampledCounter.inc
        return true;
    }
})

​

However I don't see anything on my graphite server. What am I missing?

Thanks,

Jack

[Jack Huang-2]

Found the solution to the follow-up question:
https://ci.apache.org/projects/flink/flink-docs-release-1.1/setup/config.html#metrics

On Thu, Sep 1, 2016 at 3:46 PM, Jack Huang <[hidden email]> wrote:

Hi Greg,

Following your hint, I found the solution here (https://issues.apache.org/jira/browse/FLINK-4396) and resolved the issue. I had to put all three jars to the lib directory to get it to work.

A follow up questions: can I put a prefix (e.g. flink) to all flink metrics instead of having their names starting with the host ip address?

Thanks,
Jack

On Thu, Sep 1, 2016 at 3:04 PM, Greg Hogan <[hidden email]> wrote:

Have you copied the required jar files into your lib/ directory? Only JMX support is provided in the distribution.

On Thu, Sep 1, 2016 at 5:07 PM, Jack Huang <[hidden email]> wrote:

Hi all,

I followed the instruction for reporting metrics to a Graphite server on the official document (https://ci.apache.org/projects/flink/flink-docs-release-1.1/apis/metrics.html#metric-types). 

Specifically, I have the following config/code in my project

metrics.reporters: graphite
metrics.reporter.graphite.class: org.apache.flink.metrics.graphite.GraphiteReporter
metrics.reporter.graphite.host: node12
metrics.reporter.graphite.port: 2003

​

events.filter(new RichFilterFunction[Event]() {
    lazy val incomingCounter = getRuntimeContext.getMetricGroup.counter("event.incoming")
    lazy val downsampledCounter = getRuntimeContext.getMetricGroup.counter("event.downsampled")
    def filter(event:Event):Boolean = {
        incomingCounter.inc
        if(!event.bid_id.startsWith("0")) {
            return false;
        }
        downsampledCounter.inc
        return true;
    }
})

​

However I don't see anything on my graphite server. What am I missing?

Thanks,

Jack