(DEPRECATED) Apache Flink User Mailing List archive.

Metric Registry Warnings

classic Classic list List threaded Threaded
♦
♦
 7 messages Options
Ashish Pokharel
Reply | Threaded
Open this post in threaded view
♦
♦
|

Metric Registry Warnings

Ashish Pokharel
All,

Hopefully this is a quick one. I enabled Graphite reporter in my App and I started to see the following warning messages all over the place:

2017-11-07 20:54:09,288 WARN  org.apache.flink.runtime.metrics.MetricRegistry               - Error while registering metric.
java.lang.IllegalArgumentException: A metric named flink.taskmanager.pndapoc-cdh-dn-14.8823e32fae717d08e211fceec56479b7.normalizeData.parseRawStats -> Filter.numBytesOut already exists

I saw some threads about this regarding JMX as well but I don’t think I see a resolution for it. 

One thing I made sure was I haven’t reused name (like parseRawStats) in my App. Also, this seems to happen for every metric, not only for a select few where I could have mistakenly set the same name.

Appreciate any help on this.

Thanks, Ashish
Fabian Hueske-2
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Metric Registry Warnings

Fabian Hueske-2
Hi Ashish,

this is a known issue and has been fixed for the next version [1].

Best, Fabian

2017-11-11 16:02 GMT+01:00 Ashish Pokharel <[hidden email]>:
All,

Hopefully this is a quick one. I enabled Graphite reporter in my App and I started to see the following warning messages all over the place:

2017-11-07 20:54:09,288 WARN  org.apache.flink.runtime.metrics.MetricRegistry               - Error while registering metric.
java.lang.IllegalArgumentException: A metric named flink.taskmanager.pndapoc-cdh-dn-14.8823e32fae717d08e211fceec56479b7.normalizeData.parseRawStats -> Filter.numBytesOut already exists

I saw some threads about this regarding JMX as well but I don’t think I see a resolution for it. 

One thing I made sure was I haven’t reused name (like parseRawStats) in my App. Also, this seems to happen for every metric, not only for a select few where I could have mistakenly set the same name.

Appreciate any help on this.

Thanks, Ashish
Ashish Pokharel
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Metric Registry Warnings

Ashish Pokharel
Thanks Fabian!


On Mon, Nov 13, 2017 at 4:44 AM, Fabian Hueske
Hi Ashish,

this is a known issue and has been fixed for the next version [1].

Best, Fabian

2017-11-11 16:02 GMT+01:00 Ashish Pokharel <[hidden email]>:
All,

Hopefully this is a quick one. I enabled Graphite reporter in my App and I started to see the following warning messages all over the place:

2017-11-07 20:54:09,288 WARN  org.apache.flink.runtime. metrics.MetricRegistry                - Error while registering metric.
java.lang. IllegalArgumentException: A metric named flink.taskmanager.pndapoc-cdh- dn-14. 8823e32fae717d08e211fceec56479 b7.normalizeData.parseRawStats -> Filter.numBytesOut already exists

I saw some threads about this regarding JMX as well but I don’t think I see a resolution for it. 

One thing I made sure was I haven’t reused name (like parseRawStats) in my App. Also, this seems to happen for every metric, not only for a select few where I could have mistakenly set the same name.

Appreciate any help on this.

Thanks, Ashish
PedroMrChaves
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Metric Registry Warnings

PedroMrChaves
Hello,

I still have the same issue with Flink Version 1.4.2.

java.lang.IllegalArgumentException: A metric named
<hostname-anonyn>.taskmanager.6aa8d13575228d38ae4abdfb37fa229e.CDC.Source:
EVENTS.1.numRecordsIn already exists
        at
com.codahale.metrics.MetricRegistry.register(MetricRegistry.java:91)
        at
org.apache.flink.dropwizard.ScheduledDropwizardReporter.notifyOfAddedMetric(ScheduledDropwizardReporter.java:131)
        at
org.apache.flink.runtime.metrics.MetricRegistryImpl.register(MetricRegistryImpl.java:319)
        at
org.apache.flink.runtime.metrics.groups.AbstractMetricGroup.addMetric(AbstractMetricGroup.java:370)
        at
org.apache.flink.runtime.metrics.groups.AbstractMetricGroup.counter(AbstractMetricGroup.java:303)
        at
org.apache.flink.runtime.metrics.groups.AbstractMetricGroup.counter(AbstractMetricGroup.java:293)
        at
org.apache.flink.runtime.metrics.groups.OperatorIOMetricGroup.<init>(OperatorIOMetricGroup.java:40)
        at
org.apache.flink.runtime.metrics.groups.OperatorMetricGroup.<init>(OperatorMetricGroup.java:48)
        at
org.apache.flink.runtime.metrics.groups.TaskMetricGroup.addOperator(TaskMetricGroup.java:146)
        at
org.apache.flink.streaming.api.operators.AbstractStreamOperator.setup(AbstractStreamOperator.java:182)
        at
org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.setup(AbstractUdfStreamOperator.java:82)
        at
org.apache.flink.streaming.runtime.tasks.OperatorChain.<init>(OperatorChain.java:136)
        at
org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:231)
        at org.apache.flink.runtime.taskmanager.Task.run(Task.java:718)
        at java.lang.Thread.run(Thread.java:748)



-----
Best Regards,
Pedro Chaves
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
Best Regards,
Pedro Chaves
Fabian Hueske-2
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Metric Registry Warnings

Fabian Hueske-2
Hi Pedro,

Can you reopen FLINK-7100 and post a comment with your error message and environment?

Thanks,
Fabian

2018-03-20 14:58 GMT+01:00 PedroMrChaves <[hidden email]>:
Hello,

I still have the same issue with Flink Version 1.4.2.

java.lang.IllegalArgumentException: A metric named
<hostname-anonyn>.taskmanager.6aa8d13575228d38ae4abdfb37fa229e.CDC.Source:
EVENTS.1.numRecordsIn already exists
        at
com.codahale.metrics.MetricRegistry.register(MetricRegistry.java:91)
        at
org.apache.flink.dropwizard.ScheduledDropwizardReporter.notifyOfAddedMetric(ScheduledDropwizardReporter.java:131)
        at
org.apache.flink.runtime.metrics.MetricRegistryImpl.register(MetricRegistryImpl.java:319)
        at
org.apache.flink.runtime.metrics.groups.AbstractMetricGroup.addMetric(AbstractMetricGroup.java:370)
        at
org.apache.flink.runtime.metrics.groups.AbstractMetricGroup.counter(AbstractMetricGroup.java:303)
        at
org.apache.flink.runtime.metrics.groups.AbstractMetricGroup.counter(AbstractMetricGroup.java:293)
        at
org.apache.flink.runtime.metrics.groups.OperatorIOMetricGroup.<init>(OperatorIOMetricGroup.java:40)
        at
org.apache.flink.runtime.metrics.groups.OperatorMetricGroup.<init>(OperatorMetricGroup.java:48)
        at
org.apache.flink.runtime.metrics.groups.TaskMetricGroup.addOperator(TaskMetricGroup.java:146)
        at
org.apache.flink.streaming.api.operators.AbstractStreamOperator.setup(AbstractStreamOperator.java:182)
        at
org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.setup(AbstractUdfStreamOperator.java:82)
        at
org.apache.flink.streaming.runtime.tasks.OperatorChain.<init>(OperatorChain.java:136)
        at
org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:231)
        at org.apache.flink.runtime.taskmanager.Task.run(Task.java:718)
        at java.lang.Thread.run(Thread.java:748)



-----
Best Regards,
Pedro Chaves
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
Chesnay Schepler
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Metric Registry Warnings

Chesnay Schepler
FLINK-7100 is about taskmanager metrics being registered twice, whereas here we're dealing with job metrics.

Do you have multiple sources? If so, do they have unique names?

On 20.03.2018 15:06, Fabian Hueske wrote:
Hi Pedro,

Can you reopen FLINK-7100 and post a comment with your error message and environment?

Thanks,
Fabian

2018-03-20 14:58 GMT+01:00 PedroMrChaves <[hidden email]>:
Hello,

I still have the same issue with Flink Version 1.4.2.

java.lang.IllegalArgumentException: A metric named
<hostname-anonyn>.taskmanager.6aa8d13575228d38ae4abdfb37fa229e.CDC.Source:
EVENTS.1.numRecordsIn already exists
        at
com.codahale.metrics.MetricRegistry.register(MetricRegistry.java:91)
        at
org.apache.flink.dropwizard.ScheduledDropwizardReporter.notifyOfAddedMetric(ScheduledDropwizardReporter.java:131)
        at
org.apache.flink.runtime.metrics.MetricRegistryImpl.register(MetricRegistryImpl.java:319)
        at
org.apache.flink.runtime.metrics.groups.AbstractMetricGroup.addMetric(AbstractMetricGroup.java:370)
        at
org.apache.flink.runtime.metrics.groups.AbstractMetricGroup.counter(AbstractMetricGroup.java:303)
        at
org.apache.flink.runtime.metrics.groups.AbstractMetricGroup.counter(AbstractMetricGroup.java:293)
        at
org.apache.flink.runtime.metrics.groups.OperatorIOMetricGroup.<init>(OperatorIOMetricGroup.java:40)
        at
org.apache.flink.runtime.metrics.groups.OperatorMetricGroup.<init>(OperatorMetricGroup.java:48)
        at
org.apache.flink.runtime.metrics.groups.TaskMetricGroup.addOperator(TaskMetricGroup.java:146)
        at
org.apache.flink.streaming.api.operators.AbstractStreamOperator.setup(AbstractStreamOperator.java:182)
        at
org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.setup(AbstractUdfStreamOperator.java:82)
        at
org.apache.flink.streaming.runtime.tasks.OperatorChain.<init>(OperatorChain.java:136)
        at
org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:231)
        at org.apache.flink.runtime.taskmanager.Task.run(Task.java:718)
        at java.lang.Thread.run(Thread.java:748)



-----
Best Regards,
Pedro Chaves
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
PedroMrChaves
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Metric Registry Warnings

PedroMrChaves
I have multiple sources but with distinct names and UIDs.

More information about my execution environment:

Flink Version: 1.4.2 bundled with hadoop 2.8
State backend: Hadoop 2.8
Job compiled for version 1.4.2 using the Scala version libs from Scala
version 2.11.

Am using the com.github.davidb to export the metrics to influxdb as
exemplified here:
https://github.com/jgrier/flink-stuff/tree/master/flink-influx-reporter but
recompiled for version 1.4.2.



-----
Best Regards,
Pedro Chaves
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
Best Regards,
Pedro Chaves
Free forum by Nabble Edit this page