Re: FLINK 1.11 Graphite Metrics

Posted by Chesnay Schepler on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/FLINK-1-11-Graphite-Metrics-tp38927p39000.html

How exactly did you do the upgrade? Did you copy some files from 1.11 into an existing 1.10 distribution?

The configuration is correct, but it appears as if the entire plugins directory is either a) empty or b) not shipped.

On 10/27/2020 5:22 PM, Vijayendra Yadav wrote:

Hi Robert and Chesnay,

Only  thing changed is I upgraded from Flink 1.10 to 1.11 and to support that updated conf yaml with factory class.

Here is attached Full Log with classpath etc.   (log.txt)

Regards,

Vijay

 

On Tue, Oct 27, 2020 at 9:31 AM Chesnay Schepler <[hidden email]> wrote:

Are you writing a test? (otherwise the ReporterSetupTest reporters wouldn't be around)

Do you have a dependency on the graphite reporter?

On 10/27/2020 8:27 AM, Robert Metzger wrote:

Hi Vijayendra,

can you post or upload the entire logs, so that we can see the Classpath logged on startup, as well as the effective configuration parameters?

On Tue, Oct 27, 2020 at 12:49 AM Vijayendra Yadav <[hidden email]> wrote:

Hi Chesnay,

Another log message: 

2020-10-26 23:33:08,516 WARN org.apache.flink.runtime.metrics.ReporterSetup - The reporter factory (org.apache.flink.metrics.graphite.GraphiteReporterFactory) could not be found for reporter grph. Available factories: [org.apache.flink.runtime.metrics.ReporterSetupTest$ConfigExposingReporterFactory, org.apache.flink.runtime.metrics.ReporterSetupTest$TestReporterFactory, org.apache.flink.runtime.metrics.ReporterSetupTest$InstantiationTypeTrackingTestReporterFactory, org.apache.flink.runtime.metrics.ReporterSetupTest$FailingFactory].

2020-10-26 23:33:08,517 INFO org.apache.flink.runtime.metrics.MetricRegistryImpl - No metrics reporter configured, no metrics will be exposed/reported.

Regards,

Vijay

On Mon, Oct 26, 2020 at 2:34 PM Vijayendra Yadav <[hidden email]> wrote:

Hi Chesnay,

I have the same, and I am exporting the flinkconf like below, where i have flink-conf.yaml with configuration you have given.What else can I try ?

export FLINK_CONF_DIR=${app_install_path}/flinkconf/

regards,

Vijay

On Sun, Oct 25, 2020 at 8:03 AM Chesnay Schepler <[hidden email]> wrote:

Ah wait, in 1.11 it should not longer be necessary to explicitly copy the reporter jar.

Please update your reporter configuration to this:

metrics.reporter.grph.factory.class: org.apache.flink.metrics.graphite.GraphiteReporterFactory

On 10/25/2020 4:00 PM, Chesnay Schepler wrote:

Have you followed the documentation, specifically this bit?

> In order to use this reporter you must copy /opt/flink-metrics-influxdb-1.11.2.jar into the plugins/influxdb folder of your Flink distribution.

On 10/24/2020 12:17 AM, Vijayendra Yadav wrote:

Hi Team,

for Flink 1.11 Graphite Metrics. I see the following Error in the log. 

Any suggestions?

020-10-23 21:55:14,652 ERROR org.apache.flink.runtime.metrics.ReporterSetup                - Could not instantiate metrics reporter grph. Metrics might not be exposed/reported.
java.lang.ClassNotFoundException: org.apache.flink.metrics.graphite.GraphiteReporter
	at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:264)
	at org.apache.flink.runtime.metrics.ReporterSetup.loadViaReflection(ReporterSetup.java:313)
	at org.apache.flink.runtime.metrics.ReporterSetup.loadReporter(ReporterSetup.java:274)
	at org.apache.flink.runtime.metrics.ReporterSetup.setupReporters(ReporterSetup.java:235)
	at org.apache.flink.runtime.metrics.ReporterSetup.fromConfiguration(ReporterSetup.java:148)
	at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.createMetricRegistry(ClusterEntrypoint.java:316)
	at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.initializeServices(ClusterEntrypoint.java:270)
	at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runCluster(ClusterEntrypoint.java:208)
	at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$startCluster$0(ClusterEntrypoint.java:169)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893)
	at org.apache.flink.runtime.security.contexts.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
	at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:168)
	at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runClusterEntrypoint(ClusterEntrypoint.java:517)
	at org.apache.flink.yarn.entrypoint.YarnJobClusterEntrypoint.main(YarnJobClusterEntrypoint.java:89)

Regards,

Vijay