Counter Metrics not getting removed from Flink GUI after close()

[Gaurav Luthra]

Hi,

I am using ProcessWindowFunction, and in process() function I am adding user scoped Group as mentioned below.

MetricGroup myMetricGroup = getRuntimeContext().getMetricGroup().addGroup("myGroup")

Now, I am creating counter metrics using my myMetricGroup, and I am able to see these counters in Flink GUI.

But when I call close() like mentioned below.

((AbstractMetricGroup) myMetricGroup).close(); 

   

Even then my counter metrics are not getting removed from flink GUI.

Kindly Guide how to close user scoped metric group (myMetricGroup in my case) so that all the counter metrics created using myMetricGroup shall be removed from Flink GUI.

[Chesnay Schepler]

Metrics for a given job will be available in the GUI until the Job has finished.

On 08.01.2019 17:08, Gaurav Luthra wrote:

Hi,

I am using ProcessWindowFunction, and in process() function I am adding user scoped Group as mentioned below.

MetricGroup myMetricGroup = getRuntimeContext().getMetricGroup().addGroup("myGroup")

Now, I am creating counter metrics using my myMetricGroup, and I am able to see these counters in Flink GUI.

But when I call close() like mentioned below.

((AbstractMetricGroup) myMetricGroup).close(); 

   

Even then my counter metrics are not getting removed from flink GUI.

Kindly Guide how to close user scoped metric group (myMetricGroup in my case) so that all the counter metrics created using myMetricGroup shall be removed from Flink GUI.

[Gaurav Luthra]

Hi Chesnay,

If removing the metrics is not possible from Flink GUI, while the job is
running.
Then kindly tell me how to update a metric counter.

Explaination:
Suppose I created a metric Counter with key "chesnay" and incremented the
counter to 20, by code mentioned below.
getRuntimeContext().getMetricGroup().counter("chesnay").inc(20);

/Note: I am not assigning this counter to any local/member variable as I do
not want to keep state in my job./

Now, after some time, If I want to update the value of "chesnay" metric
counter to 60 and I am not aware about the old state (which is 20).

So, If I do getRuntimeContext().getMetricGroup().counter("chesnay").inc(60);

Event then Flink GUI shows value 20 for "chesnay" metric Group. and gives a
WARN log something like this, "same name can be used, and behavior is
undefined".

Now, how to update the "chesnay" metric Group if I do not want to keep the
state in my Job???

Thats why, I though of creating user scoped metric group and thought of
closing that group to remove the metric counters and create new metrics
every time, when I want to update it.

Hope you understood my problem.



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[Chesnay Schepler]

What you're trying to do is not possible. Even if you close the group it still exists, and is returned by subsequent calls to addGroup("mygroup").
However since it is closed all registration calls will be ignored, hence why the value isn't updating.

You can only update a metric by storing a reference to it in your function.
Why do you want to avoid the member variable?

On 08.01.2019 17:24, Gaurav Luthra wrote:

Hi Chesnay,

If removing the metrics is not possible from Flink GUI, while the job is
running.
Then kindly tell me how to update a metric counter.

Explaination:
Suppose I created a metric Counter with key "chesnay" and incremented the
counter to 20, by code mentioned below.
getRuntimeContext().getMetricGroup().counter("chesnay").inc(20);

/Note: I am not assigning this counter to any local/member variable as I do
not want to keep state in my job./

Now, after some time, If I want to update the value of "chesnay" metric
counter to 60 and I am not aware about the old state (which is 20).

So, If I do getRuntimeContext().getMetricGroup().counter("chesnay").inc(60);

Event then Flink GUI shows value 20 for "chesnay" metric Group. and gives a
WARN log something like this, "same name can be used, and behavior is
undefined".

Now, how to update the "chesnay" metric Group if I do not want to keep the
state in my Job???

Thats why, I though of creating user scoped metric group and thought of
closing that group to remove the metric counters and create new metrics
every time, when I want to update it.

Hope you understood my problem.



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[Gaurav Luthra]

Hi Chesnay,

I do not want to store metric counter in reference variable because I want to create metric counter for every key of keyed stream.

There can be n number of keys and I do not want to have n number of references.

On Tue, 8 Jan, 2019, 11:01 PM Chesnay Schepler <[hidden email] wrote:

What you're trying to do is not possible. Even if you close the group it still exists, and is returned by subsequent calls to addGroup("mygroup").
However since it is closed all registration calls will be ignored, hence why the value isn't updating.

You can only update a metric by storing a reference to it in your function.
Why do you want to avoid the member variable?

On 08.01.2019 17:24, Gaurav Luthra wrote:

Hi Chesnay,

If removing the metrics is not possible from Flink GUI, while the job is
running.
Then kindly tell me how to update a metric counter.

Explaination:
Suppose I created a metric Counter with key "chesnay" and incremented the
counter to 20, by code mentioned below.
getRuntimeContext().getMetricGroup().counter("chesnay").inc(20);

/Note: I am not assigning this counter to any local/member variable as I do
not want to keep state in my job./

Now, after some time, If I want to update the value of "chesnay" metric
counter to 60 and I am not aware about the old state (which is 20).

So, If I do getRuntimeContext().getMetricGroup().counter("chesnay").inc(60);

Event then Flink GUI shows value 20 for "chesnay" metric Group. and gives a
WARN log something like this, "same name can be used, and behavior is
undefined".

Now, how to update the "chesnay" metric Group if I do not want to keep the
state in my Job???

Thats why, I though of creating user scoped metric group and thought of
closing that group to remove the metric counters and create new metrics
every time, when I want to update it.

Hope you understood my problem.



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/