Apache Flink - Gauge implementation

[M Singh]

Hi:

I am working on a Flink application and need to collect the number of events of certain type/second (per unit of time) as a metric.  I was hoping to use Flink's metric framework with a gauge but did not find any implementation that will reset the count after it is reported.

The example: https://ci.apache.org/projects/flink/flink-docs-release-1.8/monitoring/metrics.html#gauge shows how to use a gauge but does not reset the gauge after the metrics framework has reported the value every at the specified interval.   So, unless it is reset by the application (after the call to getValue), it will continue to increment.  

I was wondering if there is any resettable gauge implementation available that integrates with the Flink metrics framework.

Please let me know if there are any advice/pointers.

Thanks

Mans

[Xintong Song]

Hi Singh,

Could your problem be solved by simply record the previous value and subtract it from the new value?

On Wed, Jul 10, 2019 at 3:33 PM M Singh <[hidden email]> wrote:

Hi:

I am working on a Flink application and need to collect the number of events of certain type/second (per unit of time) as a metric.  I was hoping to use Flink's metric framework with a gauge but did not find any implementation that will reset the count after it is reported.

The example: https://ci.apache.org/projects/flink/flink-docs-release-1.8/monitoring/metrics.html#gauge shows how to use a gauge but does not reset the gauge after the metrics framework has reported the value every at the specified interval.   So, unless it is reset by the application (after the call to getValue), it will continue to increment.  

I was wondering if there is any resettable gauge implementation available that integrates with the Flink metrics framework.

Please let me know if there are any advice/pointers.

Thanks

Mans

[Chesnay Schepler]

This can't be implemented as there's no guarantee that getValue is only called once.

Why do you want to reset the count? If you're interested in rates, why aren't you using a meter?

On 10/07/2019 09:37, Xintong Song wrote:

Hi Singh,

Could your problem be solved by simply record the previous value and subtract it from the new value?

On Wed, Jul 10, 2019 at 3:33 PM M Singh <[hidden email]> wrote:

Hi:

I am working on a Flink application and need to collect the number of events of certain type/second (per unit of time) as a metric.  I was hoping to use Flink's metric framework with a gauge but did not find any implementation that will reset the count after it is reported.

The example: https://ci.apache.org/projects/flink/flink-docs-release-1.8/monitoring/metrics.html#gauge shows how to use a gauge but does not reset the gauge after the metrics framework has reported the value every at the specified interval.   So, unless it is reset by the application (after the call to getValue), it will continue to increment.  

I was wondering if there is any resettable gauge implementation available that integrates with the Flink metrics framework.

Please let me know if there are any advice/pointers.

Thanks

Mans

[M Singh]

Xintong/Chesnay - Thanks for your response.

From what I understand meter measures average throughput (https://ci.apache.org/projects/flink/flink-docs-release-1.8/monitoring/metrics.html#meter).  I would like to have the absolute count in each unit of interval.

Also, I am assuming that the metric call to getValue() is not synchronized with the processor/functions respective (process/map/etc) call.

Please let me know if I missed anything.

Thanks again.

Mans

On Wednesday, July 10, 2019, 04:21:15 AM EDT, Chesnay Schepler <[hidden email]> wrote:

This can't be implemented as there's no guarantee that getValue is only called once.

Why do you want to reset the count? If you're interested in rates, why aren't you using a meter?

On 10/07/2019 09:37, Xintong Song wrote:

Hi Singh,

Could your problem be solved by simply record the previous value and subtract it from the new value?

On Wed, Jul 10, 2019 at 3:33 PM M Singh <[hidden email]> wrote:

Hi:

I am working on a Flink application and need to collect the number of events of certain type/second (per unit of time) as a metric.  I was hoping to use Flink's metric framework with a gauge but did not find any implementation that will reset the count after it is reported.

The example: https://ci.apache.org/projects/flink/flink-docs-release-1.8/monitoring/metrics.html#gauge shows how to use a gauge but does not reset the gauge after the metrics framework has reported the value every at the specified interval.   So, unless it is reset by the application (after the call to getValue), it will continue to increment.  

I was wondering if there is any resettable gauge implementation available that integrates with the Flink metrics framework.

Please let me know if there are any advice/pointers.

Thanks

Mans