Applying Custom metrics

[Piper Piper]

Hi

I have questions regarding making my own custom metrics.

When exactly is the class RichMapFunction’s map(value) method called/invoked, and what “value” will be passed/expected as an argument to this map(value) method?

Does the RichMapFunction’s map() method have any relation to the transformation map() method, or are they completely different?

Once defined, how do I put the custom metric onto a specific source, specific operator, and onto a specific sink in my job’s DAG?

Thank you,

Piper

[Till Rohrmann]

Hi Piper,

the RichMapFunction's map function is called for every record you are processing in your DataStream/DataSet. The RichMapFunction is the definition of the map function you are applying to every record. Hence, it is basically what you pass to the DataStream.map(MapFunction myMapFunction) in order to describe the transformation you want to do.

If you want to define your own metrics, then you need to define a RichMapFunction and use its getRuntimeContext() method to call

counter = getRuntimeContext()
  .getMetricGroup()
  .addGroup("MyMetrics")
  .counter("myCounter");

in order to register a custom counter. For more information please take a look at [1].

[1] https://ci.apache.org/projects/flink/flink-docs-stable/monitoring/metrics.html#user-scope

Cheers,

Till

On Tue, Oct 6, 2020 at 1:05 AM Piper Piper <[hidden email]> wrote:

Hi

I have questions regarding making my own custom metrics.

When exactly is the class RichMapFunction’s map(value) method called/invoked, and what “value” will be passed/expected as an argument to this map(value) method?

Does the RichMapFunction’s map() method have any relation to the transformation map() method, or are they completely different?

Once defined, how do I put the custom metric onto a specific source, specific operator, and onto a specific sink in my job’s DAG?

Thank you,

Piper