The default implementation is the SimpleCounter, however I believe that
the implementation isn't of interest compared to how it is used.
Please expand on the graph ( which metric is represented by what color )
and include all of your code that interacts with the your counter.
On 24.11.2018 16:36, Anil wrote:
> I'm using Flink 1.4.2 and deploying job on Yarn Cluster.
>
> I have a streaming job, which flattens the data and outputs it. It basically
> takes a input record and produces n output record. I'm using Table Function
> for this. The logic to flatten the data is implemented in a UDF. The UDF has
> a counter which basically counts the number of records produced.
> this.context.getMetricGroup().counter("output_records_counter")
>
> I know Flink provides numRecordsOut metric which is essentially gives me the
> same number.
> When the job is started the output records count seen for
> `output_records_counter` counter and `numRecordsOut` are exactly same.
>
> When a task manager is lost and the job is restarted there's a huge
> difference in the count of output records . As seen in the graph when the
> job was started the both the counts are overlapping. When a task manager is
> lost and is re-deployed the count is different. I'm not sure why this
> number varies so much.
>
> Can someone please shed some light on how is this counter implemented or
> direct me to source code or any reference material.
>
> For numRecordsOut, each taskmanager emits the count of data. Is the same not
> true for the output_records_counter.
>
> <
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/file/t1468/Screen_Shot_2018-11-24_at_9.png>
>
>
>
> --
> Sent from:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/>
Locked
