Counter metrics for prometheus having unexepcted gaps in grafana

[Manish G]

Hi All,

I am facing an issue with counter metrics I have added to a flatmap function.

My application is deployed in kubernetes, and hence the prometheus metrics generated has pod id as one of its label. Now if pod dies and a new pod comes up, we have a brand new metrics starting from 0.

As a result, manoy of the graphs in grafana have unexpected gaps.

Can I somehow avoid it?

[Chesnay Schepler]

Not sure whether it would solve your issue, but you could maybe exclude
the pod id (I assume you mean the host?) from being reported by setting
this:
metrics.reporter.<name>.scope.variables.excludes: <host>



On 1/20/2021 7:16 AM, Manish G wrote:

> Hi All,
>
> I am facing an issue with counter metrics I have added to a flatmap
> function.
> My application is deployed in kubernetes, and hence the prometheus
> metrics generated has pod id as one of its label. Now if pod dies and
> a new pod comes up, we have a brand new metrics starting from 0.
> As a result, manoy of the graphs in grafana have unexpected gaps.
>
> Can I somehow avoid it?