Re: Flink sent/received bytes related metrics: how are they calculated?

Posted by Chesnay Schepler on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Flink-sent-received-bytes-related-metrics-how-are-they-calculated-tp28244p28251.html

How does the P1 pipeline look like? Are there 2 downstream operators reading from Sin (in this case the number of bytes would be measured twice)?

On 14/06/2019 12:09, Andrea Spina wrote:

Sorry, I totally missed the version: flink-1.6.4, Streaming API

Il giorno ven 14 giu 2019 alle ore 11:08 Chesnay Schepler <[hidden email]> ha scritto:

Which version of Flink are you using? There were some issues at some point about double-counting.

On 14/06/2019 09:49, Andrea Spina wrote:

Dear Community, I'd like to ask for some details about bytes related metrics in Flink. Precisely, I'm looking at bytes sent and bytes received metrics: what I am recording is the following:

I am reading from a Kafka topic A records with schema K using a source Sin belonging to a pipeline P1. The topic A is filled with K data using a separated pipeline p2 and a sink Sout.

Basically what I register is that the bytes sent metric of Sin - available through the Flink UI -measure twice or more respect of the bytes received of Sout. For instance, if my sink Sout records 10GB of bytes received inbound, then my source Sin emits between 20-25GB bytes sent.

Is someone able to detail how these two metrics are calculated? 

Thank you,

--

--