Re: Flink UI and records sent

Posted by Flavio Pompermaier on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Flink-UI-and-records-sent-tp12522p12530.html

My job is a batch one.

Here's an image of two different execution of the job.

The third line is where the first(1M) is called. In the left side the count is what I expect, in the second is slightly less :(

Any idea?

On Wed, Apr 5, 2017 at 12:51 PM, Chesnay Schepler <[hidden email]> wrote:

Hey Flavio,

it's unlikely that the counters skip a record.

For the webUI these metrics are transported in 2 different ways:
For running tasks they are fetched through the metric system; this provides no guarantee that the final count is ever displayed.
For finished tasks the final count is stored in the ExecutionGraph and should show an accurate final count.

So, the question is in which state your task is.

Regards,
Chesnay

On 05.04.2017 09:55, Flavio Pompermaier wrote:

Hi to all,

I'm using Flink 1.2.0 and I have a job that (at some point) calls dataset.first(1M).

Sometimes the records sent displayed by the UI are less than 1M (lik 999709).

Is it possible that the UI (or the internal Flink counters) miss some record?

Best,

Flavio

--

--