strange GC behaviour

[Zhenhao Li]

Hi there,

 

I am running a Beam job with FlinkRunner as a standalone job on YARN with 4 nodes. The job always dies after one hour or two.

Now looking deeper into the task manager metrics, I find that there is a one task manager showing strange CG behavior as shown in the attached screenshots.

 

Can anyone help interpret the numbers under GC?

 

Best,

Zhenhao

 

[Zhenhao Li]

The job died and the log is attached.

 

From: Zhenhao Li <[hidden email]>
Date: Tuesday, 1 May 2018 at 17:39
To: "[hidden email]" <[hidden email]>
Subject: strange GC behaviour

 

Hi there,

 

I am running a Beam job with FlinkRunner as a standalone job on YARN with 4 nodes. The job always dies after one hour or two.

Now looking deeper into the task manager metrics, I find that there is a one task manager showing strange CG behavior as shown in the attached screenshots.

 

Can anyone help interpret the numbers under GC?

 

Best,

Zhenhao

 

[Ted Yu]

Please see http://www.fasterj.com/articles/oraclecollectors1.shtml

On Tue, May 1, 2018 at 8:38 AM, Zhenhao Li <[hidden email]> wrote:

Hi there,

 

I am running a Beam job with FlinkRunner as a standalone job on YARN with 4 nodes. The job always dies after one hour or two.

Now looking deeper into the task manager metrics, I find that there is a one task manager showing strange CG behavior as shown in the attached screenshots.

 

Can anyone help interpret the numbers under GC?

 

Best,

Zhenhao

 

[Zhenhao Li]

Thank you, Ted.

I should have been more specific. What is the semantic of “Time” under “Garbage Collection” on the Flink UI. I couldn’t find documentation about it.

 

Cheers,

Z.

 

From: Ted Yu <[hidden email]>
Date: Tuesday, 1 May 2018 at 18:40
To: Zhenhao Li <[hidden email]>
Cc: "[hidden email]" <[hidden email]>
Subject: Re: strange GC behaviour

 

Please see http://www.fasterj.com/articles/oraclecollectors1.shtml

 

On Tue, May 1, 2018 at 8:38 AM, Zhenhao Li <[hidden email]> wrote:

Hi there,

 

I am running a Beam job with FlinkRunner as a standalone job on YARN with 4 nodes. The job always dies after one hour or two.

Now looking deeper into the task manager metrics, I find that there is a one task manager showing strange CG behavior as shown in the attached screenshots.

 

Can anyone help interpret the numbers under GC?

 

Best,

Zhenhao