Any suggestions about which GC collector to use in Flink?

[徐涛]

Hi Experts,
        In my environment, when I submit the Flink program to yarn, I do not specify which GC collector to use, in the web monitor page, I found it uses PS_Scavenge as the young generation GC collector, PS_MarkSweep as the old generation GC collector, I wonder if I can use G1 as the GC collector, does anyone has evaluated the performance of which GC collector to use? I believe that use G1 can reduce the old generation GC time, especially for the large heap.

Best
Henry

[qi luo]

+1. It would be great if someone could benchmark between difference GC in Flink (we may do it in next few months).

I’m told that the default parallel GC provides better throughput but longer pauses (we encountered 2min+ GC pauses in large dataset). Whereas the G1GC provides less pauses but also less throughput.

> On Apr 2, 2019, at 3:26 PM, 徐涛 <[hidden email]> wrote:
>
> Hi Experts,
> In my environment, when I submit the Flink program to yarn, I do not specify which GC collector to use, in the web monitor page, I found it uses PS_Scavenge as the young generation GC collector, PS_MarkSweep as the old generation GC collector, I wonder if I can use G1 as the GC collector, does anyone has evaluated the performance of which GC collector to use? I believe that use G1 can reduce the old generation GC time, especially for the large heap.
>
> Best
> Henry