Flink program，Full GC (System.gc()）

[陈Darling]

Flink Version: 1.8.1

deploy：standalone

state.backend.fs.memory-threshold=128k

A very very simple flink program and without other jar dependended；

But trigger full gc every hour by Full GC (System.gc() in jobmanager

Jobmanager 

I only find this where called System.gc()，but not sure  when will be call? 

Has anyone encountered a similar situation?

/**
 * Gets an estimate of the size of the free heap memory.
 * 
 * NOTE: This method is heavy-weight. It triggers a garbage collection to reduce fragmentation and get
 * a better estimate at the size of free memory. It is typically more accurate than the plain version
 * {@link #getSizeOfFreeHeapMemory()}.
 * 
 * @return An estimate of the size of the free heap memory, in bytes.
 */
public static long getSizeOfFreeHeapMemoryWithDefrag() {
   // trigger a garbage collection, to reduce fragmentation
   System.gc();
   
   return getSizeOfFreeHeapMemory();
}

   ..

  317  14972.571: [GC (Allocation Failure) 14972.572: [ParNew: 1870848K->236623K(1870848K), 0.0318714 secs] 2910945K->1276963K(3298304K), 0.0321537 secs] [Times: user=0.10 sys=0.00, real=0.03 secs] 

  318  15024.592: [GC (Allocation Failure) 15024.592: [ParNew: 1840207K->267264K(1870848K), 0.0420305 secs] 2880547K->1309499K(3298304K), 0.0422980 secs] [Times: user=0.12 sys=0.00, real=0.04 secs] 

  319: 15049.334: [Full GC (System.gc()) 15049.334: [CMS: 1042235K->322292K(1427456K), 0.2932806 secs] 1552421K->322292K(3298304K), [Metaspace: 63049K->63049K(1105920K)], 0.2938379 secs] [Times: user=0.30 sys=0.00, real=0.29 secs] 

  320  15068.400: [GC (Allocation Failure) 15068.400: [ParNew: 1603548K->219691K(1870848K), 0.0244228 secs] 1925840K->541984K(3298304K), 0.0246696 secs] [Times: user=0.08 sys=0.00, real=0.03 secs] 

  321  15157.833: [GC (Allocation Failure) 15157.833: [ParNew: 1823275K->169121K(1870848K), 0.0271608 secs] 2145568K->491414K(3298304K), 0.0274543 secs] [Times: user=0.08 sys=0.00, real=0.02 secs] 

  ...

  676  30481.818: [GC (Allocation Failure) 30481.819: [ParNew: 1813938K->267264K(1870848K), 0.0423833 secs] 2864487K->1320335K(3298304K), 0.0425678 secs] [Times: user=0.12 sys=0.00, real=0.05 secs] 

  677  30557.470: [GC (Allocation Failure) 30557.470: [ParNew: 1870781K->267264K(1870848K), 0.0356143 secs] 2923853K->1330480K(3298304K), 0.0358636 secs] [Times: user=0.11 sys=0.00, real=0.04 secs] 

  678: 30615.513: [Full GC (System.gc()) 30615.513: [CMS: 1063216K->325191K(1427456K), 0.4150118 secs] 2772924K->325191K(3298304K), [Metaspace: 63756K->63756K(1107968K)], 0.4156102 secs] [Times: user=0.42 sys=0.00, real=0.41 secs] 

  679  30649.417: [GC (Allocation Failure) 30649.417: [ParNew: 1603486K->154121K(1870848K), 0.0236163 secs] 1928677K->479312K(3298304K), 0.0238171 secs] [Times: user=0.07 sys=0.00, real=0.03 secs] 

  680  30734.471: [GC (Allocation Failure) 30734.471: [ParNew: 1757698K->164639K(1870848K), 0.0318273 secs] 2082889K->489830K(3298304K), 0.0320078 secs] [Times: user=0.09 sys=0.00, real=0.03 secs] 

  ...

  838  37620.283: [GC (Allocation Failure) 37620.283: [ParNew: 1819345K->267264K(1870848K), 0.0350667 secs] 2675979K->1123898K(3298304K), 0.0352966 secs] [Times: user=0.10 sys=0.00, real=0.03 secs] 

  839  37696.182: [GC (Allocation Failure) 37696.182: [ParNew: 1870725K->267264K(1870848K), 0.0318628 secs] 2727360K->1124191K(3298304K), 0.0320820 secs] [Times: user=0.10 sys=0.00, real=0.03 secs] 

  840: 37736.562: [Full GC (System.gc()) 37736.562: [CMS: 856927K->326386K(1427456K), 0.3860904 secs] 2420047K->326386K(3298304K), [Metaspace: 63914K->63914K(1107968K)], 0.3867480 secs] [Times: user=0.39 sys=0.00, real=0.38 secs] 

  841  37786.710: [GC (Allocation Failure) 37786.710: [ParNew: 1603475K->160119K(1870848K), 0.0211419 secs] 1929861K->486506K(3298304K), 0.0214152 secs] [Times: user=0.06 sys=0.00, real=0.03 secs] 

  842  37868.276: [GC (Allocation Failure) 37868.276: [ParNew: 1763703K->156520K(1870848K), 0.0274233 secs] 2090090K->482907K(3298304K), 0.0277311 secs] [Times: user=0.08 sys=0.00, real=0.03 secs] 

  ...

  957  43291.423: [GC (Allocation Failure) 43291.424: [ParNew: 1870848K->267264K(1870848K), 0.0422492 secs] 2779218K->1208332K(3298304K), 0.0425675 secs] [Times: user=0.13 sys=0.00, real=0.04 secs] 

  958  43301.420: [GC (Allocation Failure) 43301.420: [ParNew: 1870766K->267264K(1870848K), 0.0514803 secs] 2811834K->1324858K(3298304K), 0.0517214 secs] [Times: user=0.16 sys=0.00, real=0.06 secs] 

  959: 43348.937: [Full GC (System.gc()) 43348.937: [CMS: 1057594K->326740K(1427456K), 0.3976217 secs] 1966751K->326740K(3298304K), [Metaspace: 63978K->63978K(1107968K)], 0.3982278 secs] [Times: user=0.40 sys=0.00, real=0.40 secs] 

  960  43388.807: [GC (Allocation Failure) 43388.807: [ParNew: 1603565K->215196K(1870848K), 0.0229975 secs] 1930306K->541937K(3298304K), 0.0232834 secs] [Times: user=0.07 sys=0.00, real=0.02 secs] 

  961  43473.412: [GC (Allocation Failure) 43473.413: [ParNew: 1818755K->159795K(1870848K), 0.0279802 secs] 2145495K->486536K(3298304K), 0.0282288 secs] [Times: user=0.09 sys=0.00, real=0.02 secs]