Garbage collection concerns with Task Manager memory

[Marchant, Hayden]

I read in the Flink documentation that the TaskManager runs all tasks within its own JVM, and that the recommendation is to set the taskmanager.heap.mb to be as much as is available on the server. I have a very large server with 192GB so thinking of giving most of it to the Task Manager.

I recall that there are concerns with long stop-the-world garbage collection pauses about allocating too much memory to a JVM - is this still a concern with G1 ?


Thanks,
Hayden Marchant

[Kien Truong]

Hi,

Yes, GC is still a major concern. Even G1 has a hard time dealing with >64GB heap in our experience.

To mitigate, we run multiple TMs with smaller heap per machine, and use RocksDBStateBackend.

Best regards,

Kien

On 10/18/2017 4:40 PM, Marchant, Hayden wrote:

I read in the Flink documentation that the TaskManager runs all tasks within its own JVM, and that the recommendation is to set the taskmanager.heap.mb to be as much as is available on the server. I have a very large server with 192GB so thinking of giving most of it to the Task Manager.

I recall that there are concerns with long stop-the-world garbage collection pauses about allocating too much memory to a JVM - is this still a concern with G1 ?


Thanks,
Hayden Marchant