| ager.memory.process.size | (none) | MemorySize | Total Process Memory size for the TaskExecutors. This includes all the memory that a TaskExecutor consumes, consisting of Total Flink Memory, JVM Metaspace, and JVM Overhead. On containerized setups, this should be set to the container memory. See also 'taskmanager.memory.flink.size' for total Flink memory size configuration. |
|---|
The Total Flink Memory consists of off heap memory governed by fraction. In pure streaming case ( and non ROCKSDB state case )
This is the size of off-heap memory managed by the memory manager, reserved for sorting, hash tables, caching of intermediate results and RocksDB state backend. Memory consumers can either allocate memory from the memory manager in the form of MemorySegments, or reserve bytes from the memory manager and keep their memory usage within that boundary.
Is not used AFAIK . May be reduce the fraction to 0 ? We do not use offline heap ( aka batch jobs ) on our cluster ?
Any help will be appreciated.
Hello folks,As established https://ci.apache.org/projects/flink/flink-docs-stable/ops/config.html#memory-configuration , I set the taskmanager.memory.process.size and taskmanager.memory.task.off-heap.size in my flink-conf.yaml and I see the 2 properties being pulled in.- Loading configuration property: taskmanager.memory.process.size, 8000m
- Loading configuration property: taskmanager.memory.task.off-heap.size, 1024m
I am not sure how the -Xmx and -Xms are calculated but I seeStarting taskexecutor as a console application on host kafka-to-hdfs-taskmanager-dev-8597c78d9c-59dqw.
VM settings:
Min. Heap Size: 2.27G
Max. Heap Size: 2.27G
Using VM: OpenJDK 64-Bit Server VM
What gives ?I am looking through the scripts and am not sure I see any calculations based on taskmanager.memory.process.size
| Free forum by Nabble | Edit this page |