In our usage of Flink, our Yarn Job Manager never goes above ~48 MB of heap utilization. In order to maximize the heap available to the Task Managers I thought we could shrink our Job Manager heap setting
down from the 1024MB we were using to something tiny like 128MB. However, doing so results in the runtime error:
java.lang.IllegalArgumentException: The JobManager memory (64) is below the minimum required memory amount of 768 MB
at org.apache.flink.yarn.AbstractYarnClusterDescriptor.setJobManagerMemory(AbstractYarnClusterDescriptor.java:187)
…
Looking into it: this value isn’t controlled by the settings in yarn-site.xml but is actually hardcoded in Flink code base to 768 MB. (see AbstractYarnDescriptor.java where MIN_JM_MEMORY = 768.)
Why is this hardcoded?
Why not let value be set via the Yarn Site Configuration xml?
Why such a high minimum?
Thanks,
Dan
| Free forum by Nabble | Edit this page |