jvm options for individual jobs / topologies

[Benjamin Cuthbert]

Hi all,

How do I give different topologies / jobs different Xmx memory parameters?

Regards

[Kostas Kloudas]

Hi Benjamin,

I do not think you can set per job memory configuration in a shared cluster.

The reason is that if different jobs share the same TM, there are resources that are shared between them, e.g, network buffers.

If you are ok with having a separate cluster per job then this will allow you to set per job memory quotas, because this would make

the total TM memory quota specification correspond to the per job ones.

In [1] you can find the available per job options and on [2] the cluster-wide ones.

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/execution_configuration.html

[2] https://ci.apache.org/projects/flink/flink-docs-release-1.4/ops/config.html

I hope this helps,

Kostas

On May 8, 2018, at 11:46 AM, Benjamin Cuthbert <[hidden email]> wrote:

Hi all,

How do I give different topologies / jobs different Xmx memory parameters?

Regards