Monitoring memory usage of a Flink Job

[Tamara Mendt]

Hi,

I am running some experiments on Flink and was wondering if there is some way to monitor the memory usage of a Flink Job (running locally and on a cluster). I need to run multiple jobs and compare their memory usage.

Cheers,

Tamara

[Fabian Hueske-2]

Hi Tamara,

what kind of information do you need? Something like, size and usage of in-memory sort buffers or hash tables?

Some information might written in DEBUG logs, but I'm not sure about that. Besides logs, I doubt that Flink monitors memory usage.

Cheers, Fabian

2015-06-15 14:34 GMT+02:00 Tamara Mendt <[hidden email]>:

Hi,

I am running some experiments on Flink and was wondering if there is some way to monitor the memory usage of a Flink Job (running locally and on a cluster). I need to run multiple jobs and compare their memory usage.

Cheers,

Tamara

[Till Rohrmann]

Hi Tamara,

you can instruct Flink to write the current memory statistics to the log by setting taskmanager.debug.memory.startLogThread: true in the Flink configuration. Furthermore, you can control the logging interval with taskmanager.debug.memory.logIntervalMs where the interval is specified in milli seconds.

Cheers,
Till

On Mon, Jun 15, 2015 at 2:46 PM Fabian Hueske fhueske@... wrote:

Hi Tamara,

what kind of information do you need? Something like, size and usage of in-memory sort buffers or hash tables?

Some information might written in DEBUG logs, but I'm not sure about that. Besides logs, I doubt that Flink monitors memory usage.

Cheers, Fabian

2015-06-15 14:34 GMT+02:00 Tamara Mendt <[hidden email]>:

Hi,

I am running some experiments on Flink and was wondering if there is some way to monitor the memory usage of a Flink Job (running locally and on a cluster). I need to run multiple jobs and compare their memory usage.

Cheers,

Tamara

​

[Tamara Mendt]

Ok great, I will try this out and get back to you. Thanks =)

On Mon, Jun 15, 2015 at 2:52 PM, Till Rohrmann <[hidden email]> wrote:

Hi Tamara,

you can instruct Flink to write the current memory statistics to the log by setting taskmanager.debug.memory.startLogThread: true in the Flink configuration. Furthermore, you can control the logging interval with taskmanager.debug.memory.logIntervalMs where the interval is specified in milli seconds.

Cheers,
Till

On Mon, Jun 15, 2015 at 2:46 PM Fabian Hueske fhueske@... wrote:

Hi Tamara,

what kind of information do you need? Something like, size and usage of in-memory sort buffers or hash tables?

Some information might written in DEBUG logs, but I'm not sure about that. Besides logs, I doubt that Flink monitors memory usage.

Cheers, Fabian

2015-06-15 14:34 GMT+02:00 Tamara Mendt <[hidden email]>:

Hi,

I am running some experiments on Flink and was wondering if there is some way to monitor the memory usage of a Flink Job (running locally and on a cluster). I need to run multiple jobs and compare their memory usage.

Cheers,

Tamara

​

--

[rmetzger0]

You don't have to enable the logging thread.

You can also get the metrics of the job manager via the job manager web frontend. There, they also available in a JSON representation.

So if you want, you can periodically (say every 5 seconds) do a HTTP request to get the metrics of all TMs.

On Mon, Jun 15, 2015 at 6:03 AM, Tamara Mendt <[hidden email]> wrote:

Ok great, I will try this out and get back to you. Thanks =)

On Mon, Jun 15, 2015 at 2:52 PM, Till Rohrmann <[hidden email]> wrote:

Hi Tamara,

you can instruct Flink to write the current memory statistics to the log by setting taskmanager.debug.memory.startLogThread: true in the Flink configuration. Furthermore, you can control the logging interval with taskmanager.debug.memory.logIntervalMs where the interval is specified in milli seconds.

Cheers,
Till

On Mon, Jun 15, 2015 at 2:46 PM Fabian Hueske fhueske@... wrote:

Hi Tamara,

what kind of information do you need? Something like, size and usage of in-memory sort buffers or hash tables?

Some information might written in DEBUG logs, but I'm not sure about that. Besides logs, I doubt that Flink monitors memory usage.

Cheers, Fabian

2015-06-15 14:34 GMT+02:00 Tamara Mendt <[hidden email]>:

Hi,

I am running some experiments on Flink and was wondering if there is some way to monitor the memory usage of a Flink Job (running locally and on a cluster). I need to run multiple jobs and compare their memory usage.

Cheers,

Tamara

​

--

Tamara Mendt

[Hung]

Hi Robert,

Could I ask which endpoint you use to get the memory statistics of a Flink job? I checked here but don't know which one to use.
https://ci.apache.org/projects/flink/flink-docs-release-1.3/monitoring/rest_api.html

Or should we put the memory in the metrics?

Best,

Sendoh