how to start tuning to prevent OutOfMemory

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

how to start tuning to prevent OutOfMemory

Istvan Soos
Hi,

We can see an occasional OOM issue with our Flink jobs. Maybe the
input got more diverse, and the grouping has much more keys, not
really sure about that part.

How do you usually tackle these issues? We are running with
parallelism between 5-30. Would it help if we turn that down?

We do set taskmanager.heap.mb explicitly, but not the advanced config options.

Thanks,
  Istvan
Reply | Threaded
Open this post in threaded view
|

Re: how to start tuning to prevent OutOfMemory

Timo Walther
Hi Istvan,

are you running batch or streaming Flink jobs? It is often the
user-defined code that creates a lot of objects during runtime. Try to
get rid of object creation as much as possible, maybe also a library you
are using causes these issues. You might also reduce the size of network
buffers to free some memory.

Timo


Am 27/07/16 um 23:43 schrieb Istvan Soos:

> Hi,
>
> We can see an occasional OOM issue with our Flink jobs. Maybe the
> input got more diverse, and the grouping has much more keys, not
> really sure about that part.
>
> How do you usually tackle these issues? We are running with
> parallelism between 5-30. Would it help if we turn that down?
>
> We do set taskmanager.heap.mb explicitly, but not the advanced config options.
>
> Thanks,
>    Istvan


--
Freundliche Grüße / Kind Regards

Timo Walther

Follow me: @twalthr
https://www.linkedin.com/in/twalthr