Re: Not enough free slots to run the job

Posted by Fabian Hueske-2 on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Not-enough-free-slots-to-run-the-job-tp5630p16445.html

Hi David,

that's correct. A TM is a single process. A slot is just a virtual concept in the TM process and runs its program slice in multiple threads.
Besides managed memory (which is split into chunks add assigned to slots) all other resources (CPU, heap, network, disk) are not isolated and free to use for all threads.

The DataSet API operators operate almost exclusively on managed memory. Heap memory is only used for in-flight data and not to store larger amounts.

So having unused slots leaves some of the configured memory unused.

Best, Fabian

2017-10-27 3:44 GMT+02:00 David Dreyfus <[hidden email]>:

Hello,

I know this is an older thread, but ...

If some slots are left empty it doesn't necessarily mean that machine
resources are wasted. Some managed memory might be unavailable, but CPU,
heap memory, network, and disk are shared across slots. To the extent there
are multiple operators executing within a slot, multiple threads are
executing consuming those resources. It's not clear what the actual
performance degradation would be, if any. Correct?

David



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/