taskmanager.network.numberOfBuffers

[Flavio Pompermaier]

Hi to all,

I've received an error running the job saying to increase this parameter so I set it to 2048*4 and everything worked.

However, could you explain me in detail how this number is computed?

I'm running the job from my IDE (default parallelism so all my 8 cores) so I was expecting no such exception..

Thanks in advance,

Flavio

[Ufuk Celebi]

On 15 Apr 2015, at 09:37, Flavio Pompermaier <[hidden email]> wrote:

> I've received an error running the job saying to increase this parameter so I set it to 2048*4 and everything worked.
> However, could you explain me in detail how this number is computed?
> I'm running the job from my IDE (default parallelism so all my 8 cores) so I was expecting no such exception..

Check this out: http://ci.apache.org/projects/flink/flink-docs-master/config.html#configuring-the-network-buffers

This can happen if the job pipeline is somewhat long. It's n on the roadmap to get rid of this configuration in the near future.

– Ufuk