Insufficient number of network buffers running on cluster

[Guido]

Hello,

I would like to ask you something regarding an error I’m facing running java code over the cluster at DIMA.

Caused by: java.io.IOException: Insufficient number of network buffers: required 300, but only 161 available. The total number of network buffers is currently set to 70480. You can increase this number by setting the configuration key 'taskmanager.network.numberOfBuffers'.

at org.apache.flink.runtime.io.network.buffer.NetworkBufferPool.createBufferPool(NetworkBufferPool.java:176)

at org.apache.flink.runtime.io.network.NetworkEnvironment.registerTask(NetworkEnvironment.java:313)

at org.apache.flink.runtime.taskmanager.Task.run(Task.java:464)

at java.lang.Thread.run(Thread.java:853)

The error seems pretty clear so as suggested, I increased  the taskmanager.network.numberOfBuffers from the flink-conf.yaml, (first was setted to 35240), but actually this didn’t solve the problem, and the number of the available buffers is almost always the same.

Could you explain about some further configurations I should check above the cluster?

Besides solve this issue I would be interested in understanding why I get this and where these values come from.

Thanks in advance for your time,

Guido

[Fabian Hueske-2]

Hi Guido,

please check this section of the configuration documentation:
https://ci.apache.org/projects/flink/flink-docs-release-0.10/setup/config.html#configuring-the-network-buffers

It should answer your questions. Please let us know, if not.

Cheers, Fabian

2015-11-27 16:41 GMT+01:00 Guido <[hidden email]>:

Hello,

I would like to ask you something regarding an error I’m facing running java code over the cluster at DIMA.

Caused by: java.io.IOException: Insufficient number of network buffers: required 300, but only 161 available. The total number of network buffers is currently set to 70480. You can increase this number by setting the configuration key 'taskmanager.network.numberOfBuffers'.

at org.apache.flink.runtime.io.network.buffer.NetworkBufferPool.createBufferPool(NetworkBufferPool.java:176)

at org.apache.flink.runtime.io.network.NetworkEnvironment.registerTask(NetworkEnvironment.java:313)

at org.apache.flink.runtime.taskmanager.Task.run(Task.java:464)

at java.lang.Thread.run(Thread.java:853)

The error seems pretty clear so as suggested, I increased  the taskmanager.network.numberOfBuffers from the flink-conf.yaml, (first was setted to 35240), but actually this didn’t solve the problem, and the number of the available buffers is almost always the same.

Could you explain about some further configurations I should check above the cluster?

Besides solve this issue I would be interested in understanding why I get this and where these values come from.

Thanks in advance for your time,

Guido