Re: Gelly ran out of memory

Posted by Stephan Ewen on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Gelly-ran-out-of-memory-tp2462p2464.html

Actually, you ran out of "Flink Managed Memory", not user memory. User memory shortage manifests itself as Java OutofMemoryError.

At this point, the Delta iterations cannot spill. They additionally suffer a bit from memory fragmentation.

A possible workaround is to use the option "setSolutionSetUnmanaged(true)" on the iteration. That will eliminate the fragmentation issue, at least.

Stephan

On Thu, Aug 20, 2015 at 12:06 PM, Andra Lungu <[hidden email]> wrote:

Hi Flavio,

These kinds of exceptions generally arise from the fact that you ran out of `user` memory. You can try to increase that a bit.

In your flink-conf.yaml try adding
# The memory fraction allocated system -user
taskmanager.memory.fraction: 0.4

This will give 0.6 of the unit of memory to the user and 0.4 to the system.

Tell me if that helped.

Andra

On Thu, Aug 20, 2015 at 12:02 PM, Flavio Pompermaier <[hidden email]> wrote:

Hi to all,

I tried to run my gelly job on Flink 0.9-SNAPSHOT and I was having an EOFException, so I tried on 0.10-SNAPSHOT and now I have the following error:

Caused by: java.lang.RuntimeException: Memory ran out. Compaction failed. numPartitions: 32 minPartition: 73 maxPartition: 80 number of overflow segments: 0 bucketSize: 570 Overall memory: 102367232 Partition memory: 81100800 Message: null

at org.apache.flink.runtime.operators.hash.CompactingHashTable.insertRecordIntoPartition(CompactingHashTable.java:465)

at org.apache.flink.runtime.operators.hash.CompactingHashTable.insertOrReplaceRecord(CompactingHashTable.java:414)

at org.apache.flink.runtime.operators.hash.CompactingHashTable.buildTableWithUniqueKey(CompactingHashTable.java:325)

at org.apache.flink.runtime.iterative.task.IterationHeadPactTask.readInitialSolutionSet(IterationHeadPactTask.java:211)

at org.apache.flink.runtime.iterative.task.IterationHeadPactTask.run(IterationHeadPactTask.java:272)

at org.apache.flink.runtime.operators.RegularPactTask.invoke(RegularPactTask.java:354)

at org.apache.flink.runtime.taskmanager.Task.run(Task.java:581)

at java.lang.Thread.run(Thread.java:745)

Probably I'm doing something wrong but I can't understand how to estimate the required memory for my Gelly job..

Best,

Flavio