Re: Gelly ran out of memory

Posted by Flavio Pompermaier on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Gelly-ran-out-of-memory-tp2462p2470.html

Using Stephan advice make things goin on! Then I had another exception about a non existing vertex but that is another story :)

Thanks to all for the support!

On Thu, Aug 20, 2015 at 12:09 PM, Stephan Ewen <[hidden email]> wrote:

Actually, you ran out of "Flink Managed Memory", not user memory. User memory shortage manifests itself as Java OutofMemoryError.

At this point, the Delta iterations cannot spill. They additionally suffer a bit from memory fragmentation.

A possible workaround is to use the option "setSolutionSetUnmanaged(true)" on the iteration. That will eliminate the fragmentation issue, at least.

Stephan

On Thu, Aug 20, 2015 at 12:06 PM, Andra Lungu <[hidden email]> wrote:

Hi Flavio,

These kinds of exceptions generally arise from the fact that you ran out of `user` memory. You can try to increase that a bit.

In your flink-conf.yaml try adding
# The memory fraction allocated system -user
taskmanager.memory.fraction: 0.4

This will give 0.6 of the unit of memory to the user and 0.4 to the system.

Tell me if that helped.

Andra

On Thu, Aug 20, 2015 at 12:02 PM, Flavio Pompermaier <[hidden email]> wrote:

Hi to all,

I tried to run my gelly job on Flink 0.9-SNAPSHOT and I was having an EOFException, so I tried on 0.10-SNAPSHOT and now I have the following error:

Caused by: java.lang.RuntimeException: Memory ran out. Compaction failed. numPartitions: 32 minPartition: 73 maxPartition: 80 number of overflow segments: 0 bucketSize: 570 Overall memory: 102367232 Partition memory: 81100800 Message: null

at org.apache.flink.runtime.operators.hash.CompactingHashTable.insertRecordIntoPartition(CompactingHashTable.java:465)

at org.apache.flink.runtime.operators.hash.CompactingHashTable.insertOrReplaceRecord(CompactingHashTable.java:414)

at org.apache.flink.runtime.operators.hash.CompactingHashTable.buildTableWithUniqueKey(CompactingHashTable.java:325)

at org.apache.flink.runtime.iterative.task.IterationHeadPactTask.readInitialSolutionSet(IterationHeadPactTask.java:211)

at org.apache.flink.runtime.iterative.task.IterationHeadPactTask.run(IterationHeadPactTask.java:272)

at org.apache.flink.runtime.operators.RegularPactTask.invoke(RegularPactTask.java:354)

at org.apache.flink.runtime.taskmanager.Task.run(Task.java:581)

at java.lang.Thread.run(Thread.java:745)

Probably I'm doing something wrong but I can't understand how to estimate the required memory for my Gelly job..

Best,

Flavio