I have a similar problem as this topic [1], this problem was caused by a bug in the software, afaik my problem isn't the same.
The error: Caused by: java.lang.RuntimeException: Memory ran out. Compaction failed. numPartitions: 18 minPartition: 4 maxPartition: 5 number of overflow segments: 54 bucketSize: 39 Overall memory: 5963776 Partition memory: 2785280 Message: Index: 5, Size: 4
Full error [2], dataset [3], code [4]
It happens when i run a simple iteration algorithm like
Pagerank,
SSSP,
label propagation.
In the topic [1] they talk about a workaround, although this works it prevents other flin features.
So far i tried changing a bunch of configuration in Flink and increasing the memory allocation for Java fort he job, but the error stays the same, indicating that it CompactingHashTable.java has too little memory to work with and i'm out of ideas where to look what i'm doing wrong
[1]
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/RuntimeException-Gelly-API-Memory-ran-out-Compaction-failed-td866.html[2]
http://paste.thezomg.com/19947/14805185/[3]
http://konect.uni-koblenz.de/networks/munmun_twitter_social[4]
https://github.com/otherwise777/Temporal_Graph_library/blob/master/src/main/java/Tgraphs/memoryranouterror.java
Locked
