State size increasing exponentially in Flink v1.9

Posted by Almeida, Julius on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/State-size-increasing-exponentially-in-Flink-v1-9-tp42529.html

Hey,

Hope you all are doing well!

 

I am using flink v1.9 with RocksDBStateBackend, but over time the state size is increasing exponentially.

 

I am using MapState in my project & seeing memory spike, after looking at heap dump I see duplicates in it.

 

I also have logic added to remove expired events form the MapState

Eg.: MapState.remove(key)

 

Can anyone give me pointers to find more details on it.

 

Heap Dump pointed to https://github.com/apache/flink/blob/master/flink-runtime/src/main/java/org/apache/flink/runtime/state/heap/CopyOnWriteStateMap.java#L811

 

Thanks,

Julius