Re: State size increasing exponentially in Flink v1.9

Posted by Chesnay Schepler on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/State-size-increasing-exponentially-in-Flink-v1-9-tp42529p42577.html

Could you show us how you interact with the map state (ideally the full code of your function that accesses the state)?

On 3/25/2021 1:13 AM, Almeida, Julius wrote:

Hey,

Hope you all are doing well!

 

I am using flink v1.9 with RocksDBStateBackend, but over time the state size is increasing exponentially.

 

I am using MapState in my project & seeing memory spike, after looking at heap dump I see duplicates in it.

 

I also have logic added to remove expired events form the MapState

Eg.: MapState.remove(key)

 

Can anyone give me pointers to find more details on it.

 

Heap Dump pointed to https://github.com/apache/flink/blob/master/flink-runtime/src/main/java/org/apache/flink/runtime/state/heap/CopyOnWriteStateMap.java#L811

 

Thanks,

Julius