Large state RocksDb backend increases app start time

Posted by Arpith P on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Large-state-RocksDb-backend-increases-app-start-time-tp38720.html

Hi,

I'm currently storing around 70GB of data in map sate backed by RocksDB backend . Once I restore an application from savepoint currently the application takes more than 4mins to start processing events. How can I speed this up or is there any other recommended approach.

I'm using the following predefined options with RocksDB.

RocksDBStateBackend backend = new RocksDBStateBackend(checkpointDir, incrementalCheckpoints);
backend.setPredefinedOptions(PredefinedOptions.SPINNING_DISK_OPTIMIZED_HIGH_MEM);

Thanks,

Arpith