Take RocksDB state dump

[Harshvardhan Agrawal]

Hello,

We are currently using a RocksDBStateBackend for our Flink pipeline. We want to analyze the data that is stored in Rocksdb state.Is  there a recommended process to do that? The sst_dump tool available from RocksDB isn’t working for us and we keep on getting errors like “Snappy not supported or corrupted Snappy compressed block contents”. My thought was that it might be happening since I am trying to take a dump while the Flink pipeline is running. Upon cancelling the pipeline all the state was removed and I didn’t have any sst files to look at. I was wondering how have people approached this problem.

Regards,

Harsh

--
Regards,<br>Harshvardhan

[Gyula Fóra]

Hi,

If you dont mind a little trying out stuff I have some nice tooling for exactly this:

https://github.com/king/bravo

Let me know if it works :)

Gyula

Harshvardhan Agrawal <[hidden email]> ezt írta (időpont: 2018. okt. 17., Sze, 21:50):

Hello,

We are currently using a RocksDBStateBackend for our Flink pipeline. We want to analyze the data that is stored in Rocksdb state.Is  there a recommended process to do that? The sst_dump tool available from RocksDB isn’t working for us and we keep on getting errors like “Snappy not supported or corrupted Snappy compressed block contents”. My thought was that it might be happening since I am trying to take a dump while the Flink pipeline is running. Upon cancelling the pipeline all the state was removed and I didn’t have any sst files to look at. I was wondering how have people approached this problem.

Regards,

Harsh

--
Regards,<br>Harshvardhan