State backend considerations

[Nick Bendtner]

Hi guys,

I have a few questions on state backends.

Is there a guideline on how big the state has to be where it makes sense to use RocksDB rather than FsStatebackend ? Is there an analysis on latency for a full checkpoint for FsSateBackend based on increase in state size ? 

Best,

Nick. 

[Arvid Heise-3]

Hi Nick,

Both questions are hard to answer given that it depends on your hardware, access patterns (read/update), record size/structure, parallelism, and probably a ton of other parameters.

The usual approach is to simply evaluate it in your setting. Since it's a matter of configuration, you can do some A/B testing.

In general, you need RocksDB if you want to have incremental checkpoints, which is recommended if you have rather few updates and big state.

On Mon, Jun 22, 2020 at 2:13 AM Nick Bendtner <[hidden email]> wrote:

Hi guys,

I have a few questions on state backends.

Is there a guideline on how big the state has to be where it makes sense to use RocksDB rather than FsStatebackend ? Is there an analysis on latency for a full checkpoint for FsSateBackend based on increase in state size ? 

Best,

Nick. 

--