Re: Clarification on state backend parameters
Posted by
Stefan Richter on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Clarification-on-state-backend-parameters-tp11419p11424.html
Hi,
the purpose of the configuration parameter is described in the documentation under
https://ci.apache.org/projects/flink/flink-docs-release-1.2/setup/config.html. In a nutshell, state.checkpoints.dir contains the (small) meta data files for checkpoints, which typically contains pointers to the files which contain the actual state snapshot data. The state.backend.fs.checkpointdir is the directory into which the actual state from the backends is written. Finally, state.backend.rocksdb.checkpointdir is a poorly named key for the directory of the RocksDB instance data and has in fact nothing to do with checkpoints.
Best,
Stefan
Trying to understand these 3 parameters:
state.backend
state.backend.fs.checkpointdir
state.backend.rocksdb.checkpointdir
state.checkpoints.dir
As I understand stream of data and the state of operators are 2 different concepts and that both need to be checkpointed. I am bit confused about the purpose of these parameters and their applicability.