Re: Clarification on state backend parameters

Posted by Mohit Anchlia on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Clarification-on-state-backend-parameters-tp11419p11434.html

I thought rocksdb is used to as a store backend. If that is the case then why would are there 2 configuration parameter? Or in other words what is the behavior if both state.backend.fs.checkpointdir and state.backend.rocksdb is set?

On Fri, Feb 3, 2017 at 1:47 AM, Stefan Richter <[hidden email]> wrote:

Hi,

the purpose of the configuration parameter is described in the documentation under https://ci.apache.org/projects/flink/flink-docs-release-1.2/setup/config.html. In a nutshell, state.checkpoints.dir contains the (small) meta data files for checkpoints, which typically contains pointers to the files which contain the actual state snapshot data. The state.backend.fs.checkpointdir is the directory into which the actual state from the backends is written. Finally, state.backend.rocksdb.checkpointdir is a poorly named key for the directory of the RocksDB instance data and has in fact nothing to do with checkpoints.

Best,

Stefan

Am 03.02.2017 um 01:45 schrieb Mohit Anchlia <[hidden email]>:

Trying to understand these 3 parameters:

state.backend

state.backend.fs.checkpointdir

state.backend.rocksdb.checkpointdir

state.checkpoints.dir

As I understand stream of data and the state of operators are 2 different concepts and that both need to be checkpointed. I am bit confused about the purpose of these parameters and their applicability.