RocksDB and local file system

[Jaqie Chan]

Hello

I am using Flink rocksDb state backend, the documentation seems to imply i can use a regular file system such as: file:///data/flink/checkpoints, but the code javadoc only mentions hdfs or s3 option here.

I am wondering if it's possible to use local file system with flink rocksdb backend.

Thanks

嘉琪

[vino yang]

Hi Jaqie,

For testing, you can use the local file system pattern (e.g. "file:///"). Technically speaking, it's OK to specify the string path provided by you.

However, in the production environment, we do not recommend using the local file system. Because it does not provide high availability.

Best,

Vino

Jaqie Chan <[hidden email]> 于2019年11月6日周三 下午11:29写道：

Hello

I am using Flink rocksDb state backend, the documentation seems to imply i can use a regular file system such as: file:///data/flink/checkpoints, but the code javadoc only mentions hdfs or s3 option here.

I am wondering if it's possible to use local file system with flink rocksdb backend.

Thanks

嘉琪

[Congxian Qiu]

Hi

The path will store the checkpoints, and Flink will fetch the checkpoint files to restore the state if any failure occurred.  If you specify the local file system, when restoring from checkpoint, Flink may can't find the checkpoint files, and can't restore from last checkpoints.

vino yang <[hidden email]> 于2019年11月7日周四 上午10:57写道：

Hi Jaqie,

For testing, you can use the local file system pattern (e.g. "file:///"). Technically speaking, it's OK to specify the string path provided by you.

However, in the production environment, we do not recommend using the local file system. Because it does not provide high availability.

Best,

Vino

Jaqie Chan <[hidden email]> 于2019年11月6日周三 下午11:29写道：

Hello

I am using Flink rocksDb state backend, the documentation seems to imply i can use a regular file system such as: file:///data/flink/checkpoints, but the code javadoc only mentions hdfs or s3 option here.

I am wondering if it's possible to use local file system with flink rocksdb backend.

Thanks

嘉琪