Does RocksDBStateBackend need a separate RocksDB service?

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Does RocksDBStateBackend need a separate RocksDB service?

wanglei2@geekplus.com.cn

In  my  code,  I just setStateBackend with a hdfs direcoty.
           env.setStateBackend(new RocksDBStateBackend("hdfs://user/test/job")); 

Is there an embeded  RocksDB  service in the flink task? 


Reply | Threaded
Open this post in threaded view
|

Re: Does RocksDBStateBackend need a separate RocksDB service?

miki haiat
There  is no need to add an external RocksDB instance
  .
The RocksDBStateBackend holds in-flight data in a RocksDB database that is (per default) stored in the TaskManager data directories.  


 

On Wed, Aug 7, 2019 at 1:25 PM [hidden email] <[hidden email]> wrote:

In  my  code,  I just setStateBackend with a hdfs direcoty.
           env.setStateBackend(new RocksDBStateBackend("hdfs://user/test/job")); 

Is there an embeded  RocksDB  service in the flink task? 


Reply | Threaded
Open this post in threaded view
|

Re: Does RocksDBStateBackend need a separate RocksDB service?

Biao Liu
Hi wanglei,

> Is there an embeded  RocksDB  service in the flink task?

Yes, and "RocksDB is an embeddable persistent key-value store for fast storage". [1]

1. http://rocksdb.org/

Thanks,
Biao /'bɪ.aʊ/



On Wed, Aug 7, 2019 at 7:27 PM miki haiat <[hidden email]> wrote:
There  is no need to add an external RocksDB instance
  .
The RocksDBStateBackend holds in-flight data in a RocksDB database that is (per default) stored in the TaskManager data directories.  


 

On Wed, Aug 7, 2019 at 1:25 PM [hidden email] <[hidden email]> wrote:

In  my  code,  I just setStateBackend with a hdfs direcoty.
           env.setStateBackend(new RocksDBStateBackend("hdfs://user/test/job")); 

Is there an embeded  RocksDB  service in the flink task?