state.backend.fs.checkpointdir may support multi path and failover.

[yunfan123]

For now I can only config like:
state.backend.fs.checkpointdir: hdfs://namenodehost:port/XXX
But the machine running namenode may be down.
This config should support multi path in case of some become unavailable.
Normally, user should make sure any of them can be used.

[Petr Novotnik]

I think for this case you want to set up HA name nodes [1].

> https://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithNFS.html

that will allow you to set the checkpoint dir as: `hdfs://mycluster/XXX`

P.

On 05/23/2017 01:56 PM, yunfan123 wrote:

> For now I can only config like:
> state.backend.fs.checkpointdir: hdfs://namenodehost:port/XXX
> But the machine running namenode may be down.
> This config should support multi path in case of some become unavailable.
> Normally, user should make sure any of them can be used.
>
>
>
>
>
> --
> View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/state-backend-fs-checkpointdir-may-support-multi-path-and-failover-tp13264.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.
>