Re: Flink CLI properties with HA

Posted by vino yang on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Flink-CLI-properties-with-HA-tp21425p21472.html

Hi Sampath,

Flink CLI need to retrieve the JobManager leader address, so it need  to access the HA specific configuration. Because if based on Zookeeper to implement the HA, the leader address information will fetch from Zookeeper.

The main use of config item high-availability.storageDir is storage (Job graph, checkpoint and so on). Actually, the real data is stored under this path which used to recover purpose, zookeeper just store a state handle.

---

Thanks.

vino.

2018-07-16 15:28 GMT+08:00 Sampath Bhat <[hidden email]>:

---------- Forwarded message ----------
From: Sampath Bhat <[hidden email]>
Date: Fri, Jul 13, 2018 at 3:18 PM
Subject: Flink CLI properties with HA
To: user <[hidden email]>

Hello

When HA is enabled in the flink cluster and if I've to submit job via flink CLI then in the flink-conf.yaml of flink CLI should contain this properties -
high-availability: zookeeper
high-availability.cluster-id: flink
high-availability.zookeeper.path.root: flink
high-availability.storageDir: <some path>
high-availability.zookeeper.quorum: <zookeeper IP:port>

What is the need of high-availability.storageDir for flink CLI. Does this mean that even flink client should be able to access the mentioned path or is it some check being done on the property name?

Without these properties flink cli will not be able to submit job to flink cluster when HA is enabled.