HA setting for per-job YARN session

[sanmutongzi]

Hi,
i am using  the " per-job YARN session " mode deploy flink job on yarn.
The document https://ci.apache.org/projects/flink/flink-docs-release-1.5/ops/jobmanager_high_availability.html#yarn-cluster-high-availability
 says that "we don’t run multiple JobManager (ApplicationMaster)
instances" but still give an example about how to config an Highly
Available YARN Session cluster.

My question is:  does the " per-job YARN session "  mode should  also
use HA setting ? Because in my test case without using zookeeper HA ,
if the YarnApplicationMasterRunner/Jobmanager is failed or killed
,yarn can only restart the YarnApplicationMasterRunner/Jobmanager
program but no job graph is submitted to the Jobmanager again.

THANKS

[Gary Yao-2]

Hi,

The per-job YARN mode also supports HA. Even if the YARN ResourceManager only
brings up one JobManager at a time, you require ZooKeeper for storing
metadata, such as which jobs are supposed to be running. In fact, the
distributed components will still take part in a leader election with only one
contender. The keys that are required to be configured for HA are:
   
    high-availability: zookeeper
    high-availability.zookeeper.quorum: <your zk quorum>
    high-availability.zookeeper.storageDir: <path on DFS>

Best,
Gary

On Mon, Aug 6, 2018 at 6:25 AM, 杨光 <[hidden email]> wrote:

Hi,
i am using  the " per-job YARN session " mode deploy flink job on yarn.
The document https://ci.apache.org/projects/flink/flink-docs-release-1.5/ops/jobmanager_high_availability.html#yarn-cluster-high-availability
 says that "we don’t run multiple JobManager (ApplicationMaster)
instances" but still give an example about how to config an Highly
Available YARN Session cluster.

My question is:  does the " per-job YARN session "  mode should  also
use HA setting ? Because in my test case without using zookeeper HA ,
if the YarnApplicationMasterRunner/Jobmanager is failed or killed
,yarn can only restart the YarnApplicationMasterRunner/Jobmanager
program but no job graph is submitted to the Jobmanager again.

THANKS

[sanmutongzi]

Thanks Gary, I have tested  those ha settings and it works well.




--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/