Re: Replacing a server in Zookeeper Quorum
Posted by
Yang Wang on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Replacing-a-server-in-Zookeeper-Quorum-tp32304p32311.html
Hi Aaron,
I think it is not the responsibility of Flink. Flink uses zookeeper curator to connect the zk server. If
multiple zk server are specified, it has an automatic retry mechanism.
However, your problem is ip address will change when the EMR instance restarts. Currently, Flink
can not support dynamically loading configuration. One quick solution is to use a static ip for EMR
master node[1].
Best,
Yang
Hello Flink Community,
I'm working on a HA setup of Flink 1.8.1 on AWS EMR and have some questions about how Flink interacts with Zookeeper when one of the servers in the quorum specified in flink-conf.yaml goes down and is replaced by a machine with a new IP address.
Currently, I configure high-availability.zookeeper.quorum to be the IP addresses of the 3 master nodes of the EMR cluster, as this is what AWS does to enable a highly available YARN setup.
EMR master nodes may go down entirely and need to be replaced by a machine with a different instance IP address. I will almost certainly need to perform a rolling configuration update to account for this. But will I need to restart flink for this to take effect? Is there a way to dynamically reload these configs when they change?
Aaron