Re: Flink Jobmanager HA deployment on k8s
Posted by
Yang Wang on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Flink-Jobmanager-HA-deployment-on-k8s-tp40827p40840.html
If you do not want to run multiple JobManagers simultaneously, then I think the "Job" for application cluster
with HA enable is enough.
K8s will also launch a new pod/container when the old one terminated exceptionally.
Best,
Yang
Yes. Using a "Deployment" instead of "Job" for the application cluster also makes sense.
Actually, in the native K8s integration, we always use the deployment for JobManager.
But please note that the deployment may relaunch the JobManager pod even though you cancel
the Flink job.
Best,
Yang
Yang,
For Application clusters, does it make sense to deploy JobManager as "Deployment" rather than as a "Job", as suggested in docs?
I am asking this because I am thinking of deploying a job manager in HA mode even for application clusters.
Thanks
Ashish
Usually, you do not need to start multiple JobManager simultaneously. The JobManager is a deployment.
A new one pod/container will be launched once it terminated exceptionally.
If you still want to start multiple JobManagers to get a faster recovery, you could set the replica greater than 1
for standalone cluster on K8s[1]. For native integration[2], we still have not supported such configuration[2].
Please note that the key point to enable HA is not start multiple JobManagers simultaneously or sequently.
You need to set the ZooKeeperHAService[4] or KubernetesHAService[5] to ensure the Flink job could recover
from latest successful checkpoint.
Best,
Yang
Hi,
I am deploying Flink 1.12 on K8s. Can anyone confirm if we can deploy multiple job manager pods in K8s for HA or it should always be only a single job manager pod ?
Regards,
Amit Bhatia