Re: Flink Jobmanager HA deployment on k8s

Posted by Ashish Nigam on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Flink-Jobmanager-HA-deployment-on-k8s-tp40827p40835.html

Yang,

For Application clusters, does it make sense to deploy JobManager as "Deployment" rather than as a "Job", as suggested in docs?

I am asking this because I am thinking of deploying a job manager in HA mode even for application clusters.

Thanks

Ashish

On Tue, Jan 19, 2021 at 6:16 AM Yang Wang <[hidden email]> wrote:

Usually, you do not need to start multiple JobManager simultaneously. The JobManager is a deployment.

A new one pod/container will be launched once it terminated exceptionally. 

If you still want to start multiple JobManagers to get a faster recovery, you could set the replica greater than 1

for standalone cluster on K8s[1]. For native integration[2], we still have not supported such configuration[2].

Please note that the key point to enable HA is not start multiple JobManagers simultaneously or sequently.

You need to set the ZooKeeperHAService[4] or KubernetesHAService[5] to ensure the Flink job could recover

from latest successful checkpoint.

[1]. https://ci.apache.org/projects/flink/flink-docs-master/deployment/resource-providers/standalone/kubernetes.html#session-cluster-resource-definitions

[2]. https://ci.apache.org/projects/flink/flink-docs-master/deployment/resource-providers/native_kubernetes.html

[3]. https://issues.apache.org/jira/browse/FLINK-17707

[4]. https://ci.apache.org/projects/flink/flink-docs-master/deployment/ha/zookeeper_ha.html

[5]. https://ci.apache.org/projects/flink/flink-docs-master/deployment/ha/kubernetes_ha.html

Best,

Yang

Amit Bhatia <[hidden email]> 于2021年1月19日周二 下午8:45写道：

Hi,

I am deploying Flink 1.12 on K8s. Can anyone confirm if we can deploy multiple job manager pods in K8s for HA or it should always be only a single job manager pod ?

Regards,

Amit Bhatia