Re: CRD compatible with native and standalone mode

Posted by austin.ce on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/CRD-compatible-with-native-and-standalone-mode-tp43066p43084.html

Hi Gaurav,

I think the name "Native Kubernetes" is a bit misleading – this just means that you can use the Flink CLI/ scripts to run Flink applications on Kubernetes without using the Kubernetes APIs/ kubectl directly. What features are you looking to use in the native mode?

I think it would be difficult to use this directly inside an operator, but keeping "feature parity" with it is a good goal for your CRDs. Since CRDs are essentially just a new API, the design approach should be user/ feature-first. By feature parity, I mean taking currently supported "Native Kubernetes" functionality as the feature list for your CRDs, for example:

* Allowing Secrets to be mounted as files and environment variables [1]

* Allowing templating of the JobManager and TaskManager Pods (containers, etc.) [2]

* Easy use of built-in plugins [3]

* etc.

Other "Native Kubernetes" "features", like RBAC and logs, will come "out of the box" by defining Custom Resources.

Yang's resources are a great place to start, though I'd suggest defining your API spec within the CRD explicitly[4], which will be clearer for your users and will allow for schema validation by other tools.

Best,

Austin

[1]: https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/deployment/resource-providers/native_kubernetes/#using-secrets

[2]: https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/deployment/resource-providers/native_kubernetes/#pod-template

[3]: https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/deployment/resource-providers/native_kubernetes/#using-plugins

[4]: https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/#specifying-a-structural-schema

On Tue, Apr 20, 2021 at 2:13 AM Yang Wang <[hidden email]> wrote:

I think the compatibility depends on you. For example, you could have the same

CustomResourceDefinition for standalone and native Flink applications. They could

look like this[1].

Since the CR is defined in yaml[2], native and standalone could have some dedicated fields.

And you could easily parse them in your K8s operator.

[1]. https://github.com/wangyang0918/flink-native-k8s-operator/blob/master/deploy/crd.yaml

[2]. https://github.com/wangyang0918/flink-native-k8s-operator/blob/master/deploy/cr.yaml

Best,

Yang

gaurav kulkarni <[hidden email]> 于2021年4月20日周二 上午8:57写道：

Hi, 

I plan to create a flink K8s operator which supports standalone mode, and and switch to native mode sometime later. I was wondering what are some of the approaches to ensure that CRD is compatible with both native and standalone mode? 

Thanks