(DEPRECATED) Apache Flink User Mailing List archive.

CRD compatible with native and standalone mode

classic Classic list List threaded Threaded
♦
♦
 5 messages Options
gaurav kulkarni
Reply | Threaded
Open this post in threaded view
♦
♦
|

CRD compatible with native and standalone mode

gaurav kulkarni
Hi, 

I plan to create a flink K8s operator which supports standalone mode, and and switch to native mode sometime later. I was wondering what are some of the approaches to ensure that CRD is compatible with both native and standalone mode? 

Thanks 
Yang Wang
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: CRD compatible with native and standalone mode

Yang Wang
I think the compatibility depends on you. For example, you could have the same
CustomResourceDefinition for standalone and native Flink applications. They could
look like this[1].

Since the CR is defined in yaml[2], native and standalone could have some dedicated fields.
And you could easily parse them in your K8s operator.



Best,
Yang

gaurav kulkarni <[hidden email]> 于2021年4月20日周二 上午8:57写道：
Hi, 

I plan to create a flink K8s operator which supports standalone mode, and and switch to native mode sometime later. I was wondering what are some of the approaches to ensure that CRD is compatible with both native and standalone mode? 

Thanks 
austin.ce
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: CRD compatible with native and standalone mode

austin.ce
Hi Gaurav,

I think the name "Native Kubernetes" is a bit misleading – this just means that you can use the Flink CLI/ scripts to run Flink applications on Kubernetes without using the Kubernetes APIs/ kubectl directly. What features are you looking to use in the native mode?

I think it would be difficult to use this directly inside an operator, but keeping "feature parity" with it is a good goal for your CRDs. Since CRDs are essentially just a new API, the design approach should be user/ feature-first. By feature parity, I mean taking currently supported "Native Kubernetes" functionality as the feature list for your CRDs, for example:
* Allowing Secrets to be mounted as files and environment variables [1]
* Allowing templating of the JobManager and TaskManager Pods (containers, etc.) [2]
* Easy use of built-in plugins [3]
* etc.

Other "Native Kubernetes" "features", like RBAC and logs, will come "out of the box" by defining Custom Resources.

Yang's resources are a great place to start, though I'd suggest defining your API spec within the CRD explicitly[4], which will be clearer for your users and will allow for schema validation by other tools.

Best,
Austin



On Tue, Apr 20, 2021 at 2:13 AM Yang Wang <[hidden email]> wrote:
I think the compatibility depends on you. For example, you could have the same
CustomResourceDefinition for standalone and native Flink applications. They could
look like this[1].

Since the CR is defined in yaml[2], native and standalone could have some dedicated fields.
And you could easily parse them in your K8s operator.



Best,
Yang

gaurav kulkarni <[hidden email]> 于2021年4月20日周二 上午8:57写道：
Hi, 

I plan to create a flink K8s operator which supports standalone mode, and and switch to native mode sometime later. I was wondering what are some of the approaches to ensure that CRD is compatible with both native and standalone mode? 

Thanks 
gaurav kulkarni
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: CRD compatible with native and standalone mode

gaurav kulkarni
Thanks a lot for the response, folks! I appreciate it. I plan to use native mode in future mostly for the resource management it plans to offer. Let me go through the links provided.  

@Yang Wang "Since the CR is defined in yaml[2], native and standalone could have some dedicated fields. And you could easily parse them in your K8s operator.":  Do you mean having fields that are needed for both native and standalone mode in the CRD (probably making them optional in the CRD) and each operator type (standalone/native) using fields that are relevant for it? 

Thanks,
Gaurav


On Tuesday, April 20, 2021, 08:23:34 AM PDT, Austin Cawley-Edwards <[hidden email]> wrote:


Hi Gaurav,

I think the name "Native Kubernetes" is a bit misleading – this just means that you can use the Flink CLI/ scripts to run Flink applications on Kubernetes without using the Kubernetes APIs/ kubectl directly. What features are you looking to use in the native mode?

I think it would be difficult to use this directly inside an operator, but keeping "feature parity" with it is a good goal for your CRDs. Since CRDs are essentially just a new API, the design approach should be user/ feature-first. By feature parity, I mean taking currently supported "Native Kubernetes" functionality as the feature list for your CRDs, for example:
* Allowing Secrets to be mounted as files and environment variables [1]
* Allowing templating of the JobManager and TaskManager Pods (containers, etc.) [2]
* Easy use of built-in plugins [3]
* etc.

Other "Native Kubernetes" "features", like RBAC and logs, will come "out of the box" by defining Custom Resources.

Yang's resources are a great place to start, though I'd suggest defining your API spec within the CRD explicitly[4], which will be clearer for your users and will allow for schema validation by other tools.

Best,
Austin



On Tue, Apr 20, 2021 at 2:13 AM Yang Wang <[hidden email]> wrote:
I think the compatibility depends on you. For example, you could have the same
CustomResourceDefinition for standalone and native Flink applications. They could
look like this[1].

Since the CR is defined in yaml[2], native and standalone could have some dedicated fields.
And you could easily parse them in your K8s operator.



Best,
Yang

gaurav kulkarni <[hidden email]> 于2021年4月20日周二 上午8:57写道：
Hi, 

I plan to create a flink K8s operator which supports standalone mode, and and switch to native mode sometime later. I was wondering what are some of the approaches to ensure that CRD is compatible with both native and standalone mode? 

Thanks 
Yang Wang
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: CRD compatible with native and standalone mode

Yang Wang
Exactly.

I think most of the fields could be shared by standalone and native mode.


Best,
Yang

gaurav kulkarni <[hidden email]> 于2021年4月21日周三 上午10:17写道：
Thanks a lot for the response, folks! I appreciate it. I plan to use native mode in future mostly for the resource management it plans to offer. Let me go through the links provided.  

@Yang Wang "Since the CR is defined in yaml[2], native and standalone could have some dedicated fields. And you could easily parse them in your K8s operator.":  Do you mean having fields that are needed for both native and standalone mode in the CRD (probably making them optional in the CRD) and each operator type (standalone/native) using fields that are relevant for it? 

Thanks,
Gaurav


On Tuesday, April 20, 2021, 08:23:34 AM PDT, Austin Cawley-Edwards <[hidden email]> wrote:


Hi Gaurav,

I think the name "Native Kubernetes" is a bit misleading – this just means that you can use the Flink CLI/ scripts to run Flink applications on Kubernetes without using the Kubernetes APIs/ kubectl directly. What features are you looking to use in the native mode?

I think it would be difficult to use this directly inside an operator, but keeping "feature parity" with it is a good goal for your CRDs. Since CRDs are essentially just a new API, the design approach should be user/ feature-first. By feature parity, I mean taking currently supported "Native Kubernetes" functionality as the feature list for your CRDs, for example:
* Allowing Secrets to be mounted as files and environment variables [1]
* Allowing templating of the JobManager and TaskManager Pods (containers, etc.) [2]
* Easy use of built-in plugins [3]
* etc.

Other "Native Kubernetes" "features", like RBAC and logs, will come "out of the box" by defining Custom Resources.

Yang's resources are a great place to start, though I'd suggest defining your API spec within the CRD explicitly[4], which will be clearer for your users and will allow for schema validation by other tools.

Best,
Austin



On Tue, Apr 20, 2021 at 2:13 AM Yang Wang <[hidden email]> wrote:
I think the compatibility depends on you. For example, you could have the same
CustomResourceDefinition for standalone and native Flink applications. They could
look like this[1].

Since the CR is defined in yaml[2], native and standalone could have some dedicated fields.
And you could easily parse them in your K8s operator.



Best,
Yang

gaurav kulkarni <[hidden email]> 于2021年4月20日周二 上午8:57写道：
Hi, 

I plan to create a flink K8s operator which supports standalone mode, and and switch to native mode sometime later. I was wondering what are some of the approaches to ensure that CRD is compatible with both native and standalone mode? 

Thanks 
Free forum by Nabble Edit this page