User / Job Manager (permissions) for Flink

[David Magalhães]

Hi, not sure if this was discussed (for a brief search I couldn't find anything), but I would like to know if there is an application that uses Flink REST API to provide some kind of user management, like allow a certain user to login and manage some jobs running in the link, limit the parallelization for each job, etc.

The idea was for certain users can only access their job, and can stop/start their jobs, but can't interfere with other jobs that run on the same cluster.

Thanks

[r_khachatryan]

Hi David,

One option is Ververica Platform which has a notion of Namespaces: https://docs.ververica.com/administration/namespaces.html

I guess Konstantin can tell you more about it.

Disclaimer: I work for a company that develops this product.

On Tue, Jun 2, 2020 at 5:37 PM David Magalhães <[hidden email]> wrote:

Hi, not sure if this was discussed (for a brief search I couldn't find anything), but I would like to know if there is an application that uses Flink REST API to provide some kind of user management, like allow a certain user to login and manage some jobs running in the link, limit the parallelization for each job, etc.

The idea was for certain users can only access their job, and can stop/start their jobs, but can't interfere with other jobs that run on the same cluster.

Thanks

[rmetzger0]

Hi David,

I guess you could also "just" put a nginx in front of Flink's REST API that controls access.

On Tue, Jun 2, 2020 at 7:05 PM Khachatryan Roman <[hidden email]> wrote:

Hi David,

One option is Ververica Platform which has a notion of Namespaces: https://docs.ververica.com/administration/namespaces.html

I guess Konstantin can tell you more about it.

Disclaimer: I work for a company that develops this product.

Regards,
Roman

On Tue, Jun 2, 2020 at 5:37 PM David Magalhães <[hidden email]> wrote:

Hi, not sure if this was discussed (for a brief search I couldn't find anything), but I would like to know if there is an application that uses Flink REST API to provide some kind of user management, like allow a certain user to login and manage some jobs running in the link, limit the parallelization for each job, etc.

The idea was for certain users can only access their job, and can stop/start their jobs, but can't interfere with other jobs that run on the same cluster.

Thanks

[Arvid Heise-3]

If you are running in K8s, you could also directly use the ingest layer of that. That's especially convenient if you have managed to connect that your company's SSO.

On Tue, Jun 2, 2020 at 9:38 PM Robert Metzger <[hidden email]> wrote:

Hi David,

I guess you could also "just" put a nginx in front of Flink's REST API that controls access.

On Tue, Jun 2, 2020 at 7:05 PM Khachatryan Roman <[hidden email]> wrote:

Hi David,

One option is Ververica Platform which has a notion of Namespaces: https://docs.ververica.com/administration/namespaces.html

I guess Konstantin can tell you more about it.

Disclaimer: I work for a company that develops this product.

Regards,
Roman

On Tue, Jun 2, 2020 at 5:37 PM David Magalhães <[hidden email]> wrote:

Hi, not sure if this was discussed (for a brief search I couldn't find anything), but I would like to know if there is an application that uses Flink REST API to provide some kind of user management, like allow a certain user to login and manage some jobs running in the link, limit the parallelization for each job, etc.

The idea was for certain users can only access their job, and can stop/start their jobs, but can't interfere with other jobs that run on the same cluster.

Thanks

--