Flink Client job submission through SSL

[Sampath Bhat]

Hello

I would like to know if the job submission through flink command line say

./bin/flink run <jar/path>

can be authenticated. Like if SSL is enabled then will the job submission require SSL certificates. But I don't see any behavior as such. Simple flink run is able to submit the job even if SSL is enabled.

Can the flink job submission  be guarded by Kerberos, KeyCloak.

Regards

Sampath

[Shuyi Chen]

Hi Sampath,

Yes, Flink support Kerberos authentication for job submission. You can take a look at the document here for more detail (https://ci.apache.org/projects/flink/flink-docs-release-1.4/ops/security-kerberos.html).

Also, please make sure to use Flink release 1.4.1 or above, because there is some regression in previous versions that your job might fail after deploying to secure YARN.

Thanks

Shuyi

On Thu, Apr 5, 2018 at 11:37 PM, Sampath Bhat <[hidden email]> wrote:

Hello

I would like to know if the job submission through flink command line say

./bin/flink run <jar/path>

can be authenticated. Like if SSL is enabled then will the job submission require SSL certificates. But I don't see any behavior as such. Simple flink run is able to submit the job even if SSL is enabled.

Can the flink job submission  be guarded by Kerberos, KeyCloak.

Regards

Sampath

--

[Sampath Bhat]

Hi Chen

The link you shared does not speak about flink job submission and Kerberos interaction. It only speaks about kerberos support for HDFS, zookeeper, kafka and YARN.

Even if Flink supports Kerberos authentication for job submission through command line client then how should i pass the kerberos credentials (keytab and principal) to the flink client?

On Fri, Apr 6, 2018 at 12:56 PM, Shuyi Chen <[hidden email]> wrote:

Hi Sampath,

Yes, Flink support Kerberos authentication for job submission. You can take a look at the document here for more detail (https://ci.apache.org/projects/flink/flink-docs-release-1.4/ops/security-kerberos.html).

Also, please make sure to use Flink release 1.4.1 or above, because there is some regression in previous versions that your job might fail after deploying to secure YARN.

Thanks

Shuyi

On Thu, Apr 5, 2018 at 11:37 PM, Sampath Bhat <[hidden email]> wrote:

Hello

I would like to know if the job submission through flink command line say

./bin/flink run <jar/path>

can be authenticated. Like if SSL is enabled then will the job submission require SSL certificates. But I don't see any behavior as such. Simple flink run is able to submit the job even if SSL is enabled.

Can the flink job submission  be guarded by Kerberos, KeyCloak.

Regards

Sampath

--

[Shuyi Chen]

You need to set the following flag in flink-conf.yaml: security.kerberos.login.keytab and security.kerberos.login.principal.

Please refer to this link for more detail.

Thanks

Shuyi

On Fri, Apr 6, 2018 at 4:22 AM, Sampath Bhat <[hidden email]> wrote:

Hi Chen

The link you shared does not speak about flink job submission and Kerberos interaction. It only speaks about kerberos support for HDFS, zookeeper, kafka and YARN.

Even if Flink supports Kerberos authentication for job submission through command line client then how should i pass the kerberos credentials (keytab and principal) to the flink client?

On Fri, Apr 6, 2018 at 12:56 PM, Shuyi Chen <[hidden email]> wrote:

Hi Sampath,

Yes, Flink support Kerberos authentication for job submission. You can take a look at the document here for more detail (https://ci.apache.org/projects/flink/flink-docs-release-1.4/ops/security-kerberos.html).

Also, please make sure to use Flink release 1.4.1 or above, because there is some regression in previous versions that your job might fail after deploying to secure YARN.

Thanks

Shuyi

On Thu, Apr 5, 2018 at 11:37 PM, Sampath Bhat <[hidden email]> wrote:

Hello

I would like to know if the job submission through flink command line say

./bin/flink run <jar/path>

can be authenticated. Like if SSL is enabled then will the job submission require SSL certificates. But I don't see any behavior as such. Simple flink run is able to submit the job even if SSL is enabled.

Can the flink job submission  be guarded by Kerberos, KeyCloak.

Regards

Sampath

--

--