Connecting to a service (Eg: Kafka, Cassandra) with different keytabs from the same cluster

[HarshithBolar]

Hi all,

We have a central Flink cluster which will be used by multiple different teams (Data Science, Engineering etc). Each team has their own user and keytab to connect to services like Kafka, Cassandra etc. How should the jobs be configured such that different jobs use different keytabs and principals to connect to Kafka?

Right now with a single user, we have the following entry in the jaas.conf file.

KafkaClient {

   com.sun.security.auth.module.Krb5LoginModule required

   useTicketCache=false

   renewTicket=true

   useKeyTab=true

   keyTab="/opt/certs/flink-user.keytab"

   serviceName="kafka"

   principal="[hidden email]";

};

How can I add more users like this and reference them in the jobs separately?

Thanks,

Harshith

[Dawid Wysakowicz-2]

Hi Harshith,

I haven't tried it, but for Kafka you should be able to use the dynamic sasl configuration of the underlying KafkaConsumer. Try setting the `sasl.jaas.config` parameter for the FlinkKafkaConsumer as per the Kafka documentation.

As far as I know if you use a Flink's specific way of distributing security contexts[1], you can provide only a single credentials for a single Flink cluster.

Best,

Dawid

[1] https://ci.apache.org/projects/flink/flink-docs-master/ops/security-kerberos.html#kerberos-authentication-setup-and-configuration

On 20/05/2019 10:15, Kumar Bolar, Harshith wrote:

Hi all,

We have a central Flink cluster which will be used by multiple different teams (Data Science, Engineering etc). Each team has their own user and keytab to connect to services like Kafka, Cassandra etc. How should the jobs be configured such that different jobs use different keytabs and principals to connect to Kafka?

Right now with a single user, we have the following entry in the jaas.conf file.

KafkaClient {

   com.sun.security.auth.module.Krb5LoginModule required

   useTicketCache=false

   renewTicket=true

   useKeyTab=true

   keyTab="/opt/certs/flink-user.keytab"

   serviceName="kafka"

   principal=[hidden email];

};

How can I add more users like this and reference them in the jobs separately?

Thanks,

Harshith