(DEPRECATED) Apache Flink User Mailing List archive.

Issue in Flink/Zookeeper authentication via Kerberos

classic Classic list List threaded Threaded
♦
♦
 4 messages Options
Sahu, Sarthak 1. (Nokia - IN/Bangalore)
Reply | Threaded
Open this post in threaded view
♦
♦
|

Issue in Flink/Zookeeper authentication via Kerberos

Sahu, Sarthak 1. (Nokia - IN/Bangalore)

Hi Folks,

 

  Environment Setup:

  1. I have configured KDC 5 server.
  2. Configured Kerberos in zookeeper-3.4.10 wherein I can able to connect ZooKeeper Server/Client via Kerberos authentication.
  3. Now flink-1.4.0 has configured for Kerberos authentication as per below instruction.

  Success Scenario:

  1. All Kerberos configuration parameter is correct and flink/zookeeper able to connect trough TGT.

 Problem:

  1. Even if wrong Kerberos credentials given, flink able to connect ZooKeeper.

 

Please find the taskmanager/jobmanger logs and flink config file for both scenario attached.

 

Hoping for quick resolution.

 

Regards

Sarthak Sahu

 


Flink.zip (31K) Download Attachment
Timo Walther
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Issue in Flink/Zookeeper authentication via Kerberos

Timo Walther
Hi Sarthak,

I'm not a Kerberos expert but maybe Eron or Shuyi are more familiar with the details?

Would be great if somebody could help.

Thanks,
Timo

Am 22.03.18 um 10:16 schrieb Sahu, Sarthak 1. (Nokia - IN/Bangalore):

Hi Folks,

 

  Environment Setup:

  1. I have configured KDC 5 server.
  2. Configured Kerberos in zookeeper-3.4.10 wherein I can able to connect ZooKeeper Server/Client via Kerberos authentication.
  3. Now flink-1.4.0 has configured for Kerberos authentication as per below instruction.

  Success Scenario:

  1. All Kerberos configuration parameter is correct and flink/zookeeper able to connect trough TGT.

 Problem:

  1. Even if wrong Kerberos credentials given, flink able to connect ZooKeeper.

 

Please find the taskmanager/jobmanger logs and flink config file for both scenario attached.

 

Hoping for quick resolution.

 

Regards

Sarthak Sahu

 
Shuyi Chen
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Issue in Flink/Zookeeper authentication via Kerberos

Shuyi Chen
Hi Sarthak, 

Happy to help. Could you please share the jobmanager/taskmanager log and flink conf again? 

Also, Flink 1.4.0 has a regression on kerberos security (keytab path in TaskManager is set incorrectly) , which is fixed on 1.4.1. (see https://issues.apache.org/jira/browse/FLINK-8275)

Shuyi

On Mon, Apr 2, 2018 at 3:44 PM, Shuyi Chen <[hidden email]> wrote:
Hi Sarthak, 

Happy to help. Could you please share the jobmanager/taskmanager log and flink conf again? 

Also, Flink 1.4.0 has a regression on kerberos security (keytab path in TaskManager is set incorrectly) , which is fixed on 1.4.1. (see https://issues.apache.org/jira/browse/FLINK-8275)

Shuyi

On Mon, Mar 26, 2018 at 2:35 AM, Timo Walther <[hidden email]> wrote:
Hi Sarthak,

I'm not a Kerberos expert but maybe Eron or Shuyi are more familiar with the details?

Would be great if somebody could help.

Thanks,
Timo

Am 22.03.18 um 10:16 schrieb Sahu, Sarthak 1. (Nokia - IN/Bangalore):

Hi Folks,

 

  Environment Setup:

  1. I have configured KDC 5 server.
  2. Configured Kerberos in zookeeper-3.4.10 wherein I can able to connect ZooKeeper Server/Client via Kerberos authentication.
  3. Now flink-1.4.0 has configured for Kerberos authentication as per below instruction.

  Success Scenario:

  1. All Kerberos configuration parameter is correct and flink/zookeeper able to connect trough TGT.

 Problem:

  1. Even if wrong Kerberos credentials given, flink able to connect ZooKeeper.

 

Please find the taskmanager/jobmanger logs and flink config file for both scenario attached.

 

Hoping for quick resolution.

 

Regards

Sarthak Sahu

 





--
"So you have to trust that the dots will somehow connect in your future."



--
"So you have to trust that the dots will somehow connect in your future."
Eron Wright
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Issue in Flink/Zookeeper authentication via Kerberos

Eron Wright
In reply to this post by Timo Walther
I believe that the solution here is to ensure that the znodes created by Flink have an ACL that allows access only to the original creator.   For example, if a given Flink job has a Kerberos identity of "[hidden email]", it should set the znode ACL appropriately to disallow access to any client that doesn't successfully authenticate as that user.  This may be accomplished with the following Flink configuration setting:

high-availability.zookeeper.client.acl: creator

Some code links:

Hope this helps!
Eron

On Sun, Apr 15, 2018 at 2:16 AM, Sahu, Sarthak 1. (Nokia - IN/Bangalore) <[hidden email]> wrote:

Glad to get the reply. With wrong Kerberos information I am expecting an ‘access denied’.

 

As per flink log, it clear states that authentication failed due to Kerberos wrong information and trying to connect with zookeeper with unauthorised mode if zookeeper allows.

And then it connected successfully!

 

Do I missing any configuration in flink/zookeeper side.

Expecting you suggestion here.

 

Regards

Sarthak Sahu

 

From: Eron Wright [mailto:[hidden email]]
Sent: Tuesday, April 3, 2018 3:07 AM
To: Sahu, Sarthak 1. (Nokia - IN/Bangalore) <[hidden email]>
Cc: [hidden email]; Timo Walther <[hidden email]>


Subject: Re: Issue in Flink/Zookeeper authentication via Kerberos

 

Hello, I'm happy to help.  Could you elaborate on the issue that you see?  Are you saying that you expect to get 'access denied' but Zookeeper is allowing the connection anyway?   

 

My first thought is, maybe ZK allows unauthenticated connections but relies on the authorization layer to deny access to nodes based on the ACL.   FLink has a configuration setting to set the 'owner' of the znode.   

 

-Eron

 

On Mon, Apr 2, 2018 at 1:50 AM, Sahu, Sarthak 1. (Nokia - IN/Bangalore) <[hidden email]> wrote:

Hi Eron/Shuyi

 

Could you please help me on this below issue.

 

Regards

Sarthak Sahu

 

From: Timo Walther [mailto:[hidden email]]
Sent: Monday, March 26, 2018 3:05 PM
To: [hidden email]
Cc: [hidden email]; [hidden email]
Subject: Re: Issue in Flink/Zookeeper authentication via Kerberos

 

Hi Sarthak,

I'm not a Kerberos expert but maybe Eron or Shuyi are more familiar with the details?

Would be great if somebody could help.

Thanks,
Timo

Am 22.03.18 um 10:16 schrieb Sahu, Sarthak 1. (Nokia - IN/Bangalore):

Hi Folks,

 

  Environment Setup:

  1. I have configured KDC 5 server.
  2. Configured Kerberos in zookeeper-3.4.10 wherein I can able to connect ZooKeeper Server/Client via Kerberos authentication.
  3. Now flink-1.4.0 has configured for Kerberos authentication as per below instruction.

·       https://ci.apache.org/projects/flink/flink-docs-release-1.4/ops/config.html#kerberos-based-security

·       https://ci.apache.org/projects/flink/flink-docs-release-1.4/ops/config.html#kerberos-based-security-1

  Success Scenario:

  1. All Kerberos configuration parameter is correct and flink/zookeeper able to connect trough TGT.

 Problem:

  1. Even if wrong Kerberos credentials given, flink able to connect ZooKeeper.

 

Please find the taskmanager/jobmanger logs and flink config file for both scenario attached.

 

Hoping for quick resolution.

 

Regards

Sarthak Sahu

 

 

 
Free forum by Nabble Edit this page