AM Delegation Token Regeneration

[Paul Lam]

Hi,
At present, Flink distribute keytabs via YARN to the nodes that is running a Flink job, and this might be a potential security problem. I’ve read FLINK-3670 and the corresponding mail list discussions, and I think a more appropriate implementation would be like Spark’s: regenerate delegation tokens in AM and the containers just get the generated delegation token instead of the whole keytab. Also, I noticed that Dispatcher was introduced in FLIP-6 and one of its functionality is acquiring user’s authentication tokens. So, my question is, is delegation token regeneration part of FLIP-6? If not, would it be supported in the future?

Best regards,
Paul Lam

[Shuyi Chen]

Hi Paul, currently, Flink intentionally disables DT and only use keytab. I am not aware that DT regeneration is part of FLIP-6 (@till, correct me if I am wrong). I've created a security improvement design  to document some of the changes we can make to improve flink's security framework, it will be great if you can take a look and let us know what you think. Thanks a lot.

Shuyi

On Mon, Jul 30, 2018 at 4:58 AM Paul Lam <[hidden email]> wrote:

Hi,
At present, Flink distribute keytabs via YARN to the nodes that is running a Flink job, and this might be a potential security problem. I’ve read FLINK-3670 and the corresponding mail list discussions, and I think a more appropriate implementation would be like Spark’s: regenerate delegation tokens in AM and the containers just get the generated delegation token instead of the whole keytab. Also, I noticed that Dispatcher was introduced in FLIP-6 and one of its functionality is acquiring user’s authentication tokens. So, my question is, is delegation token regeneration part of FLIP-6? If not, would it be supported in the future?

Best regards,
Paul Lam

--

[Paul Lam]

Hi Chen,
Thanks for the quick reply! I’ve read the design document and it is very much what I’m looking for. And I think the design was absorbed in FLIP-26, right? I will keep watching this FLIP. Thanks again.

Best regards,
Paul Lam