Security Control of running Flink Jobs on Flink UI

[Raja.Aravapalli]

 

Hi,

 

I have started a Flink session/cluster on a existing Hadoop Yarn Cluster using Flink Yarn-Session, and submitting Flink streaming jobs to it… and everything works fine.

 

But, one problem I see with this approach is:

 

The Flink Yarn-Session is running with a yarn application id. And this application id is visible in Yarn Resource Manager UI.

 

And this flink-session can be tracked from resource manager to Flink Session UI

 

From which other users on the Hadoop cluster was able to see and CANCEL the running Flink jobs!

 

Users who are browsing the UI are un-expectedly hitting the button without knowing the impact…. !! Can someone pls guide me on how to control this in UI ?

 

 

Thanks a lot.

 

Regards,

Raja.

[rmetzger0]

Hi Raja,

you can actually disable the UI by setting the port to a negative number.

The configuration property is "jobmanager.web.port".

I'm not sure how well this is tested, but from the code it seems that this is the behavior of Flink.

If that doesn't work, I would propose to add a change to Flink to introduce a special config flag to disable the Cancel functionality in the UI.

The change is probably not too hard to do. 

Regards,

Robert

On Thu, Aug 24, 2017 at 7:04 PM, Raja.Aravapalli <[hidden email]> wrote:

 

Hi,

 

I have started a Flink session/cluster on a existing Hadoop Yarn Cluster using Flink Yarn-Session, and submitting Flink streaming jobs to it… and everything works fine.

 

But, one problem I see with this approach is:

 

The Flink Yarn-Session is running with a yarn application id. And this application id is visible in Yarn Resource Manager UI.

 

And this flink-session can be tracked from resource manager to Flink Session UI

 

From which other users on the Hadoop cluster was able to see and CANCEL the running Flink jobs!

 

Users who are browsing the UI are un-expectedly hitting the button without knowing the impact…. !! Can someone pls guide me on how to control this in UI ?

 

 

Thanks a lot.

 

Regards,

Raja.

[Ted Yu]

bq.  introduce a special config flag to disable the Cancel functionality

+1

Similar config is used in other project(s) such as hbase.

On Fri, Aug 25, 2017 at 2:54 PM, Robert Metzger <[hidden email]> wrote:

Hi Raja,

you can actually disable the UI by setting the port to a negative number.

The configuration property is "jobmanager.web.port".

I'm not sure how well this is tested, but from the code it seems that this is the behavior of Flink.

If that doesn't work, I would propose to add a change to Flink to introduce a special config flag to disable the Cancel functionality in the UI.

The change is probably not too hard to do. 

Regards,

Robert

On Thu, Aug 24, 2017 at 7:04 PM, Raja.Aravapalli <[hidden email]> wrote:

 

Hi,

 

I have started a Flink session/cluster on a existing Hadoop Yarn Cluster using Flink Yarn-Session, and submitting Flink streaming jobs to it… and everything works fine.

 

But, one problem I see with this approach is:

 

The Flink Yarn-Session is running with a yarn application id. And this application id is visible in Yarn Resource Manager UI.

 

And this flink-session can be tracked from resource manager to Flink Session UI

 

From which other users on the Hadoop cluster was able to see and CANCEL the running Flink jobs!

 

Users who are browsing the UI are un-expectedly hitting the button without knowing the impact…. !! Can someone pls guide me on how to control this in UI ?

 

 

Thanks a lot.

 

Regards,

Raja.

[Raja.Aravapalli]

Ability to disable it will be a super helpful.

 

+1 to the idea.

 

 

Regards,

Raja.

 

 

From: Ted Yu <[hidden email]>
Date: Friday, August 25, 2017 at 4:56 PM
To: Robert Metzger <[hidden email]>
Cc: Raja Aravapalli <[hidden email]>, "[hidden email]" <[hidden email]>
Subject: [EXTERNAL] Re: Security Control of running Flink Jobs on Flink UI

 

bq.  introduce a special config flag to disable the Cancel functionality

 

+1

 

Similar config is used in other project(s) such as hbase.

 

On Fri, Aug 25, 2017 at 2:54 PM, Robert Metzger <[hidden email]> wrote:

Hi Raja,

 

you can actually disable the UI by setting the port to a negative number.

The configuration property is "jobmanager.web.port".

I'm not sure how well this is tested, but from the code it seems that this is the behavior of Flink.

 

If that doesn't work, I would propose to add a change to Flink to introduce a special config flag to disable the Cancel functionality in the UI.

The change is probably not too hard to do. 

 

Regards,

Robert

 

 

 

On Thu, Aug 24, 2017 at 7:04 PM, Raja.Aravapalli <[hidden email]> wrote:

 

Hi,

 

I have started a Flink session/cluster on a existing Hadoop Yarn Cluster using Flink Yarn-Session, and submitting Flink streaming jobs to it… and everything works fine.

 

But, one problem I see with this approach is:

 

The Flink Yarn-Session is running with a yarn application id. And this application id is visible in Yarn Resource Manager UI.

 

And this flink-session can be tracked from resource manager to Flink Session UI

 

From which other users on the Hadoop cluster was able to see and CANCEL the running Flink jobs!

 

Users who are browsing the UI are un-expectedly hitting the button without knowing the impact…. !! Can someone pls guide me on how to control this in UI ?

 

 

Thanks a lot.

 

Regards,

Raja.

 

 

[Ted Yu]

Logged FLINK-7525, referring to this thread.

On Fri, Aug 25, 2017 at 3:23 PM, Raja.Aravapalli <[hidden email]> wrote:

Ability to disable it will be a super helpful.

 

+1 to the idea.

 

 

Regards,

Raja.

 

 

From: Ted Yu <[hidden email]>
Date: Friday, August 25, 2017 at 4:56 PM
To: Robert Metzger <[hidden email]>
Cc: Raja Aravapalli <[hidden email]>, "[hidden email]" <[hidden email]>
Subject: [EXTERNAL] Re: Security Control of running Flink Jobs on Flink UI

 

bq.  introduce a special config flag to disable the Cancel functionality

 

+1

 

Similar config is used in other project(s) such as hbase.

 

On Fri, Aug 25, 2017 at 2:54 PM, Robert Metzger <[hidden email]> wrote:

Hi Raja,

 

you can actually disable the UI by setting the port to a negative number.

The configuration property is "jobmanager.web.port".

I'm not sure how well this is tested, but from the code it seems that this is the behavior of Flink.

 

If that doesn't work, I would propose to add a change to Flink to introduce a special config flag to disable the Cancel functionality in the UI.

The change is probably not too hard to do. 

 

Regards,

Robert

 

 

 

On Thu, Aug 24, 2017 at 7:04 PM, Raja.Aravapalli <[hidden email]> wrote:

 

Hi,

 

I have started a Flink session/cluster on a existing Hadoop Yarn Cluster using Flink Yarn-Session, and submitting Flink streaming jobs to it… and everything works fine.

 

But, one problem I see with this approach is:

 

The Flink Yarn-Session is running with a yarn application id. And this application id is visible in Yarn Resource Manager UI.

 

And this flink-session can be tracked from resource manager to Flink Session UI

 

From which other users on the Hadoop cluster was able to see and CANCEL the running Flink jobs!

 

Users who are browsing the UI are un-expectedly hitting the button without knowing the impact…. !! Can someone pls guide me on how to control this in UI ?

 

 

Thanks a lot.

 

Regards,

Raja.