(DEPRECATED) Apache Flink User Mailing List archive.

Re: Flink WebUI on YARN behind firewall

Posted by Trevor Grant on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Flink-WebUI-on-YARN-behind-firewall-tp8721p8723.html

Ahh, sorry I misunderstood.

Your comment provided insight for me though.  

To anyone else who is having issues, maybe the following will help them.  I was trying to deploy Flink on an IBM BigInsights Cloud cluster (disclaimer, I work for IBM, not trying to promote a company, but they do give me free cluster time so I tend to play there). 

At anyrate- flink_example.py is a script that will do it nice and clean for anyone on BigInsights Cloud, and may be useful for other similar cloud YARN deployments so I'll just leave it here...
https://github.com/rawkintrevo/bluemix-extra-services

The solution in short- 
1) install flink- 
2) run yarn-session.sh
3) parse the stdout from above command (see python script, easy to follow or email me if not)
4) open the ApplicationUI via the Yarn UI, go to the #/submit tab
5) in the submit tab, IF you open with a web browser (bc it relies on javascript) it may give you a hostname:port <- save that port
6) combine the port you just got , with an IP address you got when parsing in step 3
7) open ssh tunneling to that IP:port

Ugly, but it does work.  

best

tg



Trevor Grant
Data Scientist

"Fortunate is he, who is able to know the causes of things."  -Virgil


On Fri, Aug 26, 2016 at 5:54 PM, Vijay Srinivasaraghavan <[hidden email]> wrote:
Hi Trevor,

Sorry, I think I did not explain it clearly. I am not working on the issue that you have mentioned but I have noticed similar problem for another JIRA that I am currenty working (Flink Web UI basic authentication). The redirection from Yarn UI to Flink does not propogate the authentication headers. I was planning to investigate from Yarn UI as to what exactly is happening. Please go-ahead and create a JIRA anyway and someone will pick it up.

Regards
Vijay


On Friday, August 26, 2016 1:34 PM, Trevor Grant <[hidden email]> wrote:


That's awesome Vijay, thanks for the work- will definitely help (esp as Flink gets picked up for more cloud cluster deployments)!

Do you have the JIRA so I can watch it?

Trevor Grant
Data Scientist

"Fortunate is he, who is able to know the causes of things."  -Virgil


On Fri, Aug 26, 2016 at 1:14 PM, Vijay Srinivasaraghavan <[hidden email]> wrote:
Hi Trevor,

I am seeing similar issue for a JIRA that I am working now. I am yet to trace the Yarn Web UI code to find out how the "tracking URL" is being handled. To ublock, you could use the tracking URL (Flink UI URL) directly to access Flink Web UI to by-pass Yarn UI redirection. You can find the tracking URI in the Job Manager log file from Yarn container.

Regards
Vijay


On Friday, August 26, 2016 10:52 AM, Trevor Grant <[hidden email]> wrote:


I decided it made the most sense to open up a new thread. 

I am running Flink on a cluster behind a firewall.  Things seem to be working fine, but when I access the YARN web-ui and click on the flink application-ui, i get the jobmanager ui, but it is broken.

It is a broken link to a flink image and 
but none of the links work, no images no pretty formatting.  does anyone have a quick idea why this would be?  I found http://mail-archives. apache.org/mod_mbox/flink- user/201511.mbox/%3CCAGr9p8B_ aKhbjLqVQQUZeO_eSz6P=eWKP+ Kg1sQ65SB0npsX2Q@.... com%3E

but that seems to be more related to running jobs than accessing the UI

any help would be appreciated

tg


Trevor Grant
Data Scientist

"Fortunate is he, who is able to know the causes of things."  -Virgil







Free forum by Nabble Edit this page