rest.port is reset to 0 by YarnEntrypointUtils

[Dongwon Kim-2]

Hi,

I'm trying to launch a dispatcher on top of YARN by executing "yarn-session.sh" on the command line.

To access the rest endpoint outside the cluster, I need to assign a port from an allowed range.

YarnEntrypointUtils, however, sets rest.port to 0 for random binding.

Is there any reason on it?


Best,

- Dongwon

[Gary Yao-2]

Hi Dongwon,

I think the rationale was to avoid conflicts between multiple Flink instances
running on the same YARN cluster. There is a ticket that proposes to allow
configuring a port range instead [1].
 
Best,
Gary

[1] https://issues.apache.org/jira/browse/FLINK-5758

On Tue, Apr 17, 2018 at 9:56 AM, Dongwon Kim <[hidden email]> wrote:

Hi,

I'm trying to launch a dispatcher on top of YARN by executing "yarn-session.sh" on the command line.

To access the rest endpoint outside the cluster, I need to assign a port from an allowed range.

YarnEntrypointUtils, however, sets rest.port to 0 for random binding.

Is there any reason on it?


Best,

- Dongwon

[Dongwon Kim-2]

Hi Gary,

Thanks a lot for replay.

Hope the issue is resolved soon.

I have a suggestion regarding the rest port.

Considering the role of dispatcher, it needs to have its own port range that is not shared by job managers spawned by dispatcher.

If I understand FLIP-6 correctly, only a few dispatchers are going to be launched as a kind of gateway and they should be visible outside the cluster.

it seems quite reasonable to give them a range of ports that are only consumed by dispatcher.

Best,

- Dongwon

2018. 4. 18. 오후 6:40, Gary Yao <[hidden email]> 작성:

Hi Dongwon,

I think the rationale was to avoid conflicts between multiple Flink instances
running on the same YARN cluster. There is a ticket that proposes to allow
configuring a port range instead [1].
 
Best,
Gary

[1] https://issues.apache.org/jira/browse/FLINK-5758

On Tue, Apr 17, 2018 at 9:56 AM, Dongwon Kim <[hidden email]> wrote:

Hi,

I'm trying to launch a dispatcher on top of YARN by executing "yarn-session.sh" on the command line.

To access the rest endpoint outside the cluster, I need to assign a port from an allowed range.

YarnEntrypointUtils, however, sets rest.port to 0 for random binding.

Is there any reason on it?


Best,

- Dongwon