Should Queryable State Server be listening on 127.0.1.1?

[Andrew Kowpak]

I'm running into an issue where I am starting a standalone flink cluster in an lxc container.  When my TaskManager starts up, the queryable state proxy starts listening on 127.0.1.1:9069.  Attempting to connect to that port from outside the container fails.  I'm totally willing to believe this is a configuration problem within my container, so, I just wanted to verify that it was expected behaviour to listen on that IP address.  As far as I can tell, when the TaskManagerRunner creates an RpcService, it finds the task manager address in ConnectionUtls.findConnectingAddress by:

(1) Using AkkaUtils.getInetSocketAddressFromAkkaURL to find the target address (this resolves to 127.0.0.1)

(2) Uses the LOCAL_HOST address detection strategy to find the proper address, this calls InetAddress.getLocalHost which resolves to 127.0.1.1 (as per the default /etc/hosts file on the container)

(3) Determines that a connection can be made from 127.0.1.1 to 127.0.0.1, so uses 127.0.1.1 as the task manager address.

If you can let me know if this is then intended behaviour, that would be great.  If you have any suggestions as to how I can connect to the server from outside my container, that would also be great.

Thanks.

--

[Till Rohrmann]

Hi Andrew,

I think you ran into the same problem we discussed here [1]. I think it is a bug and the KvStateServerImpl or more specifically the AbstractServerBase should bind to any address. Or at least it should be configurable similar to RestOptions#BIND_ADDRESS.

I've opened a JIRA issue to fix this problem [2].

[1] http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Problem-with-querying-state-on-Flink-1-6-td22687.html

[2] https://issues.apache.org/jira/browse/FLINK-10428

Cheers,

Till

On Tue, Sep 25, 2018 at 2:13 PM Andrew Kowpak <[hidden email]> wrote:

I'm running into an issue where I am starting a standalone flink cluster in an lxc container.  When my TaskManager starts up, the queryable state proxy starts listening on 127.0.1.1:9069.  Attempting to connect to that port from outside the container fails.  I'm totally willing to believe this is a configuration problem within my container, so, I just wanted to verify that it was expected behaviour to listen on that IP address.  As far as I can tell, when the TaskManagerRunner creates an RpcService, it finds the task manager address in ConnectionUtls.findConnectingAddress by:

(1) Using AkkaUtils.getInetSocketAddressFromAkkaURL to find the target address (this resolves to 127.0.0.1)

(2) Uses the LOCAL_HOST address detection strategy to find the proper address, this calls InetAddress.getLocalHost which resolves to 127.0.1.1 (as per the default /etc/hosts file on the container)

(3) Determines that a connection can be made from 127.0.1.1 to 127.0.0.1, so uses 127.0.1.1 as the task manager address.

If you can let me know if this is then intended behaviour, that would be great.  If you have any suggestions as to how I can connect to the server from outside my container, that would also be great.

Thanks.

--