akka.remote.ShutDownAssociation: Shut down address: akka.tcp://flink@fps-flink-jobmanager:45652

[Hao Sun]

I am running Flink 1.3.2 in my local docker environment.

I see this error, not sure how to find the root cause.

I am confused by this error message, why JM is trying to connect to JM from one random port to the RPC port: 6123?

====

2018-01-08 05:38:03,294 ERROR akka.remote.EndpointWriter - AssociationError [akka.tcp://flink@fps-flink-jobmanager:6123] <- [akka.tcp://flink@fps-flink-jobmanager:45652]: Error [Shut down address: akka.tcp://flink@fps-flink-jobmanager:45652] [
akka.remote.ShutDownAssociation: Shut down address: akka.tcp://flink@fps-flink-jobmanager:45652
Caused by: akka.remote.transport.Transport$InvalidAssociationException: The remote system terminated the association because it is shutting down.
]
====

Here you can find the full log

https://gist.github.com/zenhao/c8d13cce8601e321dd706a2ac53f5032

[Fabian Hueske-2]

Hi,

Till (in CC) might be able to help with Akka related questions.

Best, Fabian

2018-01-08 6:46 GMT+01:00 Hao Sun <[hidden email]>:

I am running Flink 1.3.2 in my local docker environment.

I see this error, not sure how to find the root cause.

I am confused by this error message, why JM is trying to connect to JM from one random port to the RPC port: 6123?

====

2018-01-08 05:38:03,294 ERROR akka.remote.EndpointWriter - AssociationError [akka.tcp://flink@fps-flink-jobmanager:6123] <- [akka.tcp://flink@fps-flink-jobmanager:45652]: Error [Shut down address: akka.tcp://flink@fps-flink-jobmanager:45652] [
akka.remote.ShutDownAssociation: Shut down address: akka.tcp://flink@fps-flink-jobmanager:45652
Caused by: akka.remote.transport.Transport$InvalidAssociationException: The remote system terminated the association because it is shutting down.
]
====

Here you can find the full log

https://gist.github.com/zenhao/c8d13cce8601e321dd706a2ac53f5032

[Till Rohrmann]

Hi,

I'm actually not sure what's going on there. I suspect that it must have something to do with the local setup. Not sure from where you submit your job. If you submitted the job from fps-flink-jobmanager then this could the client actor system which shuts down after submitting the job because you submitted the job in detached mode.

It would be helpful if you could describe a little bit more how you set up the Flink cluster and how you submit jobs. Moreover, it would be great to get access to all relevant logs (JM, TMs and client).

Cheers,

Till

On Tue, Jan 9, 2018 at 2:24 PM, Fabian Hueske <[hidden email]> wrote:

Hi,

Till (in CC) might be able to help with Akka related questions.

Best, Fabian

2018-01-08 6:46 GMT+01:00 Hao Sun <[hidden email]>:

I am running Flink 1.3.2 in my local docker environment.

I see this error, not sure how to find the root cause.

I am confused by this error message, why JM is trying to connect to JM from one random port to the RPC port: 6123?

====

2018-01-08 05:38:03,294 ERROR akka.remote.EndpointWriter - AssociationError [akka.tcp://flink@fps-flink-jobmanager:6123] <- [akka.tcp://flink@fps-flink-jobmanager:45652]: Error [Shut down address: akka.tcp://flink@fps-flink-jobmanager:45652] [
akka.remote.ShutDownAssociation: Shut down address: akka.tcp://flink@fps-flink-jobmanager:45652
Caused by: akka.remote.transport.Transport$InvalidAssociationException: The remote system terminated the association because it is shutting down.
]
====

Here you can find the full log

https://gist.github.com/zenhao/c8d13cce8601e321dd706a2ac53f5032