java.lang.IllegalStateException: The RPC connection is already closed

[Manish G]

Hi,

I have set up flink and kafka locally. When I start my flink program(configured ot read messages from kafka topic), I get error as:

2020-05-04 18:17:58.035  INFO 23516 --- [lt-dispatcher-2] o.a.f.r.taskexecutor.JobLeaderService    : Successful registration at job manager akka://flink/user/jobmanager_1 for job 4f1932f75aafb97028fdbf8cd165ee9d.
2020-05-04 18:17:58.035  INFO 23516 --- [lt-dispatcher-4] o.a.f.r.taskexecutor.JobLeaderService    : Successful registration at job manager akka://flink/user/jobmanager_1 for job 4f1932f75aafb97028fdbf8cd165ee9d.
2020-05-04 18:17:58.035  INFO 23516 --- [lt-dispatcher-4] o.a.f.runtime.taskexecutor.TaskExecutor  : Establish JobManager connection for job 4f1932f75aafb97028fdbf8cd165ee9d.
2020-05-04 18:17:58.035  WARN 23516 --- [lt-dispatcher-5] o.a.f.r.h.n.e.EmbeddedLeaderService      : Error notifying leader listener about new leader

java.lang.IllegalStateException: The RPC connection is already closed
at org.apache.flink.util.Preconditions.checkState(Preconditions.java:195) ~[flink-core-1.7.1.jar:1.7.1]
at org.apache.flink.runtime.registration.RegisteredRpcConnection.start(RegisteredRpcConnection.java:91) ~[flink-runtime_2.11-1.7.1.jar:1.7.1]

What can be the root cause for this?

[Steven Wu]

Manish, might be related to this bug, which is fixed in 1.10.1. 

https://issues.apache.org/jira/browse/FLINK-14316?focusedCommentId=16946580&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16946580

On Mon, May 4, 2020 at 5:52 AM Manish G <[hidden email]> wrote:

Hi,

I have set up flink and kafka locally. When I start my flink program(configured ot read messages from kafka topic), I get error as:

2020-05-04 18:17:58.035  INFO 23516 --- [lt-dispatcher-2] o.a.f.r.taskexecutor.JobLeaderService    : Successful registration at job manager akka://flink/user/jobmanager_1 for job 4f1932f75aafb97028fdbf8cd165ee9d.
2020-05-04 18:17:58.035  INFO 23516 --- [lt-dispatcher-4] o.a.f.r.taskexecutor.JobLeaderService    : Successful registration at job manager akka://flink/user/jobmanager_1 for job 4f1932f75aafb97028fdbf8cd165ee9d.
2020-05-04 18:17:58.035  INFO 23516 --- [lt-dispatcher-4] o.a.f.runtime.taskexecutor.TaskExecutor  : Establish JobManager connection for job 4f1932f75aafb97028fdbf8cd165ee9d.
2020-05-04 18:17:58.035  WARN 23516 --- [lt-dispatcher-5] o.a.f.r.h.n.e.EmbeddedLeaderService      : Error notifying leader listener about new leader

java.lang.IllegalStateException: The RPC connection is already closed
at org.apache.flink.util.Preconditions.checkState(Preconditions.java:195) ~[flink-core-1.7.1.jar:1.7.1]
at org.apache.flink.runtime.registration.RegisteredRpcConnection.start(RegisteredRpcConnection.java:91) ~[flink-runtime_2.11-1.7.1.jar:1.7.1]

What can be the root cause for this?

[Manish G]

I found another similar issue:

On Mon, May 4, 2020 at 9:28 PM Steven Wu <[hidden email]> wrote:

Manish, might be related to this bug, which is fixed in 1.10.1. 

https://issues.apache.org/jira/browse/FLINK-14316?focusedCommentId=16946580&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16946580

On Mon, May 4, 2020 at 5:52 AM Manish G <[hidden email]> wrote:

Hi,

I have set up flink and kafka locally. When I start my flink program(configured ot read messages from kafka topic), I get error as:

2020-05-04 18:17:58.035  INFO 23516 --- [lt-dispatcher-2] o.a.f.r.taskexecutor.JobLeaderService    : Successful registration at job manager akka://flink/user/jobmanager_1 for job 4f1932f75aafb97028fdbf8cd165ee9d.
2020-05-04 18:17:58.035  INFO 23516 --- [lt-dispatcher-4] o.a.f.r.taskexecutor.JobLeaderService    : Successful registration at job manager akka://flink/user/jobmanager_1 for job 4f1932f75aafb97028fdbf8cd165ee9d.
2020-05-04 18:17:58.035  INFO 23516 --- [lt-dispatcher-4] o.a.f.runtime.taskexecutor.TaskExecutor  : Establish JobManager connection for job 4f1932f75aafb97028fdbf8cd165ee9d.
2020-05-04 18:17:58.035  WARN 23516 --- [lt-dispatcher-5] o.a.f.r.h.n.e.EmbeddedLeaderService      : Error notifying leader listener about new leader

java.lang.IllegalStateException: The RPC connection is already closed
at org.apache.flink.util.Preconditions.checkState(Preconditions.java:195) ~[flink-core-1.7.1.jar:1.7.1]
at org.apache.flink.runtime.registration.RegisteredRpcConnection.start(RegisteredRpcConnection.java:91) ~[flink-runtime_2.11-1.7.1.jar:1.7.1]

What can be the root cause for this?

[Manish G]

https://issues.apache.org/jira/browse/FLINK-16373

On Mon, May 4, 2020 at 9:37 PM Manish G <[hidden email]> wrote:

I found another similar issue:

On Mon, May 4, 2020 at 9:28 PM Steven Wu <[hidden email]> wrote:

Manish, might be related to this bug, which is fixed in 1.10.1. 

https://issues.apache.org/jira/browse/FLINK-14316?focusedCommentId=16946580&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16946580

On Mon, May 4, 2020 at 5:52 AM Manish G <[hidden email]> wrote:

Hi,

I have set up flink and kafka locally. When I start my flink program(configured ot read messages from kafka topic), I get error as:

2020-05-04 18:17:58.035  INFO 23516 --- [lt-dispatcher-2] o.a.f.r.taskexecutor.JobLeaderService    : Successful registration at job manager akka://flink/user/jobmanager_1 for job 4f1932f75aafb97028fdbf8cd165ee9d.
2020-05-04 18:17:58.035  INFO 23516 --- [lt-dispatcher-4] o.a.f.r.taskexecutor.JobLeaderService    : Successful registration at job manager akka://flink/user/jobmanager_1 for job 4f1932f75aafb97028fdbf8cd165ee9d.
2020-05-04 18:17:58.035  INFO 23516 --- [lt-dispatcher-4] o.a.f.runtime.taskexecutor.TaskExecutor  : Establish JobManager connection for job 4f1932f75aafb97028fdbf8cd165ee9d.
2020-05-04 18:17:58.035  WARN 23516 --- [lt-dispatcher-5] o.a.f.r.h.n.e.EmbeddedLeaderService      : Error notifying leader listener about new leader

java.lang.IllegalStateException: The RPC connection is already closed
at org.apache.flink.util.Preconditions.checkState(Preconditions.java:195) ~[flink-core-1.7.1.jar:1.7.1]
at org.apache.flink.runtime.registration.RegisteredRpcConnection.start(RegisteredRpcConnection.java:91) ~[flink-runtime_2.11-1.7.1.jar:1.7.1]

What can be the root cause for this?

[rmetzger0]

Are you able to reproduce the issue reliably?

If so, it would be nice if you could test if the issue still happens with the 1.10.1 release candidate: https://dist.apache.org/repos/dist/dev/flink/flink-1.10.1-rc3/ 

On Mon, May 4, 2020 at 6:08 PM Manish G <[hidden email]> wrote:

https://issues.apache.org/jira/browse/FLINK-16373

On Mon, May 4, 2020 at 9:37 PM Manish G <[hidden email]> wrote:

I found another similar issue:

On Mon, May 4, 2020 at 9:28 PM Steven Wu <[hidden email]> wrote:

Manish, might be related to this bug, which is fixed in 1.10.1. 

https://issues.apache.org/jira/browse/FLINK-14316?focusedCommentId=16946580&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16946580

On Mon, May 4, 2020 at 5:52 AM Manish G <[hidden email]> wrote:

Hi,

I have set up flink and kafka locally. When I start my flink program(configured ot read messages from kafka topic), I get error as:

2020-05-04 18:17:58.035  INFO 23516 --- [lt-dispatcher-2] o.a.f.r.taskexecutor.JobLeaderService    : Successful registration at job manager akka://flink/user/jobmanager_1 for job 4f1932f75aafb97028fdbf8cd165ee9d.
2020-05-04 18:17:58.035  INFO 23516 --- [lt-dispatcher-4] o.a.f.r.taskexecutor.JobLeaderService    : Successful registration at job manager akka://flink/user/jobmanager_1 for job 4f1932f75aafb97028fdbf8cd165ee9d.
2020-05-04 18:17:58.035  INFO 23516 --- [lt-dispatcher-4] o.a.f.runtime.taskexecutor.TaskExecutor  : Establish JobManager connection for job 4f1932f75aafb97028fdbf8cd165ee9d.
2020-05-04 18:17:58.035  WARN 23516 --- [lt-dispatcher-5] o.a.f.r.h.n.e.EmbeddedLeaderService      : Error notifying leader listener about new leader

java.lang.IllegalStateException: The RPC connection is already closed
at org.apache.flink.util.Preconditions.checkState(Preconditions.java:195) ~[flink-core-1.7.1.jar:1.7.1]
at org.apache.flink.runtime.registration.RegisteredRpcConnection.start(RegisteredRpcConnection.java:91) ~[flink-runtime_2.11-1.7.1.jar:1.7.1]

What can be the root cause for this?