Could not retrieve the redirect address - No REST endpoint has been started

[PedroMrChaves]

Hello,

I have a running standalone Flink cluster with 2 task managers and 2 job
manages (one task manager and job manager per machine).
Sometimes, when I restart the cluster I get the following error message:
/
java.util.concurrent.CompletionException:
org.apache.flink.util.FlinkException: No REST endpoint has been started for
the JobManager.
        at
java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292)
        at
java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308)
        at
java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:593)
        at
java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
        at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
        at
java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977)
        at
org.apache.flink.runtime.concurrent.FutureUtils$1.onComplete(FutureUtils.java:442)
        at akka.dispatch.OnComplete.internal(Future.scala:258)
        at akka.dispatch.OnComplete.internal(Future.scala:256)
        at akka.dispatch.japi$CallbackBridge.apply(Future.scala:186)
        at akka.dispatch.japi$CallbackBridge.apply(Future.scala:183)
        at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)
        at
org.apache.flink.runtime.concurrent.Executors$DirectExecutionContext.execute(Executors.java:83)
        at
scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:44)
        at
scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:252)
        at scala.concurrent.Promise$class.complete(Promise.scala:55)
        at
scala.concurrent.impl.Promise$DefaultPromise.complete(Promise.scala:157)
        at scala.concurrent.Future$$anonfun$map$1.apply(Future.scala:237)
        at scala.concurrent.Future$$anonfun$map$1.apply(Future.scala:237)
        at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)
        at
scala.concurrent.BatchingExecutor$Batch$$anonfun$run$1.processBatch$1(BatchingExecutor.scala:63)
        at
scala.concurrent.BatchingExecutor$Batch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:78)
        at
scala.concurrent.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:55)
        at
scala.concurrent.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:55)
        at
scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72)
        at
scala.concurrent.BatchingExecutor$Batch.run(BatchingExecutor.scala:54)
        at
scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:601)
        at
scala.concurrent.BatchingExecutor$class.execute(BatchingExecutor.scala:106)
        at
scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:599)
        at
scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:44)
        at
scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:252)
        at akka.pattern.PromiseActorRef.$bang(AskSupport.scala:534)
        at akka.remote.DefaultMessageDispatcher.dispatch(Endpoint.scala:97)
        at
akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:982)
        at akka.actor.Actor$class.aroundReceive(Actor.scala:502)
        at akka.remote.EndpointActor.aroundReceive(Endpoint.scala:446)
        at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)
        at akka.actor.ActorCell.invoke(ActorCell.scala:495)
        at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
        at akka.dispatch.Mailbox.run(Mailbox.scala:224)
        at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
        at
scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
        at
scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
        at
scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
        at
scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
Caused by: org.apache.flink.util.FlinkException: No REST endpoint has been
started for the JobManager./

which prevents the access to the web interface.

I'm using version 1.4.2

Any idea on what might be causing this?

Regards,
Pedro.





-----
Best Regards,
Pedro Chaves
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[Andrey Zagrebin]

Hi Pedro,

when you restart the cluster, do you keep Web UI open?
and does Web UI work eventually after restart and getting this error?

Cheers,
Andrey

> On 1 Aug 2018, at 11:56, PedroMrChaves <[hidden email]> wrote:
>
> Hello,
>
> I have a running standalone Flink cluster with 2 task managers and 2 job
> manages (one task manager and job manager per machine).
> Sometimes, when I restart the cluster I get the following error message:
> /
> java.util.concurrent.CompletionException:
> org.apache.flink.util.FlinkException: No REST endpoint has been started for
> the JobManager.
>        at
> java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292)
>        at
> java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308)
>        at
> java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:593)
>        at
> java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
>        at
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
>        at
> java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977)
>        at
> org.apache.flink.runtime.concurrent.FutureUtils$1.onComplete(FutureUtils.java:442)
>        at akka.dispatch.OnComplete.internal(Future.scala:258)
>        at akka.dispatch.OnComplete.internal(Future.scala:256)
>        at akka.dispatch.japi$CallbackBridge.apply(Future.scala:186)
>        at akka.dispatch.japi$CallbackBridge.apply(Future.scala:183)
>        at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)
>        at
> org.apache.flink.runtime.concurrent.Executors$DirectExecutionContext.execute(Executors.java:83)
>        at
> scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:44)
>        at
> scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:252)
>        at scala.concurrent.Promise$class.complete(Promise.scala:55)
>        at
> scala.concurrent.impl.Promise$DefaultPromise.complete(Promise.scala:157)
>        at scala.concurrent.Future$$anonfun$map$1.apply(Future.scala:237)
>        at scala.concurrent.Future$$anonfun$map$1.apply(Future.scala:237)
>        at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)
>        at
> scala.concurrent.BatchingExecutor$Batch$$anonfun$run$1.processBatch$1(BatchingExecutor.scala:63)
>        at
> scala.concurrent.BatchingExecutor$Batch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:78)
>        at
> scala.concurrent.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:55)
>        at
> scala.concurrent.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:55)
>        at
> scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72)
>        at
> scala.concurrent.BatchingExecutor$Batch.run(BatchingExecutor.scala:54)
>        at
> scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:601)
>        at
> scala.concurrent.BatchingExecutor$class.execute(BatchingExecutor.scala:106)
>        at
> scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:599)
>        at
> scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:44)
>        at
> scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:252)
>        at akka.pattern.PromiseActorRef.$bang(AskSupport.scala:534)
>        at akka.remote.DefaultMessageDispatcher.dispatch(Endpoint.scala:97)
>        at
> akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:982)
>        at akka.actor.Actor$class.aroundReceive(Actor.scala:502)
>        at akka.remote.EndpointActor.aroundReceive(Endpoint.scala:446)
>        at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)
>        at akka.actor.ActorCell.invoke(ActorCell.scala:495)
>        at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
>        at akka.dispatch.Mailbox.run(Mailbox.scala:224)
>        at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
>        at
> scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
>        at
> scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
>        at
> scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
>        at
> scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
> Caused by: org.apache.flink.util.FlinkException: No REST endpoint has been
> started for the JobManager./
>
> which prevents the access to the web interface.
>
> AM using version 1.4.2
>
> Any idea on what might be causing this?
>
> Regards,
> Pedro.
>
>
>
>
>
> -----
> Best Regards,
> Pedro Chaves
> --
> Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[PedroMrChaves]

Hello,

It happens whether the WEB UI is opened or not and it no longer works.
When this happens I have to restart the job managers.

regards,
Pedro.



-----
Best Regards,
Pedro Chaves
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[vino yang]

Hi Pedro,

It sounds like a bug from Flink itself. 

You can create an issue in JIAR and give enough information, such as logs, completed exceptions, Flink versions, and your usage environment.

Thanks, vino.

2018-08-02 16:45 GMT+08:00 PedroMrChaves <[hidden email]>:

Hello,

It happens whether the WEB UI is opened or not and it no longer works.
When this happens I have to restart the job managers.

regards,

Pedro.



-----
Best Regards,
Pedro Chaves
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[Till Rohrmann]

Hi Pedro,

it looks as if Flink could not start the WebRuntimeMonitor. Could you maybe share the jobmanager.log of the newly started JobManager which has no WebRuntimeMonitor started with us? Maybe there is a port conflict.

Cheers,

Till

On Thu, Aug 2, 2018 at 11:35 AM vino yang <[hidden email]> wrote:

Hi Pedro,

It sounds like a bug from Flink itself. 

You can create an issue in JIAR and give enough information, such as logs, completed exceptions, Flink versions, and your usage environment.

Thanks, vino.

2018-08-02 16:45 GMT+08:00 PedroMrChaves <[hidden email]>:

Hello,

It happens whether the WEB UI is opened or not and it no longer works.
When this happens I have to restart the job managers.

regards,

Pedro.



-----
Best Regards,
Pedro Chaves
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[PedroMrChaves]

Hello,

Thank you for the reply.

The problem sometimes happens when there is a jobmanager failover. I've
attached the jobmanager logs for further debugging.

jobmanager.log

Thank you and Regards,
Pedro Chaves.



-----
Best Regards,
Pedro Chaves
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/