Hi Avi,
Just as some additional explanation. UID of operator is the way
we map state to corresponding operator. This allows loading
savepoint with changed DAG as long as the UIDs stay the same. This
as you said explain why you got the exception when you changed uid
of some of the operators.
Best,
Dawid
On 24/04/2019 07:06, Avi Levi wrote:
Might be useful for someone, Regarding this issue.
it seems that changing the uid of the operator made this mess .
I am trying to upload a new version of the
code but I am getting the exception below. The schema of
the state was not changed for a while . what can be the
reason for that (also attached the log file) ?
2019-04-16 15:14:11.112
[flink-akka.actor.default-dispatcher-1138] ERROR
org.apache.flink.runtime.dispatcher.StandaloneDispatcher
- Failed to submit job
693a02204ef5816f91ea3b135f544a7f.
java.lang.RuntimeException:
org.apache.flink.runtime.client.JobExecutionException:
Could not set up JobManager
at
org.apache.flink.util.function.CheckedSupplier.lambda$unchecked$0(CheckedSupplier.java:36)
at
java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
at
akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
at
akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
at
java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
at
java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
at
java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
at
java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
Caused by:
org.apache.flink.runtime.client.JobExecutionException:
Could not set up JobManager
at
org.apache.flink.runtime.jobmaster.JobManagerRunner.<init>(JobManagerRunner.java:176)
at
org.apache.flink.runtime.dispatcher.Dispatcher$DefaultJobManagerRunnerFactory.createJobManagerRunner(Dispatcher.java:1058)
at
org.apache.flink.runtime.dispatcher.Dispatcher.lambda$createJobManagerRunner$5(Dispatcher.java:308)
at
org.apache.flink.util.function.CheckedSupplier.lambda$unchecked$0(CheckedSupplier.java:34)
... 7
common frames omitted
Caused by: java.lang.IllegalStateException: Failed to
rollback to checkpoint/savepoint
gs://bv-flink-state/dev/state/savepoint-7cbaf2-cccc48f14797.
Cannot map checkpoint/savepoint state for operator
3cfeb06db0484d5556a7de8db2025f09 to the new program,
because the operator is not available in the new
program. If you want to allow to skip this, you can
set the --allowNonRestoredState option on the CLI.
at
org.apache.flink.runtime.checkpoint.Checkpoints.loadAndValidateCheckpoint(Checkpoints.java:205)
at
org.apache.flink.runtime.checkpoint.CheckpointCoordinator.restoreSavepoint(CheckpointCoordinator.java:1103)
at
org.apache.flink.runtime.jobmaster.JobMaster.tryRestoreExecutionGraphFromSavepoint(JobMaster.java:1241)
at
org.apache.flink.runtime.jobmaster.JobMaster.createAndRestoreExecutionGraph(JobMaster.java:1165)
at
org.apache.flink.runtime.jobmaster.JobMaster.<init>(JobMaster.java:296)
at
org.apache.flink.runtime.jobmaster.JobManagerRunner.<init>(JobManagerRunner.java:157)
... 10
common frames omitted
2019-04-16 15:14:11.242
[flink-akka.actor.default-dispatcher-1138] ERROR
org.apache.flink.runtime.webmonitor.handlers.JarRunHandler
- Exception occurred in REST handler:
org.apache.flink.runtime.client.JobSubmissionException:
Failed to submit job.
2019-04-16 15:14:11.947
[flink-akka.actor.default-dispatcher-1155] ERROR
org.apache.flink.runtime.rest.handler.job.JobDetailsHandler
- Exception occurred in REST handler: Job
e43ed04cdedd73f9bb9836e87142afbf not found
Thanks for your help.
Cheers
Avi
Locked
log.txt.zip (1M)