(DEPRECATED) Apache Flink User Mailing List archive.

ArrayIndexOutOfBoundsException

classic Classic list List threaded Threaded
♦
♦
 5 messages Options
Alexander Smirnov
Reply | Threaded
Open this post in threaded view
♦
♦
|

ArrayIndexOutOfBoundsException

Alexander Smirnov
I'm getting an exception on job starting from a savepoint. Why that could happen?

Flink 1.4.2


java.lang.IllegalStateException: Could not initialize operator state backend.
        at org.apache.flink.streaming.api.operators.AbstractStreamOperator.initOperatorState(AbstractStreamOperator.java:301)
        at org.apache.flink.streaming.api.operators.AbstractStreamOperator.initializeState(AbstractStreamOperator.java:248)
        at org.apache.flink.streaming.runtime.tasks.StreamTask.initializeOperators(StreamTask.java:692)
        at org.apache.flink.streaming.runtime.tasks.StreamTask.initializeState(StreamTask.java:679)
        at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:253)
        at org.apache.flink.runtime.taskmanager.Task.run(Task.java:718)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ArrayIndexOutOfBoundsException: -2
        at java.util.ArrayList.elementData(ArrayList.java:418)
        at java.util.ArrayList.get(ArrayList.java:431)
        at com.esotericsoftware.kryo.util.MapReferenceResolver.getReadObject(MapReferenceResolver.java:42)
        at com.esotericsoftware.kryo.Kryo.readReferenceOrNull(Kryo.java:805)
        at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:759)
        at org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.deserialize(KryoSerializer.java:249)
        at org.apache.flink.api.java.typeutils.runtime.TupleSerializer.deserialize(TupleSerializer.java:136)
        at org.apache.flink.api.java.typeutils.runtime.TupleSerializer.deserialize(TupleSerializer.java:30)
        at org.apache.flink.runtime.state.DefaultOperatorStateBackend.deserializeStateValues(DefaultOperatorStateBackend.java:584)
        at org.apache.flink.runtime.state.DefaultOperatorStateBackend.restore(DefaultOperatorStateBackend.java:399)
        at org.apache.flink.streaming.runtime.tasks.StreamTask.createOperatorStateBackend(StreamTask.java:733)
        at org.apache.flink.streaming.api.operators.AbstractStreamOperator.initOperatorState(AbstractStreamOperator.java:299)
        ... 6 more
Stefan Richter
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: ArrayIndexOutOfBoundsException

Stefan Richter
Hi,

this problem looks like https://issues.apache.org/jira/browse/FLINK-8836 which would also match to your Flink version. I suggest to update to 1.4.3 or higher to avoid the issue in the future.

Best,
Stefan

Am 25.09.2018 um 16:37 schrieb Alexander Smirnov <[hidden email]>:

I'm getting an exception on job starting from a savepoint. Why that could happen?

Flink 1.4.2


java.lang.IllegalStateException: Could not initialize operator state backend.
        at org.apache.flink.streaming.api.operators.AbstractStreamOperator.initOperatorState(AbstractStreamOperator.java:301)
        at org.apache.flink.streaming.api.operators.AbstractStreamOperator.initializeState(AbstractStreamOperator.java:248)
        at org.apache.flink.streaming.runtime.tasks.StreamTask.initializeOperators(StreamTask.java:692)
        at org.apache.flink.streaming.runtime.tasks.StreamTask.initializeState(StreamTask.java:679)
        at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:253)
        at org.apache.flink.runtime.taskmanager.Task.run(Task.java:718)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ArrayIndexOutOfBoundsException: -2
        at java.util.ArrayList.elementData(ArrayList.java:418)
        at java.util.ArrayList.get(ArrayList.java:431)
        at com.esotericsoftware.kryo.util.MapReferenceResolver.getReadObject(MapReferenceResolver.java:42)
        at com.esotericsoftware.kryo.Kryo.readReferenceOrNull(Kryo.java:805)
        at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:759)
        at org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.deserialize(KryoSerializer.java:249)
        at org.apache.flink.api.java.typeutils.runtime.TupleSerializer.deserialize(TupleSerializer.java:136)
        at org.apache.flink.api.java.typeutils.runtime.TupleSerializer.deserialize(TupleSerializer.java:30)
        at org.apache.flink.runtime.state.DefaultOperatorStateBackend.deserializeStateValues(DefaultOperatorStateBackend.java:584)
        at org.apache.flink.runtime.state.DefaultOperatorStateBackend.restore(DefaultOperatorStateBackend.java:399)
        at org.apache.flink.streaming.runtime.tasks.StreamTask.createOperatorStateBackend(StreamTask.java:733)
        at org.apache.flink.streaming.api.operators.AbstractStreamOperator.initOperatorState(AbstractStreamOperator.java:299)
        ... 6 more
Alexander Smirnov
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: ArrayIndexOutOfBoundsException

Alexander Smirnov
Thanks Stefan.

is it only Flink runtime should be updated, or the job should be recompiled too?
Is there a workaround to start the job without upgrading Flink?

Alex

On Tue, Sep 25, 2018 at 5:48 PM Stefan Richter <[hidden email]> wrote:
Hi,

this problem looks like https://issues.apache.org/jira/browse/FLINK-8836 which would also match to your Flink version. I suggest to update to 1.4.3 or higher to avoid the issue in the future.

Best,
Stefan


Am 25.09.2018 um 16:37 schrieb Alexander Smirnov <[hidden email]>:

I'm getting an exception on job starting from a savepoint. Why that could happen?

Flink 1.4.2


java.lang.IllegalStateException: Could not initialize operator state backend.
        at org.apache.flink.streaming.api.operators.AbstractStreamOperator.initOperatorState(AbstractStreamOperator.java:301)
        at org.apache.flink.streaming.api.operators.AbstractStreamOperator.initializeState(AbstractStreamOperator.java:248)
        at org.apache.flink.streaming.runtime.tasks.StreamTask.initializeOperators(StreamTask.java:692)
        at org.apache.flink.streaming.runtime.tasks.StreamTask.initializeState(StreamTask.java:679)
        at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:253)
        at org.apache.flink.runtime.taskmanager.Task.run(Task.java:718)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ArrayIndexOutOfBoundsException: -2
        at java.util.ArrayList.elementData(ArrayList.java:418)
        at java.util.ArrayList.get(ArrayList.java:431)
        at com.esotericsoftware.kryo.util.MapReferenceResolver.getReadObject(MapReferenceResolver.java:42)
        at com.esotericsoftware.kryo.Kryo.readReferenceOrNull(Kryo.java:805)
        at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:759)
        at org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.deserialize(KryoSerializer.java:249)
        at org.apache.flink.api.java.typeutils.runtime.TupleSerializer.deserialize(TupleSerializer.java:136)
        at org.apache.flink.api.java.typeutils.runtime.TupleSerializer.deserialize(TupleSerializer.java:30)
        at org.apache.flink.runtime.state.DefaultOperatorStateBackend.deserializeStateValues(DefaultOperatorStateBackend.java:584)
        at org.apache.flink.runtime.state.DefaultOperatorStateBackend.restore(DefaultOperatorStateBackend.java:399)
        at org.apache.flink.streaming.runtime.tasks.StreamTask.createOperatorStateBackend(StreamTask.java:733)
        at org.apache.flink.streaming.api.operators.AbstractStreamOperator.initOperatorState(AbstractStreamOperator.java:299)
        ... 6 more
Stefan Richter
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: ArrayIndexOutOfBoundsException

Stefan Richter
You only need to update the flink jars, the job requires no update. I think you also cannot start from this checkpoint/savepoint after the upgrade because it seems to be corrupted from the bug. You need to us an older point to restart.

Best,
Stefan

Am 25.09.2018 um 16:53 schrieb Alexander Smirnov <[hidden email]>:

Thanks Stefan.

is it only Flink runtime should be updated, or the job should be recompiled too?
Is there a workaround to start the job without upgrading Flink?

Alex

On Tue, Sep 25, 2018 at 5:48 PM Stefan Richter <[hidden email]> wrote:
Hi,

this problem looks like https://issues.apache.org/jira/browse/FLINK-8836 which would also match to your Flink version. I suggest to update to 1.4.3 or higher to avoid the issue in the future.

Best,
Stefan


Am 25.09.2018 um 16:37 schrieb Alexander Smirnov <[hidden email]>:

I'm getting an exception on job starting from a savepoint. Why that could happen?

Flink 1.4.2


java.lang.IllegalStateException: Could not initialize operator state backend.
        at org.apache.flink.streaming.api.operators.AbstractStreamOperator.initOperatorState(AbstractStreamOperator.java:301)
        at org.apache.flink.streaming.api.operators.AbstractStreamOperator.initializeState(AbstractStreamOperator.java:248)
        at org.apache.flink.streaming.runtime.tasks.StreamTask.initializeOperators(StreamTask.java:692)
        at org.apache.flink.streaming.runtime.tasks.StreamTask.initializeState(StreamTask.java:679)
        at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:253)
        at org.apache.flink.runtime.taskmanager.Task.run(Task.java:718)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ArrayIndexOutOfBoundsException: -2
        at java.util.ArrayList.elementData(ArrayList.java:418)
        at java.util.ArrayList.get(ArrayList.java:431)
        at com.esotericsoftware.kryo.util.MapReferenceResolver.getReadObject(MapReferenceResolver.java:42)
        at com.esotericsoftware.kryo.Kryo.readReferenceOrNull(Kryo.java:805)
        at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:759)
        at org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.deserialize(KryoSerializer.java:249)
        at org.apache.flink.api.java.typeutils.runtime.TupleSerializer.deserialize(TupleSerializer.java:136)
        at org.apache.flink.api.java.typeutils.runtime.TupleSerializer.deserialize(TupleSerializer.java:30)
        at org.apache.flink.runtime.state.DefaultOperatorStateBackend.deserializeStateValues(DefaultOperatorStateBackend.java:584)
        at org.apache.flink.runtime.state.DefaultOperatorStateBackend.restore(DefaultOperatorStateBackend.java:399)
        at org.apache.flink.streaming.runtime.tasks.StreamTask.createOperatorStateBackend(StreamTask.java:733)
        at org.apache.flink.streaming.api.operators.AbstractStreamOperator.initOperatorState(AbstractStreamOperator.java:299)
        ... 6 more
Alexander Smirnov
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: ArrayIndexOutOfBoundsException

Alexander Smirnov
Appreciate your help, Stefan! 👍🏻
On Tue, 25 Sep 2018 at 18:19, Stefan Richter <[hidden email]> wrote:
You only need to update the flink jars, the job requires no update. I think you also cannot start from this checkpoint/savepoint after the upgrade because it seems to be corrupted from the bug. You need to us an older point to restart.

Best,
Stefan


Am 25.09.2018 um 16:53 schrieb Alexander Smirnov <[hidden email]>:

Thanks Stefan.

is it only Flink runtime should be updated, or the job should be recompiled too?
Is there a workaround to start the job without upgrading Flink?

Alex

On Tue, Sep 25, 2018 at 5:48 PM Stefan Richter <[hidden email]> wrote:
Hi,

this problem looks like https://issues.apache.org/jira/browse/FLINK-8836 which would also match to your Flink version. I suggest to update to 1.4.3 or higher to avoid the issue in the future.

Best,
Stefan


Am 25.09.2018 um 16:37 schrieb Alexander Smirnov <[hidden email]>:

I'm getting an exception on job starting from a savepoint. Why that could happen?

Flink 1.4.2


java.lang.IllegalStateException: Could not initialize operator state backend.
        at org.apache.flink.streaming.api.operators.AbstractStreamOperator.initOperatorState(AbstractStreamOperator.java:301)
        at org.apache.flink.streaming.api.operators.AbstractStreamOperator.initializeState(AbstractStreamOperator.java:248)
        at org.apache.flink.streaming.runtime.tasks.StreamTask.initializeOperators(StreamTask.java:692)
        at org.apache.flink.streaming.runtime.tasks.StreamTask.initializeState(StreamTask.java:679)
        at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:253)
        at org.apache.flink.runtime.taskmanager.Task.run(Task.java:718)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ArrayIndexOutOfBoundsException: -2
        at java.util.ArrayList.elementData(ArrayList.java:418)
        at java.util.ArrayList.get(ArrayList.java:431)
        at com.esotericsoftware.kryo.util.MapReferenceResolver.getReadObject(MapReferenceResolver.java:42)
        at com.esotericsoftware.kryo.Kryo.readReferenceOrNull(Kryo.java:805)
        at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:759)
        at org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.deserialize(KryoSerializer.java:249)
        at org.apache.flink.api.java.typeutils.runtime.TupleSerializer.deserialize(TupleSerializer.java:136)
        at org.apache.flink.api.java.typeutils.runtime.TupleSerializer.deserialize(TupleSerializer.java:30)
        at org.apache.flink.runtime.state.DefaultOperatorStateBackend.deserializeStateValues(DefaultOperatorStateBackend.java:584)
        at org.apache.flink.runtime.state.DefaultOperatorStateBackend.restore(DefaultOperatorStateBackend.java:399)
        at org.apache.flink.streaming.runtime.tasks.StreamTask.createOperatorStateBackend(StreamTask.java:733)
        at org.apache.flink.streaming.api.operators.AbstractStreamOperator.initOperatorState(AbstractStreamOperator.java:299)
        ... 6 more
Free forum by Nabble Edit this page