IndexOutOfBoundsException on deserialization after updating to 1.6.1

[Bruno Aranda]

Hi,

We are trying to update from 1.3.2 to 1.6.1, but one of our jobs keeps throwing an exception during deserialization:

java.lang.IndexOutOfBoundsException: Index: 0, Size: 0

    at java.util.ArrayList.rangeCheck(ArrayList.java:657)

    at java.util.ArrayList.get(ArrayList.java:433)

    at com.esotericsoftware.kryo.util.MapReferenceResolver.getReadObject(MapReferenceResolver.java:42)

    at com.esotericsoftware.kryo.Kryo.readReferenceOrNull(Kryo.java:805)

    at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:655)

    at org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.copy(KryoSerializer.java:231)

    at org.apache.flink.api.scala.typeutils.OptionSerializer.copy(OptionSerializer.scala:51)

    at org.apache.flink.api.scala.typeutils.OptionSerializer.copy(OptionSerializer.scala:29)

    at org.apache.flink.api.scala.typeutils.CaseClassSerializer.copy(CaseClassSerializer.scala:101)

    at org.apache.flink.api.scala.typeutils.CaseClassSerializer.copy(CaseClassSerializer.scala:32)

    at org.apache.flink.api.scala.typeutils.TraversableSerializer$$anonfun$copy$1.apply(TraversableSerializer.scala:69)

    at org.apache.flink.api.scala.typeutils.TraversableSerializer$$anonfun$copy$1.apply(TraversableSerializer.scala:69)

    at scala.collection.immutable.List.foreach(List.scala:392)

    at org.apache.flink.api.scala.typeutils.TraversableSerializer.copy(TraversableSerializer.scala:69)

    at org.apache.flink.api.scala.typeutils.TraversableSerializer.copy(TraversableSerializer.scala:33)

    at org.apache.flink.api.scala.typeutils.CaseClassSerializer.copy(CaseClassSerializer.scala:101)

    at org.apache.flink.api.scala.typeutils.CaseClassSerializer.copy(CaseClassSerializer.scala:32)

    at org.apache.flink.runtime.state.heap.CopyOnWriteStateTable.get(CopyOnWriteStateTable.java:287)

    at org.apache.flink.runtime.state.heap.CopyOnWriteStateTable.get(CopyOnWriteStateTable.java:311)

    at org.apache.flink.runtime.state.heap.HeapValueState.value(HeapValueState.java:73)

    at org.apache.flink.streaming.api.scala.function.StatefulFunction$class.applyWithState(StatefulFunction.scala:41)

...

We haven't changed the code for the job (apart from updating to 1.6.1), so we are not sure what may have changed. This is caused in a RichMapFunction that extends a StatefulFunction. The state is a case class, and we create its state serializer with the following code:

override protected lazy val stateSerializer: TypeSerializer[ClipinState] =
  api.scala.createTypeInformation[MyCaseClass].createSerializer(getRuntimeContext.getExecutionConfig)

Any clues on what may be going on or where to look further? This was not an issue on 1.3.2...

Thanks!

Bruno

[aitozi]

Hi,Bruno Aranda

Could you provide an complete example to reproduce the exception?

Thanks,
Aitozi



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[elmosca]

Hi,

Thanks for your reply. We are still trying to isolate it, because this job was using a more complex state. I think it is caused by a case class that has an Option[MyOtherClass], and MyOtherClass is an enumerator, implemented using the enumeratum library. I have changed that option to be just a Option[Boolean], and the failure seems not to happen anymore.

We may continue with the Boolean for now, I guess.... though this was not a problem in an earlier Flink version, possible Kryo change?

Cheers,

Bruno

On Wed, 17 Oct 2018 at 15:40 aitozi <[hidden email]> wrote:

Hi,Bruno Aranda

Could you provide an complete example to reproduce the exception?

Thanks,
Aitozi



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/