Hi, When running our job, we’re seeing sporadic instances of when we have KryoExceptions. I’m new to this area of Flink so I’m not exactly too sure what I could look out for. From my understanding, Kryo is the default serializer for generic types, and whilst there is a potential performance penalty with using Kryo, it should be able to serialize / deserialize all objects without fail? Another point is that our object is mutable through as it runs through the different operators, could periodic checkpointing be a cause of the below issues? We are currently running Flink 1.7.1 11:02:43,075 INFO org.apache.flink.runtime.taskmanager.Task - Window(ProcessingTimeSessionWindows(10000), ProcessingTimeTrigger, CoGroupWindowFunction) -> Flat Map -> Sink: Unnamed (1/1) (a83e88eaf06490de c8326e4d9bd0ed26) switched from RUNNING to FAILED. TimerException{com.esotericsoftware.kryo.KryoException: java.lang.ArrayIndexOutOfBoundsException: 1024 Serialization trace: payload (com.celertech.analytics.bo.AnalyticsDataJsonMessage)} at org.apache.flink.streaming.runtime.tasks.SystemProcessingTimeService$TriggerTask.run(SystemProcessingTimeService.java:288) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: com.esotericsoftware.kryo.KryoException: java.lang.ArrayIndexOutOfBoundsException: 1024 Serialization trace: payload (com.celertech.analytics.bo.AnalyticsDataJsonMessage) at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:82) at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:495) at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:505) at org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.copy(KryoSerializer.java:224) at org.apache.flink.streaming.api.datastream.CoGroupedStreams$UnionSerializer.copy(CoGroupedStreams.java:538) at org.apache.flink.streaming.api.datastream.CoGroupedStreams$UnionSerializer.copy(CoGroupedStreams.java:507) at org.apache.flink.api.common.typeutils.base.ListSerializer.copy(ListSerializer.java:99) at org.apache.flink.api.common.typeutils.base.ListSerializer.copy(ListSerializer.java:42) at org.apache.flink.runtime.state.heap.CopyOnWriteStateTable.get(CopyOnWriteStateTable.java:287) at org.apache.flink.runtime.state.heap.CopyOnWriteStateTable.get(CopyOnWriteStateTable.java:311) at org.apache.flink.runtime.state.heap.AbstractHeapAppendingState.getInternal(AbstractHeapAppendingState.java:57) at org.apache.flink.runtime.state.heap.HeapListState.get(HeapListState.java:85) at org.apache.flink.runtime.state.heap.HeapListState.get(HeapListState.java:43) at org.apache.flink.streaming.runtime.operators.windowing.WindowOperator.onProcessingTime(WindowOperator.java:498) at org.apache.flink.streaming.api.operators.InternalTimerServiceImpl.onProcessingTime(InternalTimerServiceImpl.java:235) at org.apache.flink.streaming.runtime.tasks.SystemProcessingTimeService$TriggerTask.run(SystemProcessingTimeService.java:285) ... 7 more Caused by: java.lang.ArrayIndexOutOfBoundsException: 1024 at com.esotericsoftware.kryo.util.IdentityObjectIntMap.getStash(IdentityObjectIntMap.java:256) at com.esotericsoftware.kryo.util.IdentityObjectIntMap.get(IdentityObjectIntMap.java:247) at com.esotericsoftware.kryo.util.MapReferenceResolver.getWrittenId(MapReferenceResolver.java:28) at com.esotericsoftware.kryo.Kryo.writeReferenceOrNull(Kryo.java:619) at com.esotericsoftware.kryo.Kryo.writeObjectOrNull(Kryo.java:564) at com.esotericsoftware.kryo.serializers.MapSerializer.write(MapSerializer.java:84) at com.esotericsoftware.kryo.serializers.MapSerializer.write(MapSerializer.java:21) at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:523) at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:61) ... 22 more Regards, Scott This message, including any attachments, may include private, privileged and confidential information and is intended only for the personal and confidential use of the intended recipient(s). If the reader of this message is not an intended recipient, you are hereby notified that any review, use, dissemination, distribution, printing or copying of this message or its contents is strictly prohibited and may be unlawful. If you are not an intended recipient or have received this communication in error, please immediately notify the sender by telephone and/or a reply email and permanently delete the original message, including any attachments, without making a copy.
|
Free forum by Nabble | Edit this page |