Kyro Intermittent Exception for Large Data

[tambunanw]

Hi All, 

We create a downsampling job that will respond to a request from RabbitMQ. However we face different kind of exception intermittently when transferring large data. Is there any setting that we can tune to make Kyro able to serialize and deserialize the large data ? 

I can't find any reference in the documentation about Kyro or Default Java serialization in the latest documentation. Can someone show me the right place to look at ? As i think before fink use Java serialization ?

Thanks a lot. 

ERROR 1

ERROR 2

--

[Stephan Ewen]

Hi Welly!

There are 3 possible reasons for that error that I can think of:

 1) A bug in Kryo, or in one of the serializers used by Kryo

 2) Wrong Kryo registration in Flink

 3) A corruption in the network stream.

If it is a corruption in the network stream, I think we'd see a different problem earlier, so let's check first the other cases.

Can you try the following: For every data type that you use in the program (like "Petrolink.WITSML.Datatypes.ChannelDataItem"), can you explicitly register the type?

Call "env.registerType(Petrolink.WITSML.Datatypes.ChannelDataItem.class)";

Let's see if that solves the problem. Could you also by any chance try the first release candidate for the upcoming 1.0 release? It should have at least the reference tracing disabled, which throws one of the errors.

If we cannot get to the bottom of this via mail, is it possible by any chance that you share the data with us to debug this (privately, [hidden email]) ?

Greetings,

Stephan

On Thu, Feb 18, 2016 at 3:10 AM, Welly Tambunan <[hidden email]> wrote:

Hi All, 

We create a downsampling job that will respond to a request from RabbitMQ. However we face different kind of exception intermittently when transferring large data. Is there any setting that we can tune to make Kyro able to serialize and deserialize the large data ? 

I can't find any reference in the documentation about Kyro or Default Java serialization in the latest documentation. Can someone show me the right place to look at ? As i think before fink use Java serialization ?

Thanks a lot. 

ERROR 1

ERROR 2

--

Welly Tambunan
Triplelands 

http://weltam.wordpress.com

http://www.triplelands.com

[Ken Krugler]

I've seen this type of error when using Kryo with a Cascading scheme I'd created.

In my case it happened when serializing a large object graph, where some of the classes didn't have no-arg constructors.

The general fix was to set an instantiator strategy for Kryo - see:

https://github.com/ScaleUnlimited/cascading.utils/blob/master/src/main/java/com/scaleunlimited/cascading/local/KryoContext.java#L81

Don't know if Flink always sets up to use this (explicitly, or fallback) everywhere.

-- Ken

---

From: Welly Tambunan

Sent: February 17, 2016 6:10:03pm PST

To: [hidden email]

Subject: Kyro Intermittent Exception for Large Data

Hi All, 

We create a downsampling job that will respond to a request from RabbitMQ. However we face different kind of exception intermittently when transferring large data. Is there any setting that we can tune to make Kyro able to serialize and deserialize the large data ? 

I can't find any reference in the documentation about Kyro or Default Java serialization in the latest documentation. Can someone show me the right place to look at ? As i think before fink use Java serialization ?

Thanks a lot. 

ERROR 1

<image.png>

ERROR 2

<image.png>

--

--------------------------

Ken Krugler

[phone]

http://www.scaleunlimited.com

custom big data solutions & training

Hadoop, Cascading, Cassandra & Solr

[Till Rohrmann]

Thanks for the pointer Ken. As far as I know, we’re using the StdInstantiatorStrategy as the fallback instantiator strategy for our Kryo instances.

Cheers,
Till

​

On Fri, Feb 19, 2016 at 12:39 AM, Ken Krugler <[hidden email]> wrote:

I've seen this type of error when using Kryo with a Cascading scheme I'd created.

In my case it happened when serializing a large object graph, where some of the classes didn't have no-arg constructors.

The general fix was to set an instantiator strategy for Kryo - see:

https://github.com/ScaleUnlimited/cascading.utils/blob/master/src/main/java/com/scaleunlimited/cascading/local/KryoContext.java#L81

Don't know if Flink always sets up to use this (explicitly, or fallback) everywhere.

-- Ken

---

From: Welly Tambunan

Sent: February 17, 2016 6:10:03pm PST

To: [hidden email]

Subject: Kyro Intermittent Exception for Large Data

Hi All, 

We create a downsampling job that will respond to a request from RabbitMQ. However we face different kind of exception intermittently when transferring large data. Is there any setting that we can tune to make Kyro able to serialize and deserialize the large data ? 

I can't find any reference in the documentation about Kyro or Default Java serialization in the latest documentation. Can someone show me the right place to look at ? As i think before fink use Java serialization ?

Thanks a lot. 

ERROR 1

<image.png>

ERROR 2

<image.png>

--

Welly Tambunan
Triplelands 

http://weltam.wordpress.com

http://www.triplelands.com

--------------------------

Ken Krugler

<a href="tel:%2B1%20530-210-6378" value="[phone]" target="_blank">[phone]

http://www.scaleunlimited.com

custom big data solutions & training

Hadoop, Cascading, Cassandra & Solr