solution for this kryo serialization problem? (also: update to kryo 4.0.0?)

[Luis Mariano Guerra]

hi, I created an issue at kryo with a simple project that reproduces the problem, but still asking here in case anyone knows a solution:

https://github.com/EsotericSoftware/kryo/issues/462

basically, I get an error trying to serialize a simple tree like structure, the error happens with a 2 node, 1 leaf tree so it's not a recursion problem caused by the size of the data.

anyone knows a way to avoid this error?

also, in the repo I tried with the 3 latest stable versions of kryo (2.24.0, 3.0.3 an 4.0.0) and 4.0.0 seems to fix it, I tried building flink and testing the update and with 3.0.3 it seems to work, but the build fails for 4.0.0, didn't looked much into details since I hadn't much time.

I think at least moving to 3.0.3 would be a good thing in case this is true: https://twitter.com/novoj/status/780693354610880514

[Luis Mariano Guerra]

On Tue, Sep 27, 2016 at 11:22 AM, Luis Mariano Guerra <[hidden email]> wrote:

hi, I created an issue at kryo with a simple project that reproduces the problem, but still asking here in case anyone knows a solution:

https://github.com/EsotericSoftware/kryo/issues/462

I found a solution to the problem, it's an extra if in Kryo that solves the problem and can be applied to 2.24.0, so if they accept a PR flink can move to 2.24.1 and not 4.0.0 :) (still the tweet I mention may be a good reason to move to 3.0.4)
 

basically, I get an error trying to serialize a simple tree like structure, the error happens with a 2 node, 1 leaf tree so it's not a recursion problem caused by the size of the data.

anyone knows a way to avoid this error?

also, in the repo I tried with the 3 latest stable versions of kryo (2.24.0, 3.0.3 an 4.0.0) and 4.0.0 seems to fix it, I tried building flink and testing the update and with 3.0.3 it seems to work, but the build fails for 4.0.0, didn't looked much into details since I hadn't much time.

I think at least moving to 3.0.3 would be a good thing in case this is true: https://twitter.com/novoj/status/780693354610880514

[Timo Walther]

Hi Luis,

there is already an issue for bumping up the kryo version (https://issues.apache.org/jira/browse/FLINK-3154). You could open a PR if you like.

Timo

Am 27/09/16 um 13:10 schrieb Luis Mariano Guerra:

On Tue, Sep 27, 2016 at 11:22 AM, Luis Mariano Guerra <[hidden email]> wrote:

hi, I created an issue at kryo with a simple project that reproduces the problem, but still asking here in case anyone knows a solution:

https://github.com/EsotericSoftware/kryo/issues/462

I found a solution to the problem, it's an extra if in Kryo that solves the problem and can be applied to 2.24.0, so if they accept a PR flink can move to 2.24.1 and not 4.0.0 :) (still the tweet I mention may be a good reason to move to 3.0.4)
 

basically, I get an error trying to serialize a simple tree like structure, the error happens with a 2 node, 1 leaf tree so it's not a recursion problem caused by the size of the data.

anyone knows a way to avoid this error?

also, in the repo I tried with the 3 latest stable versions of kryo (2.24.0, 3.0.3 an 4.0.0) and 4.0.0 seems to fix it, I tried building flink and testing the update and with 3.0.3 it seems to work, but the build fails for 4.0.0, didn't looked much into details since I hadn't much time.

I think at least moving to 3.0.3 would be a good thing in case this is true: https://twitter.com/novoj/status/780693354610880514

[Stephan Ewen]

I would not bump the KRyo version easily - the serialization format changed (that's why they have a new major version), which would render all Flink savepoints and checkpoints incompatible.

On Tue, Sep 27, 2016 at 1:19 PM, Timo Walther <[hidden email]> wrote:

Hi Luis,

there is already an issue for bumping up the kryo version (https://issues.apache.org/jira/browse/FLINK-3154). You could open a PR if you like.

Timo

Am 27/09/16 um 13:10 schrieb Luis Mariano Guerra:

On Tue, Sep 27, 2016 at 11:22 AM, Luis Mariano Guerra <[hidden email]> wrote:

hi, I created an issue at kryo with a simple project that reproduces the problem, but still asking here in case anyone knows a solution:

https://github.com/EsotericSoftware/kryo/issues/462

I found a solution to the problem, it's an extra if in Kryo that solves the problem and can be applied to 2.24.0, so if they accept a PR flink can move to 2.24.1 and not 4.0.0 :) (still the tweet I mention may be a good reason to move to 3.0.4)
 

basically, I get an error trying to serialize a simple tree like structure, the error happens with a 2 node, 1 leaf tree so it's not a recursion problem caused by the size of the data.

anyone knows a way to avoid this error?

also, in the repo I tried with the 3 latest stable versions of kryo (2.24.0, 3.0.3 an 4.0.0) and 4.0.0 seems to fix it, I tried building flink and testing the update and with 3.0.3 it seems to work, but the build fails for 4.0.0, didn't looked much into details since I hadn't much time.

I think at least moving to 3.0.3 would be a good thing in case this is true: https://twitter.com/novoj/status/780693354610880514

-- 
Freundliche Grüße / Kind Regards

Timo Walther 

Follow me: @twalthr
https://www.linkedin.com/in/twalthr

[Timo Walther]

Valid argument. I will add a comment to the issue.


Am 27/09/16 um 14:25 schrieb Stephan Ewen:

I would not bump the KRyo version easily - the serialization format changed (that's why they have a new major version), which would render all Flink savepoints and checkpoints incompatible.

On Tue, Sep 27, 2016 at 1:19 PM, Timo Walther <[hidden email]> wrote:

Hi Luis,

there is already an issue for bumping up the kryo version (https://issues.apache.org/jira/browse/FLINK-3154). You could open a PR if you like.

Timo

Am 27/09/16 um 13:10 schrieb Luis Mariano Guerra:

On Tue, Sep 27, 2016 at 11:22 AM, Luis Mariano Guerra <[hidden email]> wrote:

hi, I created an issue at kryo with a simple project that reproduces the problem, but still asking here in case anyone knows a solution:

https://github.com/EsotericSoftware/kryo/issues/462

I found a solution to the problem, it's an extra if in Kryo that solves the problem and can be applied to 2.24.0, so if they accept a PR flink can move to 2.24.1 and not 4.0.0 :) (still the tweet I mention may be a good reason to move to 3.0.4)
 

basically, I get an error trying to serialize a simple tree like structure, the error happens with a 2 node, 1 leaf tree so it's not a recursion problem caused by the size of the data.

anyone knows a way to avoid this error?

also, in the repo I tried with the 3 latest stable versions of kryo (2.24.0, 3.0.3 an 4.0.0) and 4.0.0 seems to fix it, I tried building flink and testing the update and with 3.0.3 it seems to work, but the build fails for 4.0.0, didn't looked much into details since I hadn't much time.

I think at least moving to 3.0.3 would be a good thing in case this is true: https://twitter.com/novoj/status/780693354610880514

-- 
Freundliche Grüße / Kind Regards

Timo Walther 

Follow me: @twalthr
https://www.linkedin.com/in/twalthr

[Luis Mariano Guerra]

I opened a PR on kryo https://github.com/EsotericSoftware/kryo/pull/463 that fixes 2.24.0, we should push for a 2.24.1 release, in the meantime if you want to try the fix it's in this branch: https://github.com/marianoguerra/kryo/tree/fix-462-for-2.24.0

I made a build of flink 1.1.2 with this fix and my problem went away.

On Tue, Sep 27, 2016 at 1:55 PM, Timo Walther <[hidden email]> wrote:

Valid argument. I will add a comment to the issue.


Am 27/09/16 um 14:25 schrieb Stephan Ewen:

I would not bump the KRyo version easily - the serialization format changed (that's why they have a new major version), which would render all Flink savepoints and checkpoints incompatible.

On Tue, Sep 27, 2016 at 1:19 PM, Timo Walther <[hidden email]> wrote:

Hi Luis,

there is already an issue for bumping up the kryo version (https://issues.apache.org/jira/browse/FLINK-3154). You could open a PR if you like.

Timo

Am 27/09/16 um 13:10 schrieb Luis Mariano Guerra:

On Tue, Sep 27, 2016 at 11:22 AM, Luis Mariano Guerra <[hidden email]> wrote:

hi, I created an issue at kryo with a simple project that reproduces the problem, but still asking here in case anyone knows a solution:

https://github.com/EsotericSoftware/kryo/issues/462

I found a solution to the problem, it's an extra if in Kryo that solves the problem and can be applied to 2.24.0, so if they accept a PR flink can move to 2.24.1 and not 4.0.0 :) (still the tweet I mention may be a good reason to move to 3.0.4)
 

basically, I get an error trying to serialize a simple tree like structure, the error happens with a 2 node, 1 leaf tree so it's not a recursion problem caused by the size of the data.

anyone knows a way to avoid this error?

also, in the repo I tried with the 3 latest stable versions of kryo (2.24.0, 3.0.3 an 4.0.0) and 4.0.0 seems to fix it, I tried building flink and testing the update and with 3.0.3 it seems to work, but the build fails for 4.0.0, didn't looked much into details since I hadn't much time.

I think at least moving to 3.0.3 would be a good thing in case this is true: https://twitter.com/novoj/status/780693354610880514

-- 
Freundliche Grüße / Kind Regards

Timo Walther 

Follow me: @twalthr
https://www.linkedin.com/in/twalthr

-- 
Freundliche Grüße / Kind Regards

Timo Walther 

Follow me: @twalthr
https://www.linkedin.com/in/twalthr