Error "Could not load deserializer from the configuration"

[Stefan Bunk]

Hi Flinkers,

I get the following error right after starting/deploying my program on the cluster:

java.lang.RuntimeException: The initialization of the DataSource's outputs caused an error: Could not load deserializer from the configuration.

...

Caused by: java.io.InvalidClassException: org.apache.flink.api.scala.typeutils.CaseClassSerializer; local class incompatible: stream classdesc serialVersionUID = 7795994801161867172, local class serialVersionUID = 1572449491425739294

I see this is an serialization issue, but I do not get why.

My workflow:

I call `mvn package` to build a jar with all dependencies, then execute my program with the remote environment, referencing my newly created *.jar.

What exactly does that tell me, and where do I have to look?

Cheers

Stefan

[Aljoscha Krettek]

Hi,
which version of Flink are you using?

Cheers,
Aljoscha

On Tue, Nov 11, 2014 at 12:09 AM, Stefan Bunk
<[hidden email]> wrote:

> Hi Flinkers,
>
> I get the following error right after starting/deploying my program on the
> cluster:
> java.lang.RuntimeException: The initialization of the DataSource's outputs
> caused an error: Could not load deserializer from the configuration.
> ...
> Caused by: java.io.InvalidClassException:
> org.apache.flink.api.scala.typeutils.CaseClassSerializer; local class
> incompatible: stream classdesc serialVersionUID = 7795994801161867172, local
> class serialVersionUID = 1572449491425739294
>
> I see this is an serialization issue, but I do not get why.
> My workflow:
> I call `mvn package` to build a jar with all dependencies, then execute my
> program with the remote environment, referencing my newly created *.jar.
>
> What exactly does that tell me, and where do I have to look?
>
> Cheers
> Stefan

[Stefan Bunk]

Hi,

I am using Flink 0.7 both in my program and on the cluster.

Cheers

Stefan

On Tue, Nov 11, 2014 at 7:48 AM, Aljoscha Krettek <[hidden email]> wrote:

Hi,
which version of Flink are you using?

Cheers,
Aljoscha

On Tue, Nov 11, 2014 at 12:09 AM, Stefan Bunk
<[hidden email]> wrote:
> Hi Flinkers,
>
> I get the following error right after starting/deploying my program on the
> cluster:
> java.lang.RuntimeException: The initialization of the DataSource's outputs
> caused an error: Could not load deserializer from the configuration.
> ...
> Caused by: java.io.InvalidClassException:
> org.apache.flink.api.scala.typeutils.CaseClassSerializer; local class
> incompatible: stream classdesc serialVersionUID = 7795994801161867172, local
> class serialVersionUID = 1572449491425739294
>
> I see this is an serialization issue, but I do not get why.
> My workflow:
> I call `mvn package` to build a jar with all dependencies, then execute my
> program with the remote environment, referencing my newly created *.jar.
>
> What exactly does that tell me, and where do I have to look?
>
> Cheers
> Stefan

[Stefan Bunk]

Hi,

this turned out to be a version conflict. The cluster was running on the latest 0.7 snapshot, and my code ran on the 0.7 release. Even when I used the 0.7 snapshot in my code again, it did not work.

We updated the cluster to the release now as well, and it works again.

For the future: do they need to be at the exact same version (i.e. commit) or only when there are serious changes in-between?

Thanks for pointing me in the right direction.

On Tue, Nov 11, 2014 at 11:17 AM, Stefan Bunk <[hidden email]> wrote:

Hi,

I am using Flink 0.7 both in my program and on the cluster.

Cheers

Stefan

On Tue, Nov 11, 2014 at 7:48 AM, Aljoscha Krettek <[hidden email]> wrote:

Hi,
which version of Flink are you using?

Cheers,
Aljoscha

On Tue, Nov 11, 2014 at 12:09 AM, Stefan Bunk
<[hidden email]> wrote:
> Hi Flinkers,
>
> I get the following error right after starting/deploying my program on the
> cluster:
> java.lang.RuntimeException: The initialization of the DataSource's outputs
> caused an error: Could not load deserializer from the configuration.
> ...
> Caused by: java.io.InvalidClassException:
> org.apache.flink.api.scala.typeutils.CaseClassSerializer; local class
> incompatible: stream classdesc serialVersionUID = 7795994801161867172, local
> class serialVersionUID = 1572449491425739294
>
> I see this is an serialization issue, but I do not get why.
> My workflow:
> I call `mvn package` to build a jar with all dependencies, then execute my
> program with the remote environment, referencing my newly created *.jar.
>
> What exactly does that tell me, and where do I have to look?
>
> Cheers
> Stefan

[Stephan Ewen]

I think this is because there was no static serialVersionUID added. In that case, Java computes one by itself as a checksum.

It makes sense to add those to all serializable classes.

On Tue, Nov 11, 2014 at 2:02 PM, Stefan Bunk <[hidden email]> wrote:

Hi,

this turned out to be a version conflict. The cluster was running on the latest 0.7 snapshot, and my code ran on the 0.7 release. Even when I used the 0.7 snapshot in my code again, it did not work.

We updated the cluster to the release now as well, and it works again.

For the future: do they need to be at the exact same version (i.e. commit) or only when there are serious changes in-between?

Thanks for pointing me in the right direction.

On Tue, Nov 11, 2014 at 11:17 AM, Stefan Bunk <[hidden email]> wrote:

Hi,

I am using Flink 0.7 both in my program and on the cluster.

Cheers

Stefan

On Tue, Nov 11, 2014 at 7:48 AM, Aljoscha Krettek <[hidden email]> wrote:

Hi,
which version of Flink are you using?

Cheers,
Aljoscha

On Tue, Nov 11, 2014 at 12:09 AM, Stefan Bunk
<[hidden email]> wrote:
> Hi Flinkers,
>
> I get the following error right after starting/deploying my program on the
> cluster:
> java.lang.RuntimeException: The initialization of the DataSource's outputs
> caused an error: Could not load deserializer from the configuration.
> ...
> Caused by: java.io.InvalidClassException:
> org.apache.flink.api.scala.typeutils.CaseClassSerializer; local class
> incompatible: stream classdesc serialVersionUID = 7795994801161867172, local
> class serialVersionUID = 1572449491425739294
>
> I see this is an serialization issue, but I do not get why.
> My workflow:
> I call `mvn package` to build a jar with all dependencies, then execute my
> program with the remote environment, referencing my newly created *.jar.
>
> What exactly does that tell me, and where do I have to look?
>
> Cheers
> Stefan