State Serializer/Deserializer between savepoints

[Daniel Santos]

Hello,

Is it possible to change the object that is being serialized or
deserialized?

Let's say we have something like the following :

stream

     .window(<SomeWindow>)

     .fold(InitialValueA)(FoldFunction)


Now, InitialValueA is a case class A(n1 : Int).

Restoring the same job from a savepoint but with a slightly diferent
initial value - InitialValueA case class A(n1 : Int, n2 : Int).

Will result in an error.

Is it possible to change the initial value, between savepoints ?

Maybe implementing a different/custom serializer for classA ?


Best Regards,

Daniel Santos

[rmetzger0]

Hi Daniel,

This is currently a limitation in Flink's savepoints. You can not change the serialization schema of the state between savepoints.

In Flink 1.2 there might be the first building blocks available for using serializers aware of savepoints. 

Exposing this feature to the API will probably take some more time.

Regards,

Robert

On Wed, Nov 23, 2016 at 7:16 PM, Daniel Santos <[hidden email]> wrote:

Hello,

Is it possible to change the object that is being serialized or deserialized?

Let's say we have something like the following :

stream

    .window(<SomeWindow>)

    .fold(InitialValueA)(FoldFunction)


Now, InitialValueA is a case class A(n1 : Int).

Restoring the same job from a savepoint but with a slightly diferent initial value - InitialValueA case class A(n1 : Int, n2 : Int).

Will result in an error.

Is it possible to change the initial value, between savepoints ?

Maybe implementing a different/custom serializer for classA ?


Best Regards,

Daniel Santos