Savepointing with Avro Schema change

[Aneesha Kaushal-2]

Hello,

I have a flink job which processes a stream of Event (an avro object) and creates Sessions (another avro object) using Session-Windows.

I am not able to recover my job from save point when I try to make some changes in the schema of Event object or in schema of Sessions.

Is there anyone who has tried this, and figured out a way to restart from save point while changing object schemas?

I have made sure that the changes are backward compatible(for the beginning I am only adding a new field)
I am using Rocks DB as my state backend, and, event time for processing.

Thanks
Aneesha

[Timo Walther]

Hi Aneesha,

as far as I know Avro objects were serialized with Flink's POJO
serializer in the past. This behavior changed in 1.4. @Gordon: do you
have more information how good we support Avro schema evolution today?

Regards,
Timo


Am 03.04.18 um 12:11 schrieb Aneesha Kaushal:

> Hello,
>
> I have a flink job which processes a stream of Event (an avro object) and creates Sessions (another avro object) using Session-Windows.
>
> I am not able to recover my job from save point when I try to make some changes in the schema of Event object or in schema of Sessions.
>
> Is there anyone who has tried this, and figured out a way to restart from save point while changing object schemas?
>
> I have made sure that the changes are backward compatible(for the beginning I am only adding a new field)
> I am using Rocks DB as my state backend, and, event time for processing.
>
> Thanks
> Aneesha