[ANNOUNCE] 1.12.1 may still produce corrupted checkpoints

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[ANNOUNCE] 1.12.1 may still produce corrupted checkpoints

Arvid Heise-4
Dear users,

Unfortunately, the bug in the unaligned checkpoint that we fixed in 1.12.1 still occurs under certain circumstances, such that we recommend to not use unaligned checkpoints in production until 1.12.2. While the normal processing is not affected by this bug, a recovery with corrupted checkpoints will not succeed.

If you have used unaligned checkpoints, you can change back to aligned checkpoint when starting from an uncorrupted unaligned checkpoint. There is no easy way to check if a checkpoint is corrupted or not, however, the rare corruption happens most likely when you have short checkpointing intervals (<1s), high backpressure, and the previous checkpoint was declined for some reason. So to be safe, before switching back, make sure that the last handful of checkpoints all succeeded.

We have already prepared a fix that we will merge into the release branch today, but the discussion on when to release 1.12.2 has not started yet.

Best,

Arvid