Iterate Operator Checkpoint Failure

[Lu Niu]

Hi, Flink Users

When we migrate from flink 1.9.1 to flink 1.11, we notice job will always fail on checkpoint if job uses Iterator Operator, no matter we use unaligned checkpoint or not. Those jobs don't have checkpoint issues in 1.9. Is this a known issue? Thank you!

Best

Lu

[Fabian Paul]

Hi Lu,

Can you provide some more detailed logs of what happened during the checkpointing phase? If it is possible please enable debug logs enabled.

It would be also great know whether you have implemented your own Iterator Operator or what kind of Flink program you are trying to execute.

Best,

Fabian

CC user-list

On 15. Apr 2021, at 22:34, Lu Niu <[hidden email]> wrote:

Hi, Flink Users

When we migrate from flink 1.9.1 to flink 1.11, we notice job will always fail on checkpoint if job uses Iterator Operator, no matter we use unaligned checkpoint or not. Those jobs don't have checkpoint issues in 1.9. Is this a known issue? Thank you!

Best

Lu

[Lu Niu]

Hi, Fabian

Thanks for replying. I created this ticket. It contains how to reproduce it using code in flink-example package: https://issues.apache.org/jira/browse/FLINK-22326

Best

Lu 

On Fri, Apr 16, 2021 at 1:25 AM Fabian Paul <[hidden email]> wrote:

Hi Lu,

Can you provide some more detailed logs of what happened during the checkpointing phase? If it is possible please enable debug logs enabled.

It would be also great know whether you have implemented your own Iterator Operator or what kind of Flink program you are trying to execute.

Best,

Fabian

CC user-list

On 15. Apr 2021, at 22:34, Lu Niu <[hidden email]> wrote:

Hi, Flink Users

When we migrate from flink 1.9.1 to flink 1.11, we notice job will always fail on checkpoint if job uses Iterator Operator, no matter we use unaligned checkpoint or not. Those jobs don't have checkpoint issues in 1.9. Is this a known issue? Thank you!

Best

Lu