Strange behavior of FsStateBackend checkpoint when local executing

[徐涛]

Hi All,
        I use using a FsStateBackend in local executing, I set the DELETE_ON_CANCELLATION of checkpoint. When I click the “stop” button in Intellij IDEA, the log shows that it has been switched CANCELED state, but I check the local file system, the checkpoint directory and file still exists.
        And further more, I rm the checkpoint directory, but when I execute the main function, the flink program run again, it seems that it still can read the old data(the dirty data that can lead to exception in Kafka), but in my understood, because there is no checkpoint now, it by default should read the latest kafka data. So why does it happens?
        Thanks a lot!

Best
Henry

[Till Rohrmann]

Hi Henry,

which version of Flink are you using. If you could us provide with a working example to reproduce the problem, then I'm sure that we can figure out why it is not working as expected.

Cheers,

Till

On Tue, Sep 25, 2018 at 8:44 AM 徐涛 <[hidden email]> wrote:

Hi All,
        I use using a FsStateBackend in local executing, I set the DELETE_ON_CANCELLATION of checkpoint. When I click the “stop” button in Intellij IDEA, the log shows that it has been switched CANCELED state, but I check the local file system, the checkpoint directory and file still exists.
        And further more, I rm the checkpoint directory, but when I execute the main function, the flink program run again, it seems that it still can read the old data(the dirty data that can lead to exception in Kafka), but in my understood, because there is no checkpoint now, it by default should read the latest kafka data. So why does it happens?
        Thanks a lot!

Best
Henry