Flink Batch mode checkpointing

[Soheil Pourbafrani]

Hi,

While in Streaming mode I'm using the Flink checkpointing and restart strategy, I could not find any checkpointing or restart strategy for Batch mode! Does Flink have any support for that? Actually I'm gonna read some huge text files and I need the application to be at least restarted on any failure.

[Yun Tang]

Hi

Current Batch API does not really relay on checkpoint mechanism and not support for checkpoints. In the long term, Flink targets for the unification of streaming and batch [1]. Once this is completed, we can enable some checkpoint future when dealing batch cases.

[1] https://flink.apache.org/roadmap.html#batch-and-streaming-unification

Best

Yun Tang

---

From: Soheil Pourbafrani <[hidden email]>
Sent: Friday, January 17, 2020 1:46
To: user <[hidden email]>
Subject: Flink Batch mode checkpointing

 

Hi,

While in Streaming mode I'm using the Flink checkpointing and restart strategy, I could not find any checkpointing or restart strategy for Batch mode! Does Flink have any support for that? Actually I'm gonna read some huge text files and I need the application to be at least restarted on any failure.

[Soheil Pourbafrani]

Thanks a lot. 

On Thu, Jan 16, 2020 at 8:12 PM Yun Tang <[hidden email]> wrote:

Hi

Current Batch API does not really relay on checkpoint mechanism and not support for checkpoints. In the long term, Flink targets for the unification of streaming and batch [1]. Once this is completed, we can enable some checkpoint future when dealing batch cases.

[1] https://flink.apache.org/roadmap.html#batch-and-streaming-unification

Best

Yun Tang

---

From: Soheil Pourbafrani <[hidden email]>
Sent: Friday, January 17, 2020 1:46
To: user <[hidden email]>
Subject: Flink Batch mode checkpointing

 

Hi,

While in Streaming mode I'm using the Flink checkpointing and restart strategy, I could not find any checkpointing or restart strategy for Batch mode! Does Flink have any support for that? Actually I'm gonna read some huge text files and I need the application to be at least restarted on any failure.