Incremental checkpoint branch

[vishnuviswanath]

Hi,

Can someone point me to the branch where the ongoing work for incremental checkpoint is going on, I would like to try it out even if the work is not complete.

I have a use case where the state size increase about ~1gb every 5 minutes.

Thanks,

Vishnu

[Ted Yu]

I checked https://github.com/apache/flink and FLINK-5053

However, I haven't found such information.

On Fri, Mar 3, 2017 at 1:14 PM, Vishnu Viswanath <[hidden email]> wrote:

Hi,

Can someone point me to the branch where the ongoing work for incremental checkpoint is going on, I would like to try it out even if the work is not complete.

I have a use case where the state size increase about ~1gb every 5 minutes.

Thanks,

Vishnu

[Chen Qin]

Hi Vishnu,

My best gussing is there are lots of customized "incremental checkpointing" done via patch around rocksdb statebackend and rocksdb checkpoints. 

http://rocksdb.org/blog/2015/11/10/use-checkpoints-for-efficient-snapshots.html

Thanks,

Chen

On Fri, Mar 3, 2017 at 1:16 PM, Ted Yu <[hidden email]> wrote:

I checked https://github.com/apache/flink and FLINK-5053

However, I haven't found such information.

On Fri, Mar 3, 2017 at 1:14 PM, Vishnu Viswanath <[hidden email]> wrote:

Hi,

Can someone point me to the branch where the ongoing work for incremental checkpoint is going on, I would like to try it out even if the work is not complete.

I have a use case where the state size increase about ~1gb every 5 minutes.

Thanks,

Vishnu

[Shaoxuan Wang]

Vinshnu,

You can find the latest design discussion for incremental checkpoint in http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Support-Incremental-Checkpointing-in-Flink-td15931.html

 @Stefan Richter and @Xiaogang Shi might be the right person to provide you its on-going status. 

cheers,

Shaoxuan

On Sat, Mar 4, 2017 at 5:14 AM, Vishnu Viswanath <[hidden email]> wrote:

Hi,

Can someone point me to the branch where the ongoing work for incremental checkpoint is going on, I would like to try it out even if the work is not complete.

I have a use case where the state size increase about ~1gb every 5 minutes.

Thanks,

Vishnu