Increased Size of Incremental Checkpoint

[burgesschen]

Hi guys,
I enabled incremental flink checkpoint for my flink job. I had the job read
messages at a stable rate. For each message, the flink job store something
in the keyed state. My question is: For every minute, the increased state
size is the same, shouldn't the incremental checkpoint size remain
relatively constant also? How come it is increasing as shown in the picture?
Thank you!


<http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/file/t969/incremental_check_point.png>



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[Yun Tang]

+ user mail list

---

From: Yun Tang <[hidden email]>
Sent: Thursday, September 6, 2018 14:36
To: burgesschen
Subject: Re: Increased Size of Incremental Checkpoint

 

Hi

I think the "checkpoint size" metrics showed in your graph means the total checkpoint size of each time. The incremental checkpoint, take rocksDB state-backend as example, means during each checkpoint's async-snapshot process, only the newly generated sst files would be uploaded to HDFS, which reducing the pressure to network. Since your job continues to read message at a stable rate, the total checkpoint size would continue to increase.

Best

Yun

---

From: burgesschen <[hidden email]>
Sent: Thursday, September 6, 2018 4:29
To: [hidden email]
Subject: Increased Size of Incremental Checkpoint

 

Hi guys,
I enabled incremental flink checkpoint for my flink job. I had the job read
messages at a stable rate. For each message, the flink job store something
in the keyed state. My question is: For every minute, the increased state
size is the same, shouldn't the incremental checkpoint size remain
relatively constant also? How come it is increasing as shown in the picture?
Thank you!


<http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/file/t969/incremental_check_point.png>



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[Stefan Richter]

Hi,

you should expect that the size can vary for some checkpoints, even if the change rate is constant. Some checkpoints will upload compacted replacements for previous checkpoints to prevent that the checkpoint history will grow without bounds. Whenever that
happens, the checkpoint does some „extra work“ by re-uploading compacted/merged versions of previous deltas.

Best,
Stefan

> Am 05.09.2018 um 22:29 schrieb burgesschen <[hidden email]>:
>
> Hi guys,
> I enabled incremental flink checkpoint for my flink job. I had the job read
> messages at a stable rate. For each message, the flink job store something
> in the keyed state. My question is: For every minute, the increased state
> size is the same, shouldn't the incremental checkpoint size remain
> relatively constant also? How come it is increasing as shown in the picture?
> Thank you!
>
>
> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/file/t969/incremental_check_point.png>
>
>
>
> --
> Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/