Questions about checkpointAlignmentTime in unaligned checkpoint

[Kai Fu]

Hi team,

I'm observing the metrics reporter still emits checkpointAlignmentTime metric in the unaligned checkpoint setting as shown in the figure below. Is it a meaningful in unaligned checkpoint, since I suppose the alignment operation only happens in aligned checkpoint.  

--

[Yun Gao]

Hi Kai,

Under unaligned checkpoint settings, there are still alignment process. Although

the task could snapshot the state of the operators on received the first barrier and

emit barriers to the following tasks, it still need to wait till all the barriers to be received

before finalize the checkpoint, and during this process, it need to snapshot the buffers

that are skipped by the barrier, and the final snapshot would compose of both the operator

snapshots and the snapshots of the skipped buffers. 

Therefore, the checkpointAlignmentTime metric still exists.

Best,

Yun

------------------Original Mail ------------------

Sender:Kai Fu <[hidden email]>

Send Date:Mon Apr 5 09:18:39 2021

Recipients:user <[hidden email]>

Subject:Questions about checkpointAlignmentTime in unaligned checkpoint

Hi team,

I'm observing the metrics reporter still emits checkpointAlignmentTime metric in the unaligned checkpoint setting as shown in the figure below. Is it a meaningful in unaligned checkpoint, since I suppose the alignment operation only happens in aligned checkpoint.  

--

[Kai Fu]

Hi Yun,

Thank you for the explanation, it clarifies a lot.

-- Best wishes

Kai

On Mon, Apr 5, 2021 at 12:13 PM Yun Gao <[hidden email]> wrote:

Hi Kai,

Under unaligned checkpoint settings, there are still alignment process. Although

the task could snapshot the state of the operators on received the first barrier and

emit barriers to the following tasks, it still need to wait till all the barriers to be received

before finalize the checkpoint, and during this process, it need to snapshot the buffers

that are skipped by the barrier, and the final snapshot would compose of both the operator

snapshots and the snapshots of the skipped buffers. 

Therefore, the checkpointAlignmentTime metric still exists.

Best,

Yun

------------------Original Mail ------------------

Sender:Kai Fu <[hidden email]>

Send Date:Mon Apr 5 09:18:39 2021

Recipients:user <[hidden email]>

Subject:Questions about checkpointAlignmentTime in unaligned checkpoint

Hi team,

I'm observing the metrics reporter still emits checkpointAlignmentTime metric in the unaligned checkpoint setting as shown in the figure below. Is it a meaningful in unaligned checkpoint, since I suppose the alignment operation only happens in aligned checkpoint.  

--

Best regards,

- Kai

--