Posted by
Narendra Joshi on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Question-about-concurrent-checkpoints-tp15755p15764.html
Nico Kruber <
[hidden email]> writes:
> Hi Narendra,
> according to [1], even with asynchronous state snapshots (see [2]), a
> checkpoint is only complete after all sinks have received the barriers and all
> (asynchronous) snapshots have been processed. Since, if the number of
> concurrent checkpoints is 0, no checkpoint barriers will be emitted until the
> previous checkpoint is complete (see [1]), you will not get into the situation
> where two asynchronous snapshots are being taken concurrently.
Does this mean that operators would stop processing streams (because
they received all barriers for a new checkpoint) and wait for
the ongoing asynchronous checkpoint to complete or it means that no
barriers would be injected into sources before checkpoint finishes?
> If you enable concurrent checkpoints and asynchronous snapshots , they will
> process concurrently but on different snapshots of the state, i.e. although
> they are running in parallel, each stores the expected state.
> If you want to know more about the details of how this is done, I can
> recommend Stefan's (cc'd) talk at Flink Forward last week [4]. He may also be
> able to answer in more detail in case I missed something.
Thanks for the reference to the talk! :)
--
Narendra Joshi