slow checkpoints

[yuvraj singh]

Hi all , 

I am facing one issue , when i have high back pressure my checkpoints start failing . please let me know how to deal with this kind  of situations . 

Thanks 

Yubraj Singh .

Sender notified by Mailtrack 11/15/19, 02:15:52 PM

[Congxian Qiu]

Hi

Currently, checkpoint may be faile in high back pressure scenario, because the barrier alignment can't be done in expected time, you should fix the back pressure problem first. There is a FLIP[1] that wants to fix this issue.

[1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-76%3A+Unaligned+Checkpoints

yuvraj singh <[hidden email]> 于2019年11月15日周五 下午4:47写道：

Hi all , 

I am facing one issue , when i have high back pressure my checkpoints start failing . please let me know how to deal with this kind  of situations . 

Thanks 

Yubraj Singh .

Sender notified by Mailtrack 11/15/19, 02:15:52 PM

[yuvraj singh]

@Congxian , back pressure is due to job failure , some times job can fail and we need to catch up . 

Thanks 

Yubraj Singh 

Sender notified by Mailtrack 11/15/19, 02:53:23 PM

On Fri, Nov 15, 2019 at 2:39 PM Congxian Qiu <[hidden email]> wrote:

Hi

Currently, checkpoint may be faile in high back pressure scenario, because the barrier alignment can't be done in expected time, you should fix the back pressure problem first. There is a FLIP[1] that wants to fix this issue.

[1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-76%3A+Unaligned+Checkpoints

Best,

Congxian

yuvraj singh <[hidden email]> 于2019年11月15日周五 下午4:47写道：

Hi all , 

I am facing one issue , when i have high back pressure my checkpoints start failing . please let me know how to deal with this kind  of situations . 

Thanks 

Yubraj Singh .

Sender notified by Mailtrack 11/15/19, 02:15:52 PM

[vino yang]

Hi Yubraj,

So the frequent job failure is the root reason, you need to fix it. Yes, when too many messages are squashed into the message system. If the messages can not be consumed normally, there would exist catchup consuming which will cause your streaming system more pressure than usual.

Best,

Vino

yuvraj singh <[hidden email]> 于2019年11月15日周五 下午5:25写道：

@Congxian , back pressure is due to job failure , some times job can fail and we need to catch up . 

Thanks 

Yubraj Singh 

Sender notified by Mailtrack 11/15/19, 02:53:23 PM

On Fri, Nov 15, 2019 at 2:39 PM Congxian Qiu <[hidden email]> wrote:

Hi

Currently, checkpoint may be faile in high back pressure scenario, because the barrier alignment can't be done in expected time, you should fix the back pressure problem first. There is a FLIP[1] that wants to fix this issue.

[1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-76%3A+Unaligned+Checkpoints

Best,

Congxian

yuvraj singh <[hidden email]> 于2019年11月15日周五 下午4:47写道：

Hi all , 

I am facing one issue , when i have high back pressure my checkpoints start failing . please let me know how to deal with this kind  of situations . 

Thanks 

Yubraj Singh .

Sender notified by Mailtrack 11/15/19, 02:15:52 PM