How to recovery from last count when using CUMULATE window after restart flink-sql job?

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

How to recovery from last count when using CUMULATE window after restart flink-sql job?

1095193290@qq.com
Hi
   I have tried cumalate window function in Flink-1.13 sql to accumulate data from Kafka. When I restart a cumulate window sql job,  last count state is not considered and the count state accumulates from 1. Any solutions can help recovery from last count state when restarting Flink-sql job?
Thank you 

Reply | Threaded
Open this post in threaded view
|

Re: How to recovery from last count when using CUMULATE window after restart flink-sql job?

Kurt Young
Hi, please use user mailing list only to discuss these issues. 

Best,
Kurt


On Sat, May 8, 2021 at 1:05 PM [hidden email] <[hidden email]> wrote:
Hi
   I have tried cumalate window function in Flink-1.13 sql to accumulate data from Kafka. When I restart a cumulate window sql job,  last count state is not considered and the count state accumulates from 1. Any solutions can help recovery from last count state when restarting Flink-sql job?
Thank you 

Reply | Threaded
Open this post in threaded view
|

Re: How to recovery from last count when using CUMULATE window after restart flink-sql job?

Jark Wu-3
Hi,

When restarting a Flink job, Flink will start the job with an empty state, because this is a new job. 
This is not a special for CUMULATE window, but for all Flink jobs. 
If you want to restore a Flink job from a state/savepoint, you have to specify the savepoint path, see [1].

Best,
Jark



On Sat, 8 May 2021 at 13:30, Kurt Young <[hidden email]> wrote:
Hi, please use user mailing list only to discuss these issues. 

Best,
Kurt


On Sat, May 8, 2021 at 1:05 PM [hidden email] <[hidden email]> wrote:
Hi
   I have tried cumalate window function in Flink-1.13 sql to accumulate data from Kafka. When I restart a cumulate window sql job,  last count state is not considered and the count state accumulates from 1. Any solutions can help recovery from last count state when restarting Flink-sql job?
Thank you