Does Flink SQL support stop with savepoint draining?

[Yuval Itzchakov]

Hi,

I tried running a stop with savepoint on my Flink job, which includes numerous Flink SQL streams. While running the command I used `-d` to drain all streams with MAX_WATERMARK. 

Looking at the Flink UI, all sources successfully finished, yet all Flink SQL streams were in a Running state, and none of them transitioned although none of them were in a state of backpressure at all. I waited around 10 minutes, yet nothing seemed to change.

My question is, do Flink SQL streams support draining the streams before stopping?

--

[Jingsong Li]

Hi Yuval:

I think yes.

You can take a look to FLIP-34 [1].

It is already finished in 1.9.

And document in [2]

[1] https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=103090212

[2] https://ci.apache.org/projects/flink/flink-docs-master/ops/cli.html#stop

Best,

Jingsong Lee

On Tue, Dec 31, 2019 at 6:03 PM Yuval Itzchakov <[hidden email]> wrote:

Hi,

I tried running a stop with savepoint on my Flink job, which includes numerous Flink SQL streams. While running the command I used `-d` to drain all streams with MAX_WATERMARK. 

Looking at the Flink UI, all sources successfully finished, yet all Flink SQL streams were in a Running state, and none of them transitioned although none of them were in a state of backpressure at all. I waited around 10 minutes, yet nothing seemed to change.

My question is, do Flink SQL streams support draining the streams before stopping?

--

Best Regards,
Yuval Itzchakov.

--