Statefun delayed message

[Timothy Bess]

Hi everyone,

I have a question about how delayed messages work, I tried to dig through some docs on it, but not sure it addresses exactly my question. Basically, if I send a delayed message with exactly-once mode on, does Flink need to wait until the delayed message sends to commit Kafka offsets? Otherwise I'm not sure how the message gets delivered. I was under the impression that with Flink you are always reacting to an event, so if I send a delayed message, Kafka commits, and we receive no new incoming events, how does the delayed message actually get sent? Maybe statefun is doing something I didn't realize like publishing some sort of "tick" event for timing things, just curious.

Thanks,

Tim

[rmetzger0]

Hey Tim,

delayed messages are stored in Flink's state while they are waiting to be sent again. Thus they are not blocking any checkpoints (and thus the persisting of Kafka offsets).

If you are restoring from a checkpoint (or savepoint), the pending delayed messages will be reloaded into Flink's state to be triggered.

I hope this helps!

Regards,

Robert

On Tue, Nov 24, 2020 at 10:37 PM Timothy Bess <[hidden email]> wrote:

Hi everyone,

I have a question about how delayed messages work, I tried to dig through some docs on it, but not sure it addresses exactly my question. Basically, if I send a delayed message with exactly-once mode on, does Flink need to wait until the delayed message sends to commit Kafka offsets? Otherwise I'm not sure how the message gets delivered. I was under the impression that with Flink you are always reacting to an event, so if I send a delayed message, Kafka commits, and we receive no new incoming events, how does the delayed message actually get sent? Maybe statefun is doing something I didn't realize like publishing some sort of "tick" event for timing things, just curious.

Thanks,

Tim