Discarded messages on filter and checkpoint ack

[Francis Aranda]

Hello,

Working with flink checkpointing i see the messages discarded by a filter function are no acked for the checkpoint ack.

How can i auto confirm those discarded messages?   

The ack notification is trigged by a sink ? 

Thanks in advance!

[Stephan Ewen]

Hi!

There are no acks for individual messages in Flink. All messages that an operator receives between two checkpoint barriers fail or succeed together.

That's why dropped messages in filters need no dedicated acks. If the next checkpoint barrier passes through the whole topology, the set of messages from before that checkpoint is "acked" as a whole.

Greetings,

Stephan

On Tue, Jan 19, 2016 at 1:42 PM, Don Frascuchon <[hidden email]> wrote:

Hello,

Working with flink checkpointing i see the messages discarded by a filter function are no acked for the checkpoint ack.

How can i auto confirm those discarded messages?   

The ack notification is trigged by a sink ? 

Thanks in advance!

[Francis Aranda]

Yes...

In fact, my function filter was wrong. It's working well now

Thanks for your response Stephan!

El mar., 19 ene. 2016 a las 14:25, Stephan Ewen (<[hidden email]>) escribió:

Hi!

There are no acks for individual messages in Flink. All messages that an operator receives between two checkpoint barriers fail or succeed together.

That's why dropped messages in filters need no dedicated acks. If the next checkpoint barrier passes through the whole topology, the set of messages from before that checkpoint is "acked" as a whole.

Greetings,

Stephan

On Tue, Jan 19, 2016 at 1:42 PM, Don Frascuchon <[hidden email]> wrote:

Hello,

Working with flink checkpointing i see the messages discarded by a filter function are no acked for the checkpoint ack.

How can i auto confirm those discarded messages?   

The ack notification is trigged by a sink ? 

Thanks in advance!