Re: Fink: KafkaProducer Data Loss

Posted by Piotr Nowojski on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Fink-KafkaProducer-Data-Loss-tp11413p14245.html

Ops, sorry, I forgot that this issue was relevant to FlinkKafkaProducer010 only.

Piotrek

On Jul 13, 2017, at 9:33 AM, Tzu-Li (Gordon) Tai <[hidden email]> wrote:

Hi Ninad & Piotr,

AFAIK, when this issue was reported, Ninad was using 09.

FLINK-6996 only affects Flink Kafka Producer 010, so I don’t think that’s the cause here.

@Ninad

Code to reproduce this would definitely be helpful here, thanks. If you prefer to provide that privately, that would also be fine.

Cheers,

Gordon

On 13 July 2017 at 4:13:07 PM, Piotr Nowojski ([hidden email]) wrote:

Hi,

I’m not sure how relevant is this, but recently I have found and fixed a bug, that in certain conditions was causing data losses for all of the FlinkKafkaProducers in Flink:

https://issues.apache.org/jira/browse/FLINK-6996

Namely on checkpoint “flush” method was not being called. It should be fixed in Flink 1.3.2 and 1.4 releases.

Piotrek

On Jul 12, 2017, at 7:32 PM, ninad <[hidden email]> wrote:

Hey guys, any update on this? If needed I can attach our code.



--
View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Fink-KafkaProducer-Data-Loss-tp11413p14224.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.