No output when using event time with multiple Kafka partitions

Posted by Yassin Marzouki on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/No-output-when-using-event-time-with-multiple-Kafka-partitions-tp8132.html

Hi everyone,

I am reading messages from a Kafka topic with 2 partitions and using event time. This is my code:

.assignTimestampsAndWatermarks(new AscendingTimestampExtractor<Request>() {

    @Override

    public long extractAscendingTimestamp(Request req) {

        return req.ts;

    }

})

.windowAll(TumblingEventTimeWindows.of(Time.days(1)))

.apply((TimeWindow window, Iterable<Request> iterable, Collector<String> collector) -> {

    collector.collect("Window: " + window.toString());

    for (Request req : iterable) {

        collector.collect(req.toString());

    }

})

.print()

I could get an output only when setting the kafka source parallelism to 1. I guess that is because messages from multiple partitions arrive out-of-order to the timestamp exctractor according to this thread, correct?

So I replaced the AscendingTimestampExtractor with a BoundedOutOfOrdernessGenerator as in the documentation example (with a higher delay) in order to handle out-of-order events, but I still can't get any output. Why is that?

Best,

Yassine