RE: Flink to get historical data from kafka between timespan t1 & t2

Posted by VINAY.RAICHUR on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Flink-to-get-historical-data-from-kafka-between-timespan-t1-t2-tp40479p40511.html

Hi Aljoscha,
Currently in our set-up we are using docker container version of Flink:1.11.2 along with containerized version of Kafka running on our Kubernetes cluster as seen in attached screenshot .png file to this mail of our Vagrant VM.

a) As mentioned by you "KafkaSource" was introduced in Flink 1.12 so, I suppose we have to upgrade to this version of Flink. Can you share the link of the stable Flink image (containerized version) to be used in our set-up keeping in mind we are to use   KafkaSource.

b) Also kindly share details of the corresponding compatible version of "Kafka" (containerized version) to be used for making this work as I understand there is constraint on version of Kafka required to make it work.

Kind regards,
Vinay

-----Original Message-----
From: Aljoscha Krettek <[hidden email]>
Sent: 11 January 2021 16:06
To: [hidden email]
Cc: RAICHUR, VINAY <[hidden email]>
Subject: Re: Flink to get historical data from kafka between timespan t1 & t2

On 2021/01/08 16:55, [hidden email] wrote:
>Could you also attach the code snippet for KafkaSource`, `KafkaSourceBuilder`, and `OffsetInitializers` that you were referring to in your previous reply, for my reference please to make it more clearer for me.

Ah sorry, but this I was referring to the Flink code. You can start with `KafkaSource` [1], which has an example block that shows how to use it in the Javadocs.

[1]
https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSource.java