Flink job testing with

[Chauvet, Thomas]

Hi everybody,

 

I would like to test a kafka / flink process in scala. I would like to proceed as in the integration testing documentation : https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/stream/testing.html#integration-testing with Kafka as source and sink.

 

For example, I have a topic kafka as source for flink (I use FlinkKafkaConsumer011), then I do some process with Flink, then I send the stream to Kafka (FlinkKafkaProducer011).

 

Any idea on how to do that ? Or better, any example ?

 

Thanks

[Tzu-Li (Gordon) Tai]

Hi,

The docs here [1] provide some example snippets of using the Kafka connector
to consume from / write to Kafka topics.

Once you consumed a `DataStream` from a Kafka topic using the Kafka
consumer, you can use Flink transformations such as map, flatMap, etc. to
perform processing on the records.

Hope this helps!

Cheers,
Gordon

[1]
https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/connectors/kafka.html



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/