Contain topic name in the stream

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Contain topic name in the stream

Hung
Hi Flink users,

Would it be possible that the current kafka connector contain the topic name in the stream (each event)?

For example, when reading a list of topics, we often want to count the events in each topic which used keyBy().
I remember already saw the implementation mentioned somewhere, and it would look like to join the configuration of Kafka connector with the data stream? With this feature we can append static data to each event in the stream and will be very useful?

Best,

Sendoh
Reply | Threaded
Open this post in threaded view
|

Re: Contain topic name in the stream

rmetzger0
Hi Sendoh,

the KeyedDeserializationSchema allows you to access the topic name, partition id and offset of the message.
So you need to implement a custom deserialization schema, extending the  KeyedDeserializationSchema to get this information.

Regards,
Robert

On Mon, Aug 22, 2016 at 10:58 AM, Sendoh <[hidden email]> wrote:
Hi Flink users,

Would it be possible that the current kafka connector contain the topic name
in the stream (each event)?

For example, when reading a list of topics, we often want to count the
events in each topic which used keyBy().
I remember already saw the implementation mentioned somewhere, and it would
look like to join the configuration of Kafka connector with the data stream?
With this feature we can append static data to each event in the stream and
will be very useful?

Best,

Sendoh



--
View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Contain-topic-name-in-the-stream-tp8595.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.