Flink Avro Kafka Reading/Writing

[david]

Hi All,

Does anyone know if Flink can read and write Avro schema to Kafka?

Thanks

daviD

[rmetzger0]

Hi,

yes, Flink can read and write Avro schema to Kafka, using a custom serialization / deser schema.

On Fri, Nov 11, 2016 at 6:05 AM, daviD <[hidden email]> wrote:

Hi All,

Does anyone know if Flink can read and write Avro schema to Kafka?

Thanks

daviD

[Niels Basjes]

In Avro 1.8.2 (which is close to being released) we built a new feature that allows serializing Avro records into a byte array that still contains the schema information.

So even if you keep messages for weeks and the schema of your application changes you can still read the messages at the consumer side.

See:

https://issues.apache.org/jira/browse/AVRO-1704

https://github.com/apache/avro/blob/master/lang/java/ipc/src/test/java/org/apache/avro/message/TestCustomSchemaStore.java

Niels Basjes

On Fri, Nov 11, 2016 at 3:05 PM, daviD <[hidden email]> wrote:

Hi All,

Does anyone know if Flink can read and write Avro schema to Kafka?

Thanks

daviD

--