Creating flink byte[] deserialiser

[Soheil Pourbafrani]

Hi,

I want to read(consume) data from Kafka as byte array just something like Kafka byte array deserializer. In Flink I just find SimpleStringSchema and it is note suitable for my data.

Is any built-in byte array deserializer in Flink or if not how can I create a simple byte array deserializer?

Thanks

[Tzu-Li (Gordon) Tai]

Hi Soheil,

AFAIK, there is no built-in byte array deserializer in Flink.

However, it is very simple to implement one.

You can do that by implementing the `DeserializationSchema` interface, and for the implementation of the `deserialize` method, simply return the fetched bytes from Kafka as the deserialized event.

For the `getProducedType` method implementation, you can return the built-in `PrimitiveArrayTypeInfo.BYTE_PRIMITIVE_ARRAY_TYPE_INFO`.

You can then use the implemented deserialization schema with the FlinkKafkaConsumer to read a `DataStream<byte[]>` from Kafka.

Cheers,

Gordon

On 27 November 2017 at 4:19:47 PM, Soheil Pourbafrani ([hidden email]) wrote:

Hi,

I want to read(consume) data from Kafka as byte array just something like Kafka byte array deserializer. In Flink I just find SimpleStringSchema and it is note suitable for my data.

Is any built-in byte array deserializer in Flink or if not how can I create a simple byte array deserializer?

Thanks