schema to just read as "byte[] array" from kafka

[Raja.Aravapalli]

 

Hi

 

I am using SimpleStringSchema to deserialize a message read from kafka, but need some help to know if there is any schema available I can use rather than “SimpleStringSchema()” and instead just get “byte[]” without any deserialization happening!

 

Below is code I am currently using, but instead of SimpleStringSchema() which is giving me Strings, but I want the a raw byte array Byte[]:

 

FlinkKafkaConsumer08<String> myConsumer = new FlinkKafkaConsumer08<>("xxx_topic", new SimpleStringSchema(), properties);

 

 

Thanks a lot.

 

 

Regards,

Raja.

[Chao Wang]

Hi Raja,

I just happened to work on the similar thing, and here is how to do it in general, I think (In my case, I did a bit more, to deserialize a tuple of <byte[],byte[]>) :

FlinkKafkaConsumer010<byte[]> consumer = new FlinkKafkaConsumer010<>("topic_name", new MyDe<byte[]>(), properties);

and for MyDe the schema:

public static class MyDe<T> extends AbstractDeserializationSchema<byte[]> {
  @Override
  public byte[] deserialize(byte[] arg0) {
    return new e;
  }
}


Chao

On 08/07/2017 10:47 AM, Raja.Aravapalli wrote:

 

Hi

 

I am using SimpleStringSchema to deserialize a message read from kafka, but need some help to know if there is any schema available I can use rather than “SimpleStringSchema()” and instead just get “byte[]” without any deserialization happening!

 

Below is code I am currently using, but instead of SimpleStringSchema() which is giving me Strings, but I want the a raw byte array Byte[]:

 

FlinkKafkaConsumer08<String> myConsumer = new FlinkKafkaConsumer08<>("xxx_topic", new SimpleStringSchema(), properties);

 

 

Thanks a lot.

 

 

Regards,

Raja.

[Chao Wang]

A quick update, in class MyDe:

public static class MyDe<T> extends AbstractDeserializationSchema<byte[]> {
  @Override
  public byte[] deserialize(byte[] arg0) {
    // Perform deserialization here, if needed;
    // otherwise, probably we can simply return arg0 as raw byte[]
    return arg0;
  }
}


Chao

On 08/07/2017 12:23 PM, Chao Wang wrote:

Hi Raja,

I just happened to work on the similar thing, and here is how to do it in general, I think (In my case, I did a bit more, to deserialize a tuple of <byte[],byte[]>) :

FlinkKafkaConsumer010<byte[]> consumer = new FlinkKafkaConsumer010<>("topic_name", new MyDe<byte[]>(), properties);

and for MyDe the schema:

public static class MyDe<T> extends AbstractDeserializationSchema<byte[]> {
  @Override
  public byte[] deserialize(byte[] arg0) {
    return new e;
  }
}


Chao

On 08/07/2017 10:47 AM, Raja.Aravapalli wrote:

 

Hi

 

I am using SimpleStringSchema to deserialize a message read from kafka, but need some help to know if there is any schema available I can use rather than “SimpleStringSchema()” and instead just get “byte[]” without any deserialization happening!

 

Below is code I am currently using, but instead of SimpleStringSchema() which is giving me Strings, but I want the a raw byte array Byte[]:

 

FlinkKafkaConsumer08<String> myConsumer = new FlinkKafkaConsumer08<>("xxx_topic", new SimpleStringSchema(), properties);

 

 

Thanks a lot.

 

 

Regards,

Raja.

[Raja.Aravapalli]

 

Thank you very much Chao. That helps me.

 

 

Regards,

Raja.

 

From: Chao Wang <[hidden email]>
Date: Monday, August 7, 2017 at 12:28 PM
To: Raja Aravapalli <[hidden email]>
Cc: "[hidden email]" <[hidden email]>
Subject: [EXTERNAL] Re: schema to just read as "byte[] array" from kafka

 

A quick update, in class MyDe:

public static class MyDe<T> extends AbstractDeserializationSchema<byte[]> {
  @Override
  public byte[] deserialize(byte[] arg0) {
    // Perform deserialization here, if needed;
    // otherwise, probably we can simply return arg0 as raw byte[]
    return arg0;
  }
}


Chao

On 08/07/2017 12:23 PM, Chao Wang wrote:

Hi Raja,

I just happened to work on the similar thing, and here is how to do it in general, I think (In my case, I did a bit more, to deserialize a tuple of <byte[],byte[]>) :

FlinkKafkaConsumer010<byte[]> consumer = new FlinkKafkaConsumer010<>("topic_name", new MyDe<byte[]>(), properties);

and for MyDe the schema:

public static class MyDe<T> extends AbstractDeserializationSchema<byte[]> {
  @Override
  public byte[] deserialize(byte[] arg0) {
    return new e;
  }
}


Chao

On 08/07/2017 10:47 AM, Raja.Aravapalli wrote:

 

Hi

 

I am using SimpleStringSchema to deserialize a message read from kafka, but need some help to know if there is any schema available I can use rather than “SimpleStringSchema()” and instead just get “byte[]” without any deserialization happening!

 

Below is code I am currently using, but instead of SimpleStringSchema() which is giving me Strings, but I want the a raw byte array Byte[]:

 

FlinkKafkaConsumer08<String> myConsumer = new FlinkKafkaConsumer08<>("xxx_topic", new SimpleStringSchema(), properties);

 

 

Thanks a lot.

 

 

Regards,

Raja.