Flink Question

[Steve Bistline]

I am having problems with the Flink Kinesis adapter. I have some native KCL code that works fine. I want to replace the .addSource with the CSV String data that is coming in from my KCL code. How can I do that?

// Consume the data streams from AWS Kinesis stream

DataStream<Event> dataStream = env.addSource(new FlinkKinesisConsumer<>(

pt.getRequired("stream"),

new EventSchema(),

kinesisConsumerConfig))

.name("Kinesis Stream Consumer");

Any help would be appreciated

Thanks,

Steve

[Tzu-Li (Gordon) Tai]

Hi Steve,

I’m not sure what you mean by “replacing addSource with CSV String data”. Are your Kinesis records CSV and you want to parse them into Events?

If so, you should be able to do that in the provided DeserializationSchema.

Cheers,

Gordon

On 9 November 2018 at 10:54:22 PM, Steve Bistline ([hidden email]) wrote:

I am having problems with the Flink Kinesis adapter. I have some native KCL code that works fine. I want to replace the .addSource with the CSV String data that is coming in from my KCL code. How can I do that?

// Consume the data streams from AWS Kinesis stream

DataStream<Event> dataStream = env.addSource(new FlinkKinesisConsumer<>(

pt.getRequired("stream"),

new EventSchema(),

kinesisConsumerConfig))

.name("Kinesis Stream Consumer");

Any help would be appreciated

Thanks,

Steve