Flink Kinesis Producer and records

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Flink Kinesis Producer and records

Yoandy Rodríguez

Hello everyone,

I have a Flink job that uses a Kinesis Data Stream as a sink.

producerConfig.put("CollectionMaxCount", "500"); // TODO: Move this into an external property
        FlinkKinesisProducer<String> kinesis = new FlinkKinesisProducer<>(new SimpleStringSchema(), producerConfig);
        kinesis.setFailOnError(true);
        kinesis.setDefaultStream((String)props.get("OutputStream"));
        kinesis.setQueueLimit(500); // TODO: Move this into an external property
        kinesis.setDefaultPartition("0");

Output data are simple strings, but once in the consumer application I get multiple output items for each call to GetRecord. Is this expected behavior?

Thanks in advance

-- 
Best Regards
Yoandy Rodríguez