Re: Help needed to increase throughput of simple flink app

Posted by ashwinkonale on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Help-needed-to-increase-throughput-of-simple-flink-app-tp39289p39427.html

Hi Arvid,
Thank you so much for your detailed reply. I think I will go with one schema
per topic using GenericRecordAvroTypeInfo for genericRecords and not do any
custom magic.
----
Approach of sending records as byte array also seems quite interesting.
Right now I am deserializing avro records so that I can pass it to
StreamingFileSink's AvroWriters(Which accepts only avro objects) so that it
merges bunch of avro records before dumping to sink. It seems unnecessary
for me, since there could be some bulk writer implementation which could do
this at byte level itself. Do you know any of such implementations ?



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/