Architecture question

Posted by robert on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Architecture-question-tp18341.html

I need to grab avro data from a kafka topic and write to the local file
system

Inside the avro record there is a date time field. From that field I need to
name the file accordingly. (20180103) as an example


I was thinking of using flink to read, unpack this generic record then put
to a sink that will sort to make sure it goes into the right file.

Does anyhow have a high-level approach for this ?

The bucketing sink look promising. Any examples of this type of problem for
flink to solve ?

Thanks



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/