Does Flink support long term storage and complex queries

[Basanth Gowda]

We are looking at Druid for long term storage and roll up metrics and adhoc queries on time series data. 

If Flink can support dynamic queries and rollups like Druid does, we would like to just use Flink and leave Druid aside. Does Flink support such kind of queries and what are the guidelines for data retention ?

thank you

[Fabian Hueske-2]

Hi Basanth,

Flink is not a storage system (neither for stream nor for batch data).
Flink applications can be stateful and maintain very large state (in the order of several TBs) but state is always associated with an application.

State can be queryable, so outside applications can run key-lookup (point) queries against state but bulk reading of active state is not supported.

So it depends on your use case if Flink is a good fit.

Flink excels at processing streams and can emit the results to a variety of different storage systems.

Best, Fabian

2017-08-01 2:17 GMT+02:00 Basanth Gowda <[hidden email]>:

We are looking at Druid for long term storage and roll up metrics and adhoc queries on time series data. 

If Flink can support dynamic queries and rollups like Druid does, we would like to just use Flink and leave Druid aside. Does Flink support such kind of queries and what are the guidelines for data retention ?

thank you