Dynamic index(by day) to sink to elastic search

[Jacky Yin 殷传旺]

Hello There,

 

We are using flink sql to build a stream pipeline which reads data from kafka, aggregates the data and finally sinks to elastic search.

 

For the table sink to elastic search, we expect to create index by day (e.g. index1-2019-04-08, index1-2019-04-09…). Is this function supported?

 

Another question is about the sink mode of elastic search connector. We found even we set the mode to `Append only`, the document id is still set. As is known, if the document id is set when make bulk request, elastic search will firstly do a key(document id) look-up. This action will impact the performance especially when the index is very large. Where can I configure to force the `real append` ?

 

Thanks!

 

Jacky Yin