Optimizations not performed - please confirm
Posted by
Ovidiu-Cristian MARCU on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Optimizations-not-performed-please-confirm-tp7713.html
Hi,
The optimizer internals described in this document [1] are probably not up-to-date.
Can you please confirm if this is still valid:
“The following optimizations are not performed
- Join reordering (or operator reordering in general): Joins / Filters / Reducers are not re-ordered in Flink. This is a high opportunity optimization, but with high risk in the absence of good estimates about the data characteristics. Flink is not doing these optimizations at this point.
- Index vs. Table Scan selection: In Flink, all data sources are always scanned. The data source (the input format) may apply clever mechanism to not scan all the data, but pre-select and project. Examples are the RCFile / ORCFile / Parquet input formats."
Any update of this page will be very helpful.
Thank you.
Best,
Ovidiu