Logical plan optimization with Calcite

[gallenvara]

Hello, everyone. I'm new to Calcite and have some problems with it. Flink uses the Calcite to parse the sql and construct ast and logical plan. Would the plan be optimized by caicite? For example,
multi filter condition:

val ds = CollectionDataSets.get3TupleDataSet(env).toTable(tEnv, 'a, 'b, 'c)
val filterDs = ds.filter( 'a % 2 === 0 ).filter( 'a % 3 === 0 )

Would the filter condition be optimized to filter('a % 6 === 0) by the calcite or by flink internal optimization rule?

Thanks,
gallenvara

[Maximilian Michels]

Hi Gallenvara,

As far as I know, the Table API is now translated into a Calcite plan
which is then optimized according to Calcite's optimization rules.

Cheers,
Max

On Wed, Jul 20, 2016 at 7:24 AM, gallenvara <[hidden email]> wrote:

>
> Hello, everyone. I'm new to Calcite and have some problems with it. Flink
> uses the Calcite to parse the sql and construct ast and logical plan. Would
> the plan be optimized by caicite? For example,
> multi filter condition:
>
> val ds = CollectionDataSets.get3TupleDataSet(env).toTable(tEnv, 'a, 'b, 'c)
> val filterDs = ds.filter( 'a % 2 === 0 ).filter( 'a % 3 === 0 )
>
> Would the filter condition be optimized to filter('a % 6 === 0) by the
> calcite or by flink internal optimization rule?
>
> Thanks,
> gallenvara
>
>
>
> --
> View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Logical-plan-optimization-with-Calcite-tp8037.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.

[Timo Walther]

Max is right, Flink uses Calcite rules for optimization. The following
rules are applied so far:
https://github.com/apache/flink/blob/master/flink-libraries/flink-table/src/main/scala/org/apache/flink/api/table/plan/rules/FlinkRuleSets.scala

The filter condition you metioned will not be optimized to filter('a % 6
=== 0) but to filter('a % 2 === 0 && 'a % 3 === 0).

FLINK-4068 will further improve this.

Regards,
Timo

Am 20/07/16 um 13:21 schrieb Maximilian Michels:

> Hi Gallenvara,
>
> As far as I know, the Table API is now translated into a Calcite plan
> which is then optimized according to Calcite's optimization rules.
>
> Cheers,
> Max
>
> On Wed, Jul 20, 2016 at 7:24 AM, gallenvara <[hidden email]> wrote:
>> Hello, everyone. I'm new to Calcite and have some problems with it. Flink
>> uses the Calcite to parse the sql and construct ast and logical plan. Would
>> the plan be optimized by caicite? For example,
>> multi filter condition:
>>
>> val ds = CollectionDataSets.get3TupleDataSet(env).toTable(tEnv, 'a, 'b, 'c)
>> val filterDs = ds.filter( 'a % 2 === 0 ).filter( 'a % 3 === 0 )
>>
>> Would the filter condition be optimized to filter('a % 6 === 0) by the
>> calcite or by flink internal optimization rule?
>>
>> Thanks,
>> gallenvara
>>
>>
>>
>> --
>> View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Logical-plan-optimization-with-Calcite-tp8037.html
>> Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.

--
Freundliche Grüße / Kind Regards

Timo Walther

Follow me: @twalthr
https://www.linkedin.com/in/twalthr

[gallenvara]

Thanks Max and Timo for the explanation. :)