Re: Hello-world example of Flink Table API using a edited Calcite rule

Posted by Felipe Gutierrez on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Hello-world-example-of-Flink-Table-API-using-a-edited-Calcite-rule-tp28422p28425.html

Hi JingsongLee,

it is still not very clear to me. I imagine that I can create an InMemoryExternalCatalog and insert some tuples there (which will be in memory). Then I can use Calcite to use the values of my InMemoryExternalCatalog and change my plan. Is that correct?

Do you have an example of how to create an InMemoryExternalCatalog using Flink 1.8? Because I guess the Csv [1] class does not exist anymore.

[1] https://github.com/srapisarda/stypes-flink/blob/feature/sql/src/test/scala/uk/ac/bbk/dcs/stypes/flink/CalciteEmptyConsistencyTest.scala#L73

Kind Regards,
Felipe

On Wed, Jun 26, 2019 at 1:36 PM JingsongLee <[hidden email]> wrote:

Hi Felipe:

I think your approach is absolutely right. You can try to do some plan test just like [1].

You can find more CalciteConfigBuilder API test in [2].

1.https://github.com/apache/flink/blob/release-1.8/flink-table/flink-table-planner/src/test/scala/org/apache/flink/table/api/batch/table/CorrelateTest.scala#L168

2.https://github.com/apache/flink/blob/release-1.8/flink-table/flink-table-planner/src/test/scala/org/apache/flink/table/calcite/CalciteConfigBuilderTest.scala

Best, JingsongLee

------------------------------------------------------------------

From:Felipe Gutierrez <[hidden email]>

Send Time:2019年6月26日(星期三) 18:04

To:user <[hidden email]>

Subject:Hello-world example of Flink Table API using a edited Calcite rule

Hi,

does someone have a simple example using Table API and a Calcite rule which change/optimize the query execution plan of a query in Flink?

From the official documentation, I know that I have to create a CalciteConfig object [1]. Then, I based my firsts tests on this stackoverflow post [2] and I implemented this piece of code:

// change the current calcite config plan
CalciteConfigBuilder ccb = new CalciteConfigBuilder();
RuleSet ruleSets = RuleSets.ofList(FilterMergeRule.INSTANCE);
ccb.addLogicalOptRuleSet(ruleSets);
TableConfig tableConfig = new TableConfig();
tableConfig.setCalciteConfig(ccb.build());

I suppose that with this I can change the query plan of the Flink Table API. I am also not sure if I will need to use an external catalog like this post assumes to use [3].
In a nutshell, I would like to have a simple example where I can execute a query using Flink Table API and change its query execution plan using a Calcite rule. Does anyone have a Hello world of it? I plan to use it on this example [4].

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.8/dev/table/common.html#query-optimization
[2] https://stackoverflow.com/questions/54278508/how-can-i-create-an-external-catalog-table-in-apache-flink
[3] https://stackoverflow.com/questions/54278508/how-can-i-create-an-external-catalog-table-in-apache-flink
[4] https://github.com/felipegutierrez/explore-flink/blob/master/src/main/java/org/sense/flink/examples/stream/table/MqttSensorDataAverageTableAPI.java#L40

Kind Regards,
Felipe

--

-- Felipe Gutierrez

-- skype: felipe.o.gutierrez

-- https://felipeogutierrez.blogspot.com