Question on the SQL "GROUPING SETS" and "CUBE" syntax usability

[DONG, Weike]

Hi,

From the Flink 1.10  official document (https://ci.apache.org/projects/flink/flink-docs-release-1.10/dev/table/sql/queries.html), we could see that GROUPING SETS is only supported in Batch mode. 

However, we also found that in https://issues.apache.org/jira/browse/FLINK-12192, stream query using GROUPING SETS and CUBE is already a fixed issues in 1.9.

Here I would like to know if SQL support for GROUPING SETS, ROLLUP, CUBE is ready for use or not, and whether the document needs to be updated or not.

Thank you

Best regards,

Weike

[Arvid Heise-3]

Hi Weike,

according to the linked documentation, the operations are ready but as you have mentioned only for SQL batch mode, which is not surprising as they don't have a well-behaved semantics on streams. See also Calcites explanations [1].

Could you maybe outline your use case and what you'd expect these operations to be? Would you like to combine them with windows?

I'm CCing Jark, as he knows SQL much better than me.

[1] https://calcite.apache.org/docs/stream.html#grouping-sets

On Mon, Mar 9, 2020 at 8:27 AM DONG, Weike <[hidden email]> wrote:

Hi,

From the Flink 1.10  official document (https://ci.apache.org/projects/flink/flink-docs-release-1.10/dev/table/sql/queries.html), we could see that GROUPING SETS is only supported in Batch mode. 

However, we also found that in https://issues.apache.org/jira/browse/FLINK-12192, stream query using GROUPING SETS and CUBE is already a fixed issues in 1.9.

Here I would like to know if SQL support for GROUPING SETS, ROLLUP, CUBE is ready for use or not, and whether the document needs to be updated or not.

Thank you

Best regards,

Weike

[Jark Wu-3]

Thanks Arvid for reminding me this topic. 

Actually, it is supported in streaming mode in blink planner (since Flink v1.9), but we missed to update the documentation. 

You can also find it is supported in the integration tests [1].

I created an issue to update docs [2]. 

Best,

Jark

[1]: https://github.com/apache/flink/blob/master/flink-table/flink-table-planner-blink/src/test/scala/org/apache/flink/table/planner/runtime/stream/sql/AggregateRemoveITCase.scala#L68

[2]: https://issues.apache.org/jira/browse/FLINK-16530

On Tue, 10 Mar 2020 at 16:11, Arvid Heise <[hidden email]> wrote:

Hi Weike,

according to the linked documentation, the operations are ready but as you have mentioned only for SQL batch mode, which is not surprising as they don't have a well-behaved semantics on streams. See also Calcites explanations [1].

Could you maybe outline your use case and what you'd expect these operations to be? Would you like to combine them with windows?

I'm CCing Jark, as he knows SQL much better than me.

[1] https://calcite.apache.org/docs/stream.html#grouping-sets

On Mon, Mar 9, 2020 at 8:27 AM DONG, Weike <[hidden email]> wrote:

Hi,

From the Flink 1.10  official document (https://ci.apache.org/projects/flink/flink-docs-release-1.10/dev/table/sql/queries.html), we could see that GROUPING SETS is only supported in Batch mode. 

However, we also found that in https://issues.apache.org/jira/browse/FLINK-12192, stream query using GROUPING SETS and CUBE is already a fixed issues in 1.9.

Here I would like to know if SQL support for GROUPING SETS, ROLLUP, CUBE is ready for use or not, and whether the document needs to be updated or not.

Thank you

Best regards,

Weike