Hi All,
Do you know if window function supported on SQL yet? I got the error message when trying to use group function in SQL. My query below: val query = "SELECT nd_key, concept_rank, event_timestamp FROM "+streamName + " GROUP BY TUMBLE(event_timestamp, INTERVAL '1' HOUR), nd_key" Error Message: Exception in thread "main" org.apache.flink.table.api. 'TUMBLE(<DATETIME>, <DATETIME_INTERVAL>, <TIME>)' at org.apache.flink.table. at org.apache.flink.table.api. at com.udacity.data.pipeline. at com.udacity.data.pipeline. Caused by: org.apache.calcite.runtime. 'TUMBLE(<DATETIME>, <DATETIME_INTERVAL>, <TIME>)' at sun.reflect. at sun.reflect. at sun.reflect. at java.lang.reflect.Constructor. at org.apache.calcite.runtime. at org.apache.calcite.sql. at org.apache.calcite.sql. at org.apache.calcite.sql. at org.apache.calcite.sql. at org.apache.calcite.sql.type. at org.apache.calcite.sql.type. at org.apache.calcite.sql.type. at org.apache.calcite.sql. at org.apache.calcite.sql. at org.apache.calcite.sql. at org.apache.calcite.sql. at org.apache.calcite.sql. at org.apache.calcite.sql. at org.apache.calcite.sql. at org.apache.calcite.sql. at org.apache.calcite.sql. at org.apache.calcite.sql. at org.apache.calcite.sql. at org.apache.calcite.sql. at org.apache.calcite.sql. at org.apache.calcite.sql. at org.apache.calcite.sql. at org.apache.calcite.sql. at org.apache.calcite.sql. at org.apache.calcite.sql. at org.apache.calcite.sql. at org.apache.flink.table. ... 3 more Caused by: org.apache.calcite.sql. 'TUMBLE(<DATETIME>, <DATETIME_INTERVAL>, <TIME>)' at sun.reflect. at sun.reflect. at sun.reflect. at java.lang.reflect.Constructor. at org.apache.calcite.runtime. at org.apache.calcite.runtime. ... 30 more |
Hi, yes, Apache Calcite's group window functions are supported.[1] https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/table/sql.html#group-windows 2017-12-04 22:17 GMT+01:00 Tao Xia <[hidden email]>:
|
Thanks for the quick response Fabian I have DataStream of avro objects. Not sure how to add a TIMESTAMP attribute or convert the event_timestramp field to Timestamp Attribute for my SQL use cases. Most docs only covers the Table API with static schema. p.s. my Avro schema has 100+ fields. Can you guide me how to prepare my query to aggregate by nd_key and event_timestamp per hour? val testData = List( UnifiedEvent.newBuilder().setNdKey("nd101").setConceptRank(10).setEventTimestamp(1512172415.longValue()).build(), val kinesisStream = env.fromCollection(testData) tableEnv.registerDataStream(streamName, avroStream); val query = "SELECT nd_key, sum(concept_rank) FROM "+streamName + " GROUP BY nd_key" Thanks, Tao On Mon, Dec 4, 2017 at 3:32 PM, Fabian Hueske <[hidden email]> wrote:
|
Hi Tao, computing a group window requires that the event-time timestamp of the DataStream is exposed as a time attribute (in your case as an event time attribute). If you register DataStream at the TableEnvironment, this has to be done in two steps:val kinesisStream = env2) declare the event timestamp of the DataStream as an attribute in the schema of the table [2]: tableEnv.registerDataStream( Once event_timestamp is declared as time attribute, it can be used in window functions. 2017-12-05 1:40 GMT+01:00 Tao Xia <[hidden email]>:
|
Free forum by Nabble | Edit this page |