cc this to user & user-zh mailing list because this will affect lots of users, and also quite a lot of users were asking questions around this topic. Let me try to understand this from user's perspective. Your proposal will affect five functions, which are:
Before the changes, as I am writing this reply, the local time here is 2021-01-21 12:03:35 (Beijing time, UTC+8). And I tried these 5 functions in sql client, and got: Flink SQL> select now(), PROCTIME(), CURRENT_TIMESTAMP, CURRENT_DATE, CURRENT_TIME; +-------------------------+-------------------------+-------------------------+--------------+--------------+ | EXPR$0 | EXPR$1 | CURRENT_TIMESTAMP | CURRENT_DATE | CURRENT_TIME | +-------------------------+-------------------------+-------------------------+--------------+--------------+ | 2021-01-21T04:03:35.228 | 2021-01-21T04:03:35.228 | 2021-01-21T04:03:35.228 | 2021-01-21 | 04:03:35.228 | +-------------------------+-------------------------+-------------------------+--------------+--------------+ After the changes, the expected behavior will change to: Flink SQL> select now(), PROCTIME(), CURRENT_TIMESTAMP, CURRENT_DATE, CURRENT_TIME; +-------------------------+-------------------------+-------------------------+--------------+--------------+ | EXPR$0 | EXPR$1 | CURRENT_TIMESTAMP | CURRENT_DATE | CURRENT_TIME | +-------------------------+-------------------------+-------------------------+--------------+--------------+ | 2021-01-21T12:03:35.228 | 2021-01-21T12:03:35.228 | 2021-01-21T12:03:35.228 | 2021-01-21 | 12:03:35.228 | +-------------------------+-------------------------+-------------------------+--------------+--------------+ The return type of now(), proctime() and CURRENT_TIMESTAMP still be TIMESTAMP; Best, Kurt On Tue, Jan 19, 2021 at 6:42 PM Leonard Xu <[hidden email]> wrote: I found above example format may mess up in different mail client, I post a picture here[1]. |
Great examples to understand the problem and the proposed changes, @Kurt! Thanks Leonard for investigating this problem. The time-zone problems around time functions and windows have bothered a lot of users. It's time to fix them! The return value changes sound reasonable to me, and keeping the return type unchanged will minimize the surprise to the users. Besides that, I think it would be better to mention how this affects the window behaviors, and the interoperability with DataStream. I think this definitely deserves a FLIP. ==================================================== Hi zhisheng, Do you have examples to illustrate which case will get the wrong window boundaries? That will help to verify whether the proposed changes can solve your problem. Best, Jark On Thu, 21 Jan 2021 at 12:54, zhisheng <[hidden email]> wrote: Thanks to Leonard Xu for discussing this tricky topic. At present, there are many Flink jobs in our production environment that are used to count day-level reports (eg: count PV/UV ). |
In reply to this post by Kurt Young
To Kurt, thanks for the intuitive case, it really clear, you’re wright that I want to propose to change the return value of these functions. It’s the most important part of the topic from user's perspective. I think this definitely deserves a FLIP.To Jark, nice suggestion, I prepared a FLIP for this topic, and will start the FLIP discussion soon. If use the default Flink SQL, the window time range of the To zhisheng, sorry to hear that this problem influenced your production jobs, Could you share your SQL pattern? we can have more inputs and try to resolve them. Best, Leonard
|
Free forum by Nabble | Edit this page |