Re: Registering RawType with LegacyTypeInfo via Flink CREATE TABLE DDL

Posted by Timo Walther on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Registering-RawType-with-LegacyTypeInfo-via-Flink-CREATE-TABLE-DDL-tp40289p40326.html

Hi Yuval,

the legacy type has no string representation that can be used in a SQL
DDL statement. The current string representation LEGACY(...) is only a
temporary work around to persist the old types in catalogs.

Until FLIP-136 is fully implemented, toAppendStream/toRetractStream
support only legacy type info. So I would recommend to use the legacy
type in the UDF return type as well. Either you use the old
`getResultType` method or you override `getTypeInference` and call
`TypeConversions.fromLegacyInfoToDataType`.

Another work around could be that you simply use `BYTES` as the return
type and pass around a byte array instead.

Maybe you can show us a little end-to-end example, what you are trying
to achieve?

Regards,
Timo


On 28.12.20 07:47, Yuval Itzchakov wrote:

> Hi Danny,
>
> Yes, I tried implementing the DataTypeFactory for the UDF using
> TypeInformationRawType (which is deprecated BTW, and there's no support
> for RawType in the conversion), didn't help.
>
> I did manage to get the conversion working using
> TableEnvironment.toAppendStream (I was previously directly calling
> TypeConversions) but still remains the problem that Flink can't register
> LEGACY types via the CREATE TABLE DDL
>
> On Mon, Dec 28, 2020, 04:25 Danny Chan <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>      > SQL parse failed. Encount
>     What syntax did you use ?
>
>      > TypeConversions.fromDataTypeToLegacyInfo cannot convert a plain
>     RAW type back to TypeInformation.
>
>     Did you try to construct type information by a new
>     fresh TypeInformationRawType ?
>
>     Yuval Itzchakov <[hidden email] <mailto:[hidden email]>> 于
>     2020年12月24日周四 下午7:24写道：
>
>         An expansion to my question:
>
>         What I really want is for the UDF to return `RAW(io.circe.Json,
>         ?)` type, but I have to do a conversion between Table and
>         DataStream, and TypeConversions.fromDataTypeToLegacyInfo cannot
>         convert a plain RAW type back to TypeInformation.
>
>         On Thu, Dec 24, 2020 at 12:59 PM Yuval Itzchakov
>         <[hidden email] <mailto:[hidden email]>> wrote:
>
>             Hi,
>
>             I have a UDF which returns a type of MAP<STRING,
>             LEGACY('RAW', 'ANY<io.circe.Json>')>. When I try to register
>             this type with Flink via the CREATE TABLE DDL, I encounter
>             an exception:
>
>             - SQL parse failed. Encountered "(" at line 2, column 256.
>             Was expecting one of:
>                  "NOT" ...
>                  "NULL" ...
>                  ">" ...
>                  "MULTISET" ...
>                  "ARRAY" ...
>                  "." ...
>
>             Which looks like the planner doesn't like the round brackets
>             on the LEGACY type. What is the correct way to register the
>             table with this type with Flink?
>             --
>             Best Regards,
>             Yuval Itzchakov.
>
>
>
>         --
>         Best Regards,
>         Yuval Itzchakov.
>