Re: How can I set charset for flink sql？

Posted by Fabian Hueske-2 on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/How-can-I-set-charset-for-flink-sql-tp14431p14467.html

As Timo proposed, I would implement a Scalar user-defined function which returns a boolean and use that instead of LIKE.

Have a look here [1].

Best, Fabian

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/table/udfs.html#scalar-functions

2017-07-26 3:47 GMT+02:00 Ted Yu <[hidden email]>:

Logged CALCITE-1903 for this bug.

FYI

On Tue, Jul 25, 2017 at 6:39 PM, 程骥 <[hidden email]> wrote:

OK，thanks for remind me.

My sql like this(contain a Chinese word):

SELECT

'HIGH' AS LEVEL,

'Firewall uplink bandwidth exception:greater than 10000' AS content,

`system.process.username`,

`system.process.memory.rss.bytes`

FROM

test

WHERE

`system.process.username` LIKE '%高危%'

AND 

`system.process.memory.rss.bytes` > 10000

Get exception when I submit the job to cluster.

Caused by: org.apache.calcite.runtime.CalciteException: Failed to encode '%高危%' in character set 'ISO-8859-1'

at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[na:1.8.0_45]

at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source) ~[na:1.8.0_45]

at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source) ~[na:1.8.0_45]

at java.lang.reflect.Constructor.newInstance(Unknown Source) ~[na:1.8.0_45]

at org.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:463) ~[flink-table_2.11-1.3.1.jar:1.3.1]

at org.apache.calcite.runtime.Resources$ExInst.ex(Resources.java:572) ~[flink-table_2.11-1.3.1.jar:1.3.1]

at org.apache.calcite.util.NlsString.<init>(NlsString.java:81) ~[flink-table_2.11-1.3.1.jar:1.3.1]

at org.apache.calcite.rex.RexBuilder.makeLiteral(RexBuilder.java:864) ~[flink-table_2.11-1.3.1.jar:1.3.1]

at org.apache.calcite.rex.RexBuilder.makeCharLiteral(RexBuilder.java:1051) ~[flink-table_2.11-1.3.1.jar:1.3.1]

at org.apache.calcite.sql2rel.SqlNodeToRexConverterImpl.convertLiteral(SqlNodeToRexConverterImpl.java:117) ~[flink-table_2.11-1.3.1.jar:1.3.1]

at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:4408) ~[flink-table_2.11-1.3.1.jar:1.3.1]

at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:3787) ~[flink-table_2.11-1.3.1.jar:1.3.1]

at org.apache.calcite.sql.SqlLiteral.accept(SqlLiteral.java:427) ~[flink-table_2.11-1.3.1.jar:1.3.1]

at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.convertExpression(SqlToRelConverter.java:4321) ~[flink-table_2.11-1.3.1.jar:1.3.1]

at org.apache.calcite.sql2rel.StandardConvertletTable.convertExpressionList(StandardConvertletTable.java:968) ~[flink-table_2.11-1.3.1.jar:1.3.1]

at org.apache.calcite.sql2rel.StandardConvertletTable.convertCall(StandardConvertletTable.java:944) ~[flink-table_2.11-1.3.1.jar:1.3.1]

at org.apache.calcite.sql2rel.StandardConvertletTable.convertCall(StandardConvertletTable.java:928) ~[flink-table_2.11-1.3.1.jar:1.3.1]

... 50 common frames omitted

Is there anyone tell me how to deal with it,thanks!

------------------ 原始邮件 ------------------

发件人: "Nico Kruber";<[hidden email]>;

发送时间: 2017年7月25日(星期二) 晚上11:48

收件人: "user"<[hidden email]>;

抄送: "程骥"<[hidden email]>;

主题: Re: How can I set charset for flink sql？

Please, for the sake of making your email searchable, do not post stack traces
as screenshots but rather text into your email.

On Tuesday, 25 July 2017 12:18:56 CEST 程骥 wrote:
> My sql like this(contain a Chinese word)
>
> Get exception when I submit the job to cluster.
>
>
>
> Is there anyone tell me how to deal with it,thanks!