(DEPRECATED) Apache Flink User Mailing List archive.

NullPointerException while calling TableEnvironment.sqlQuery in Flink 1.12

classic Classic list List threaded Threaded
♦
♦
 3 messages Options
Yuval Itzchakov
Reply | Threaded
Open this post in threaded view
♦
♦
|

NullPointerException while calling TableEnvironment.sqlQuery in Flink 1.12

Yuval Itzchakov
Hi,

While trying to execute a query via TableEnvironment.sqlQuery in Flink 1.12, I receive the following exception:

java.lang.NullPointerException
<init>:114, RelMetadataQuery (org.apache.calcite.rel.metadata)
<clinit>:76, RelMetadataQuery (org.apache.calcite.rel.metadata)
get:39, FlinkRelOptClusterFactory$$anon$1 (org.apache.flink.table.planner.calcite)
get:38, FlinkRelOptClusterFactory$$anon$1 (org.apache.flink.table.planner.calcite)
getMetadataQuery:178, RelOptCluster (org.apache.calcite.plan)
create:108, LogicalFilter (org.apache.calcite.rel.logical)
createFilter:344, RelFactories$FilterFactoryImpl (org.apache.calcite.rel.core)
convertWhere:1042, SqlToRelConverter (org.apache.calcite.sql2rel)
convertSelectImpl:666, SqlToRelConverter (org.apache.calcite.sql2rel)
convertSelect:644, SqlToRelConverter (org.apache.calcite.sql2rel)
convertQueryRecursive:3438, SqlToRelConverter (org.apache.calcite.sql2rel)
convertQuery:570, SqlToRelConverter (org.apache.calcite.sql2rel)
org$apache$flink$table$planner$calcite$FlinkPlannerImpl$$rel:165, FlinkPlannerImpl (org.apache.flink.table.planner.calcite)
rel:157, FlinkPlannerImpl (org.apache.flink.table.planner.calcite)
toQueryOperation:823, SqlToOperationConverter (org.apache.flink.table.planner.operations)
convertSqlQuery:795, SqlToOperationConverter (org.apache.flink.table.planner.operations)
convert:250, SqlToOperationConverter (org.apache.flink.table.planner.operations)
parse:78, ParserImpl (org.apache.flink.table.planner.delegation)
sqlQuery:639, TableEnvironmentImpl (org.apache.flink.table.api.internal)
$anonfun$translateTemplate$2:476, Foo$ (Foo)
apply:-1, 644680650 (ai.hunters.pipeline.Processors$$$Lambda$1597)
evaluateNow:361, FiberContext (zio.internal)
$anonfun$evaluateLater$1:778, FiberContext (zio.internal)
run:-1, 289594359 (zio.internal.FiberContext$$Lambda$617)
runWorker:1149, ThreadPoolExecutor (java.util.concurrent)
run:624, ThreadPoolExecutor$Worker (java.util.concurrent)
run:748, Thread (java.lang)

This seems to be coming from the FlinkRelMetadataQuery class attempting to initialize all handlers:

image.png

This seems to be coming from the calcite shaded JAR inside "flink-table-planner-blink-1.12"

Has anyone ran into this issue? I saw a thread in the chinese user group but I don't understand what's been said there (https://www.mail-archive.com/user-zh@.../msg05874.html)
--
Best Regards,
Yuval Itzchakov.
Danny Chan-2
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: NullPointerException while calling TableEnvironment.sqlQuery in Flink 1.12

Danny Chan-2
Hi Yuval Itzchakov ~

The thread you paste has a different stake trace with your case.

In the pasted thread, the JaninoRelMetadataProvider was missed because we only set it once in a thread local variable, when the RelMetadataQuery was used in a different working thread, the JaninoRelMetadataProvider caused an NPE.

For your case, based on the stack trace, this line throws ~

RelMetadataQuery line 114:

super(null);

But actually this line allows an empty argument and it should not throw.

Can you give a re-producecable case here so that we can debug and find more evidence ?

Yuval Itzchakov <[hidden email]> 于2020年12月22日周二 上午1:52写道：
Hi,

While trying to execute a query via TableEnvironment.sqlQuery in Flink 1.12, I receive the following exception:

java.lang.NullPointerException
<init>:114, RelMetadataQuery (org.apache.calcite.rel.metadata)
<clinit>:76, RelMetadataQuery (org.apache.calcite.rel.metadata)
get:39, FlinkRelOptClusterFactory$$anon$1 (org.apache.flink.table.planner.calcite)
get:38, FlinkRelOptClusterFactory$$anon$1 (org.apache.flink.table.planner.calcite)
getMetadataQuery:178, RelOptCluster (org.apache.calcite.plan)
create:108, LogicalFilter (org.apache.calcite.rel.logical)
createFilter:344, RelFactories$FilterFactoryImpl (org.apache.calcite.rel.core)
convertWhere:1042, SqlToRelConverter (org.apache.calcite.sql2rel)
convertSelectImpl:666, SqlToRelConverter (org.apache.calcite.sql2rel)
convertSelect:644, SqlToRelConverter (org.apache.calcite.sql2rel)
convertQueryRecursive:3438, SqlToRelConverter (org.apache.calcite.sql2rel)
convertQuery:570, SqlToRelConverter (org.apache.calcite.sql2rel)
org$apache$flink$table$planner$calcite$FlinkPlannerImpl$$rel:165, FlinkPlannerImpl (org.apache.flink.table.planner.calcite)
rel:157, FlinkPlannerImpl (org.apache.flink.table.planner.calcite)
toQueryOperation:823, SqlToOperationConverter (org.apache.flink.table.planner.operations)
convertSqlQuery:795, SqlToOperationConverter (org.apache.flink.table.planner.operations)
convert:250, SqlToOperationConverter (org.apache.flink.table.planner.operations)
parse:78, ParserImpl (org.apache.flink.table.planner.delegation)
sqlQuery:639, TableEnvironmentImpl (org.apache.flink.table.api.internal)
$anonfun$translateTemplate$2:476, Foo$ (Foo)
apply:-1, 644680650 (ai.hunters.pipeline.Processors$$$Lambda$1597)
evaluateNow:361, FiberContext (zio.internal)
$anonfun$evaluateLater$1:778, FiberContext (zio.internal)
run:-1, 289594359 (zio.internal.FiberContext$$Lambda$617)
runWorker:1149, ThreadPoolExecutor (java.util.concurrent)
run:624, ThreadPoolExecutor$Worker (java.util.concurrent)
run:748, Thread (java.lang)

This seems to be coming from the FlinkRelMetadataQuery class attempting to initialize all handlers:

image.png

This seems to be coming from the calcite shaded JAR inside "flink-table-planner-blink-1.12"

Has anyone ran into this issue? I saw a thread in the chinese user group but I don't understand what's been said there (https://www.mail-archive.com/user-zh@.../msg05874.html)
--
Best Regards,
Yuval Itzchakov.
Yuval Itzchakov
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: NullPointerException while calling TableEnvironment.sqlQuery in Flink 1.12

Yuval Itzchakov
Hi Danny,

I will try to provide a compiling reproduction of the issue. 
But to my further investigation it seems that this problem is actually related JaninoRelMetadataProvider and thread local being null:

I see that:

image.png

THREAD_PROVIDERS.get returns null, and immediately following that there's an `Objects.requireNotNull` check in the constructor:

image.png
Which is what throws the NPE.

Full stack:
requireNonNull:203, Objects (java.util)
<init>:144, RelMetadataQuery (org.apache.calcite.rel.metadata)
<init>:108, RelMetadataQuery (org.apache.calcite.rel.metadata)
<init>:73, FlinkRelMetadataQuery (org.apache.flink.table.planner.plan.metadata)
instance:52, FlinkRelMetadataQuery (org.apache.flink.table.planner.plan.metadata)
get:39, FlinkRelOptClusterFactory$$anon$1 (org.apache.flink.table.planner.calcite)
get:38, FlinkRelOptClusterFactory$$anon$1 (org.apache.flink.table.planner.calcite)
getMetadataQuery:178, RelOptCluster (org.apache.calcite.plan)
create:118, LogicalProject (org.apache.calcite.rel.logical)
create:111, LogicalProject (org.apache.calcite.rel.logical)
createProject:177, RelFactories$ProjectFactoryImpl (org.apache.calcite.rel.core)
project_:1516, RelBuilder (org.apache.calcite.tools)
project:1311, RelBuilder (org.apache.calcite.tools)
projectNamed:1565, RelBuilder (org.apache.calcite.tools)
createAggImpl:3145, SqlToRelConverter (org.apache.calcite.sql2rel)
convertAgg:3050, SqlToRelConverter (org.apache.calcite.sql2rel)
convertSelectImpl:682, SqlToRelConverter (org.apache.calcite.sql2rel)
convertSelect:644, SqlToRelConverter (org.apache.calcite.sql2rel)
convertQueryRecursive:3438, SqlToRelConverter (org.apache.calcite.sql2rel)
convertFrom:2169, SqlToRelConverter (org.apache.calcite.sql2rel)
convertFrom:2093, SqlToRelConverter (org.apache.calcite.sql2rel)
convertFrom:2050, SqlToRelConverter (org.apache.calcite.sql2rel)
convertSelectImpl:663, SqlToRelConverter (org.apache.calcite.sql2rel)
convertSelect:644, SqlToRelConverter (org.apache.calcite.sql2rel)
convertQueryRecursive:3438, SqlToRelConverter (org.apache.calcite.sql2rel)
convertFrom:2169, SqlToRelConverter (org.apache.calcite.sql2rel)
convertFrom:2093, SqlToRelConverter (org.apache.calcite.sql2rel)
convertFrom:2050, SqlToRelConverter (org.apache.calcite.sql2rel)
convertSelectImpl:663, SqlToRelConverter (org.apache.calcite.sql2rel)
convertSelect:644, SqlToRelConverter (org.apache.calcite.sql2rel)
convertQueryRecursive:3438, SqlToRelConverter (org.apache.calcite.sql2rel)
convertQuery:570, SqlToRelConverter (org.apache.calcite.sql2rel)
org$apache$flink$table$planner$calcite$FlinkPlannerImpl$$rel:165, FlinkPlannerImpl (org.apache.flink.table.planner.calcite)
rel:157, FlinkPlannerImpl (org.apache.flink.table.planner.calcite)
toQueryOperation:823, SqlToOperationConverter (org.apache.flink.table.planner.operations)
convertSqlQuery:795, SqlToOperationConverter (org.apache.flink.table.planner.operations)
convert:250, SqlToOperationConverter (org.apache.flink.table.planner.operations)
parse:78, ParserImpl (org.apache.flink.table.planner.delegation)
sqlQuery:639, TableEnvironmentImpl (org.apache.flink.table.api.internal)
$anonfun$translateTemplate$2:476, Processors$ (ai.hunters.pipeline)
apply:-1, 1722705773 (ai.hunters.pipeline.Processors$$$Lambda$1594)
evaluateNow:361, FiberContext (zio.internal)
$anonfun$evaluateLater$1:778, FiberContext (zio.internal)
run:-1, 2031795786 (zio.internal.FiberContext$$Lambda$617)
runWorker:1149, ThreadPoolExecutor (java.util.concurrent)
run:624, ThreadPoolExecutor$Worker (java.util.concurrent)
run:748, Thread (java.lang)

On Tue, Dec 22, 2020 at 4:13 AM Danny Chan <[hidden email]> wrote:
Hi Yuval Itzchakov ~

The thread you paste has a different stake trace with your case.

In the pasted thread, the JaninoRelMetadataProvider was missed because we only set it once in a thread local variable, when the RelMetadataQuery was used in a different working thread, the JaninoRelMetadataProvider caused an NPE.

For your case, based on the stack trace, this line throws ~

RelMetadataQuery line 114:

super(null);

But actually this line allows an empty argument and it should not throw.

Can you give a re-producecable case here so that we can debug and find more evidence ?

Yuval Itzchakov <[hidden email]> 于2020年12月22日周二 上午1:52写道：
Hi,

While trying to execute a query via TableEnvironment.sqlQuery in Flink 1.12, I receive the following exception:

java.lang.NullPointerException
<init>:114, RelMetadataQuery (org.apache.calcite.rel.metadata)
<clinit>:76, RelMetadataQuery (org.apache.calcite.rel.metadata)
get:39, FlinkRelOptClusterFactory$$anon$1 (org.apache.flink.table.planner.calcite)
get:38, FlinkRelOptClusterFactory$$anon$1 (org.apache.flink.table.planner.calcite)
getMetadataQuery:178, RelOptCluster (org.apache.calcite.plan)
create:108, LogicalFilter (org.apache.calcite.rel.logical)
createFilter:344, RelFactories$FilterFactoryImpl (org.apache.calcite.rel.core)
convertWhere:1042, SqlToRelConverter (org.apache.calcite.sql2rel)
convertSelectImpl:666, SqlToRelConverter (org.apache.calcite.sql2rel)
convertSelect:644, SqlToRelConverter (org.apache.calcite.sql2rel)
convertQueryRecursive:3438, SqlToRelConverter (org.apache.calcite.sql2rel)
convertQuery:570, SqlToRelConverter (org.apache.calcite.sql2rel)
org$apache$flink$table$planner$calcite$FlinkPlannerImpl$$rel:165, FlinkPlannerImpl (org.apache.flink.table.planner.calcite)
rel:157, FlinkPlannerImpl (org.apache.flink.table.planner.calcite)
toQueryOperation:823, SqlToOperationConverter (org.apache.flink.table.planner.operations)
convertSqlQuery:795, SqlToOperationConverter (org.apache.flink.table.planner.operations)
convert:250, SqlToOperationConverter (org.apache.flink.table.planner.operations)
parse:78, ParserImpl (org.apache.flink.table.planner.delegation)
sqlQuery:639, TableEnvironmentImpl (org.apache.flink.table.api.internal)
$anonfun$translateTemplate$2:476, Foo$ (Foo)
apply:-1, 644680650 (ai.hunters.pipeline.Processors$$$Lambda$1597)
evaluateNow:361, FiberContext (zio.internal)
$anonfun$evaluateLater$1:778, FiberContext (zio.internal)
run:-1, 289594359 (zio.internal.FiberContext$$Lambda$617)
runWorker:1149, ThreadPoolExecutor (java.util.concurrent)
run:624, ThreadPoolExecutor$Worker (java.util.concurrent)
run:748, Thread (java.lang)

This seems to be coming from the FlinkRelMetadataQuery class attempting to initialize all handlers:

image.png

This seems to be coming from the calcite shaded JAR inside "flink-table-planner-blink-1.12"

Has anyone ran into this issue? I saw a thread in the chinese user group but I don't understand what's been said there (https://www.mail-archive.com/user-zh@.../msg05874.html)
--
Best Regards,
Yuval Itzchakov.


--
Best Regards,
Yuval Itzchakov.
Free forum by Nabble Edit this page