(DEPRECATED) Apache Flink User Mailing List archive.

Why consecutive calls of orderBy are forbidden?

classic Classic list List threaded Threaded
♦
♦
 3 messages Options
洪帆(既起)
Reply | Threaded
Open this post in threaded view
♦
♦
|

Why consecutive calls of orderBy are forbidden?

洪帆(既起)
Hi, all.
I tried calling two consecutive orderBy for a Table, but got an exception.
Can anyone explain why this happens? 
In my mind, orderBy should be able to be called by any Tables. But obviously, it is not with no explanation.

Here is a simplified version of code:

Table table = btenv.scan("source").orderBy("cola");
table.insertInto("sink");
Table table2 = table.orderBy("colb");
table2.insertInto("sink2");
btenv.execute("testest");

The exception is as follows:

java.lang.NullPointerException
 at org.apache.flink.optimizer.dag.SingleInputNode.computeInterestingPropertiesForInputs(SingleInputNode.java:224)
 at org.apache.flink.optimizer.traversals.InterestingPropertyVisitor.preVisit(InterestingPropertyVisitor.java:51)
 at org.apache.flink.optimizer.traversals.InterestingPropertyVisitor.preVisit(InterestingPropertyVisitor.java:29)
 at org.apache.flink.optimizer.dag.SingleInputNode.accept(SingleInputNode.java:513)
 at org.apache.flink.optimizer.dag.SingleInputNode.accept(SingleInputNode.java:515)
 at org.apache.flink.optimizer.dag.SingleInputNode.accept(SingleInputNode.java:515)
 at org.apache.flink.optimizer.dag.SingleInputNode.accept(SingleInputNode.java:515)
 at org.apache.flink.optimizer.dag.SingleInputNode.accept(SingleInputNode.java:515)
 at org.apache.flink.optimizer.dag.SingleInputNode.accept(SingleInputNode.java:515)
 at org.apache.flink.optimizer.dag.DataSinkNode.accept(DataSinkNode.java:248)
 at org.apache.flink.optimizer.dag.TwoInputNode.accept(TwoInputNode.java:751)
 at org.apache.flink.optimizer.Optimizer.compile(Optimizer.java:493)
 at org.apache.flink.optimizer.Optimizer.compile(Optimizer.java:399)
 at org.apache.flink.test.util.TestEnvironment.compileProgram(TestEnvironment.java:132)
 at org.apache.flink.test.util.TestEnvironment.execute(TestEnvironment.java:105)
 at org.apache.flink.table.api.internal.BatchTableEnvImpl.execute(BatchTableEnvImpl.scala:225)
 at com.alibaba.alink.operator.batch.dataproc.SqlBatchOpsTest.testOrderBy(SqlBatchOpsTest.java:306)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
 at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
 at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
 at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
 at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
 at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
 at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
 at org.junit.rules.RunRules.evaluate(RunRules.java:20)
 at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
 at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
 at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
 at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
 at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
 at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
 at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
 at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
 at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
 at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
 at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
 at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
 at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69)
 at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
 at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:220)
 at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:53)
Dawid Wysakowicz-2
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Why consecutive calls of orderBy are forbidden?

Dawid Wysakowicz-2

Hi,

I think you are hitting a bug here. It should be possible what you are trying to do. Would you like to open a bug for it?

However, the bug applies to the legacy batch planner (you are using the BatchTableEnvironment), which is no longer maintained and there were discussions already to drop it in one of the upcoming releases (maybe even 1.12). I'd suggest upgrading to the Blink planner.

Moreover as a workaround you could apply the second orderBy on a non-sorted table. There is no point ub applying two different orderings as the second one will just override the order from the first sorting.

Best,

Dawid

On 24/08/2020 13:26, 洪帆(既起) wrote:
Hi, all.
I tried calling two consecutive orderBy for a Table, but got an exception.
Can anyone explain why this happens? 
In my mind, orderBy should be able to be called by any Tables. But obviously, it is not with no explanation.

Here is a simplified version of code:

Table table = btenv.scan("source").orderBy("cola");
table.insertInto("sink");
Table table2 = table.orderBy("colb");
table2.insertInto("sink2");
btenv.execute("testest");

The exception is as follows:

java.lang.NullPointerException
 at org.apache.flink.optimizer.dag.SingleInputNode.computeInterestingPropertiesForInputs(SingleInputNode.java:224)
 at org.apache.flink.optimizer.traversals.InterestingPropertyVisitor.preVisit(InterestingPropertyVisitor.java:51)
 at org.apache.flink.optimizer.traversals.InterestingPropertyVisitor.preVisit(InterestingPropertyVisitor.java:29)
 at org.apache.flink.optimizer.dag.SingleInputNode.accept(SingleInputNode.java:513)
 at org.apache.flink.optimizer.dag.SingleInputNode.accept(SingleInputNode.java:515)
 at org.apache.flink.optimizer.dag.SingleInputNode.accept(SingleInputNode.java:515)
 at org.apache.flink.optimizer.dag.SingleInputNode.accept(SingleInputNode.java:515)
 at org.apache.flink.optimizer.dag.SingleInputNode.accept(SingleInputNode.java:515)
 at org.apache.flink.optimizer.dag.SingleInputNode.accept(SingleInputNode.java:515)
 at org.apache.flink.optimizer.dag.DataSinkNode.accept(DataSinkNode.java:248)
 at org.apache.flink.optimizer.dag.TwoInputNode.accept(TwoInputNode.java:751)
 at org.apache.flink.optimizer.Optimizer.compile(Optimizer.java:493)
 at org.apache.flink.optimizer.Optimizer.compile(Optimizer.java:399)
 at org.apache.flink.test.util.TestEnvironment.compileProgram(TestEnvironment.java:132)
 at org.apache.flink.test.util.TestEnvironment.execute(TestEnvironment.java:105)
 at org.apache.flink.table.api.internal.BatchTableEnvImpl.execute(BatchTableEnvImpl.scala:225)
 at com.alibaba.alink.operator.batch.dataproc.SqlBatchOpsTest.testOrderBy(SqlBatchOpsTest.java:306)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
 at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
 at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
 at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
 at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
 at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
 at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
 at org.junit.rules.RunRules.evaluate(RunRules.java:20)
 at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
 at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
 at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
 at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
 at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
 at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
 at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
 at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
 at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
 at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
 at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
 at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
 at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69)
 at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
 at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:220)
 at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:53)


signature.asc (849 bytes) Download Attachment
hongfanxo
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Why consecutive calls of orderBy are forbidden?

hongfanxo
Hi,
Thanks for your reply.
I'll look in to the Blink planner later.

For the workaround you mentioned, in the actual usage, the second orderBy is
wrapped in a function.
So I've no idea what has been done for the input Table.



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
Free forum by Nabble Edit this page