(DEPRECATED) Apache Flink User Mailing List archive.

NPE in JDBCInputFormat

classic Classic list List threaded Threaded
♦
♦
 4 messages Options
Martin Scholl
Reply | Threaded
Open this post in threaded view
♦
♦
|

NPE in JDBCInputFormat

Martin Scholl
Hello everyone,


JDBCInputFormat of flink 1.1-SNAPSHOT fails with an NPE in Row.productArity:

%% snip %%
java.io.IOException: Couldn't access resultSet
        at org.apache.flink.api.java.io.jdbc.JDBCInputFormat.nextRecord(JDBCInputFormat.java:288)
        at org.apache.flink.api.java.io.jdbc.JDBCInputFormat.nextRecord(JDBCInputFormat.java:98)
        at org.apache.flink.runtime.operators.DataSourceTask.invoke(DataSourceTask.java:162)
        at org.apache.flink.runtime.taskmanager.Task.run(Task.java:588)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
        at org.apache.flink.api.table.Row.productArity(Row.scala:28)
        at org.apache.flink.api.java.io.jdbc.JDBCInputFormat.nextRecord(JDBCInputFormat.java:279)
        ... 4 more
%% snip %%

Find the example code triggering this attached to this email.

The reason, I believe, is the way through which Flink creates Row instances through Kryo. As Row expects the number of fields to allocate as a parameter, which Kryo does not provide, the ‘fields’ member of Row ends up being null. As I’m not a reflection, etc. expert, I rather leave a true analysis to more knowledgable programmers.

Part of the attached example is a not very elegant workaround though a custom type and a cast (see jdbcNoIssue).

Am I doing something wrong as to the example code, or shall I open a JIRA ticket?


Thank you in advance,
Martin




JDBCIssue.scala (1K) Download Attachment
signature.asc (817 bytes) Download Attachment
Flavio Pompermaier
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: NPE in JDBCInputFormat

Flavio Pompermaier
I think the problem is somehow related to 
  val DB_ROWTYPE = new RowTypeInfo(
    Seq(BasicTypeInfo.INT_TYPE_INFO),
    Seq("id"))

You have only one filed, I think Seq("id") should be removed.

However this is a bug IMHO, this kind of error should be checked with and handler with a proper error.

On Tue, Jun 21, 2016 at 11:49 AM, Martin Scholl <[hidden email]> wrote:
Hello everyone,


JDBCInputFormat of flink 1.1-SNAPSHOT fails with an NPE in Row.productArity:

%% snip %%
java.io.IOException: Couldn't access resultSet
        at org.apache.flink.api.java.io.jdbc.JDBCInputFormat.nextRecord(JDBCInputFormat.java:288)
        at org.apache.flink.api.java.io.jdbc.JDBCInputFormat.nextRecord(JDBCInputFormat.java:98)
        at org.apache.flink.runtime.operators.DataSourceTask.invoke(DataSourceTask.java:162)
        at org.apache.flink.runtime.taskmanager.Task.run(Task.java:588)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
        at org.apache.flink.api.table.Row.productArity(Row.scala:28)
        at org.apache.flink.api.java.io.jdbc.JDBCInputFormat.nextRecord(JDBCInputFormat.java:279)
        ... 4 more
%% snip %%

Find the example code triggering this attached to this email.

The reason, I believe, is the way through which Flink creates Row instances through Kryo. As Row expects the number of fields to allocate as a parameter, which Kryo does not provide, the ‘fields’ member of Row ends up being null. As I’m not a reflection, etc. expert, I rather leave a true analysis to more knowledgable programmers.

Part of the attached example is a not very elegant workaround though a custom type and a cast (see jdbcNoIssue).

Am I doing something wrong as to the example code, or shall I open a JIRA ticket?


Thank you in advance,
Martin
rmetzger0
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: NPE in JDBCInputFormat

rmetzger0
Hi,

I also think this is a bug. Can you file a JIRA issue for it?

Regards,
Robert


On Tue, Jun 21, 2016 at 12:15 PM, Flavio Pompermaier <[hidden email]> wrote:
I think the problem is somehow related to 
  val DB_ROWTYPE = new RowTypeInfo(
    Seq(BasicTypeInfo.INT_TYPE_INFO),
    Seq("id"))

You have only one filed, I think Seq("id") should be removed.

However this is a bug IMHO, this kind of error should be checked with and handler with a proper error.

On Tue, Jun 21, 2016 at 11:49 AM, Martin Scholl <[hidden email]> wrote:
Hello everyone,


JDBCInputFormat of flink 1.1-SNAPSHOT fails with an NPE in Row.productArity:

%% snip %%
java.io.IOException: Couldn't access resultSet
        at org.apache.flink.api.java.io.jdbc.JDBCInputFormat.nextRecord(JDBCInputFormat.java:288)
        at org.apache.flink.api.java.io.jdbc.JDBCInputFormat.nextRecord(JDBCInputFormat.java:98)
        at org.apache.flink.runtime.operators.DataSourceTask.invoke(DataSourceTask.java:162)
        at org.apache.flink.runtime.taskmanager.Task.run(Task.java:588)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
        at org.apache.flink.api.table.Row.productArity(Row.scala:28)
        at org.apache.flink.api.java.io.jdbc.JDBCInputFormat.nextRecord(JDBCInputFormat.java:279)
        ... 4 more
%% snip %%

Find the example code triggering this attached to this email.

The reason, I believe, is the way through which Flink creates Row instances through Kryo. As Row expects the number of fields to allocate as a parameter, which Kryo does not provide, the ‘fields’ member of Row ends up being null. As I’m not a reflection, etc. expert, I rather leave a true analysis to more knowledgable programmers.

Part of the attached example is a not very elegant workaround though a custom type and a cast (see jdbcNoIssue).

Am I doing something wrong as to the example code, or shall I open a JIRA ticket?


Thank you in advance,
Martin
Martin Scholl
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: NPE in JDBCInputFormat

Martin Scholl
In reply to this post by Flavio Pompermaier
Hello Flavio,


thank you for looking into the matter.

The 2nd Sequence given to RowTypeInfo has only informational value AFAICS. It does not prevent the issue from happening, I'm afraid.

Martin

P.S.: The behavior is now reported as 'FLINK-4108'.

On Tue, Jun 21, 2016 at 12:16 PM Flavio Pompermaier <[hidden email]> wrote:
I think the problem is somehow related to 
  val DB_ROWTYPE = new RowTypeInfo(
    Seq(BasicTypeInfo.INT_TYPE_INFO),
    Seq("id"))

You have only one filed, I think Seq("id") should be removed.

However this is a bug IMHO, this kind of error should be checked with and handler with a proper error.

On Tue, Jun 21, 2016 at 11:49 AM, Martin Scholl <[hidden email]> wrote:
Hello everyone,


JDBCInputFormat of flink 1.1-SNAPSHOT fails with an NPE in Row.productArity:

%% snip %%
java.io.IOException: Couldn't access resultSet
        at org.apache.flink.api.java.io.jdbc.JDBCInputFormat.nextRecord(JDBCInputFormat.java:288)
        at org.apache.flink.api.java.io.jdbc.JDBCInputFormat.nextRecord(JDBCInputFormat.java:98)
        at org.apache.flink.runtime.operators.DataSourceTask.invoke(DataSourceTask.java:162)
        at org.apache.flink.runtime.taskmanager.Task.run(Task.java:588)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
        at org.apache.flink.api.table.Row.productArity(Row.scala:28)
        at org.apache.flink.api.java.io.jdbc.JDBCInputFormat.nextRecord(JDBCInputFormat.java:279)
        ... 4 more
%% snip %%

Find the example code triggering this attached to this email.

The reason, I believe, is the way through which Flink creates Row instances through Kryo. As Row expects the number of fields to allocate as a parameter, which Kryo does not provide, the ‘fields’ member of Row ends up being null. As I’m not a reflection, etc. expert, I rather leave a true analysis to more knowledgable programmers.

Part of the attached example is a not very elegant workaround though a custom type and a cast (see jdbcNoIssue).

Am I doing something wrong as to the example code, or shall I open a JIRA ticket?


Thank you in advance,
Martin
Free forum by Nabble Edit this page