Java 8 and keyBy in 1.0.0

[Andrew Ge Wu]

Hi,

This is not very obvious and looks like a bug.

I have a lambda expression to get key from objects in stream:

This works:

stream.keyBy(value -> value.getId())

This does not:

stream.keyBy(myClass::getId)

Exception:

Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: -1

at org.apache.flink.api.java.typeutils.TypeExtractor.getUnaryOperatorReturnType(TypeExtractor.java:294)

at org.apache.flink.api.java.typeutils.TypeExtractor.getKeySelectorTypes(TypeExtractor.java:253)

at org.apache.flink.api.java.typeutils.TypeExtractor.getKeySelectorTypes(TypeExtractor.java:246)

at org.apache.flink.streaming.api.datastream.KeyedStream.<init>(KeyedStream.java:87)

at org.apache.flink.streaming.api.datastream.DataStream.keyBy(DataStream.java:241)

Can someone confirm this? or is there a better way to do it?

Thanks!

Andrew

Confidentiality Notice: This e-mail transmission may contain confidential or legally privileged information that is intended only for the individual or entity named in the e-mail address. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or reliance upon the contents of this e-mail is strictly prohibited and may be unlawful. If you have received this e-mail in error, please notify the sender immediately by return e-mail and delete all copies of this message.

[Chesnay Schepler]

based on https://issues.apache.org/jira/browse/FLINK-3138 this is not supported for non-static methods.

On 30.03.2016 10:33, Andrew Ge Wu wrote:

Hi,

This is not very obvious and looks like a bug.

I have a lambda expression to get key from objects in stream:

This works:

stream.keyBy(value -> value.getId())

This does not:

stream.keyBy(myClass::getId)

Exception:

Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: -1

at org.apache.flink.api.java.typeutils.TypeExtractor.getUnaryOperatorReturnType(TypeExtractor.java:294)

at org.apache.flink.api.java.typeutils.TypeExtractor.getKeySelectorTypes(TypeExtractor.java:253)

at org.apache.flink.api.java.typeutils.TypeExtractor.getKeySelectorTypes(TypeExtractor.java:246)

at org.apache.flink.streaming.api.datastream.KeyedStream.<init>(KeyedStream.java:87)

at org.apache.flink.streaming.api.datastream.DataStream.keyBy(DataStream.java:241)

Can someone confirm this? or is there a better way to do it?

Thanks!

Andrew

Confidentiality Notice: This e-mail transmission may contain confidential or legally privileged information that is intended only for the individual or entity named in the e-mail address. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or reliance upon the contents of this e-mail is strictly prohibited and may be unlawful. If you have received this e-mail in error, please notify the sender immediately by return e-mail and delete all copies of this message.

[Stephan Ewen]

Looks like something we should fix though. Probably just needs a case distinction in the TypeExtractor.

@Andrew, can you post the stack trace into the me linked issue?

We'll try to get at it until the next release...

On Wed, Mar 30, 2016 at 10:58 AM, Chesnay Schepler <[hidden email]> wrote:

based on https://issues.apache.org/jira/browse/FLINK-3138 this is not supported for non-static methods.

On 30.03.2016 10:33, Andrew Ge Wu wrote:

Hi,

This is not very obvious and looks like a bug.

I have a lambda expression to get key from objects in stream:

This works:

stream.keyBy(value -> value.getId())

This does not:

stream.keyBy(myClass::getId)

Exception:

Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: -1

at org.apache.flink.api.java.typeutils.TypeExtractor.getUnaryOperatorReturnType(TypeExtractor.java:294)

at org.apache.flink.api.java.typeutils.TypeExtractor.getKeySelectorTypes(TypeExtractor.java:253)

at org.apache.flink.api.java.typeutils.TypeExtractor.getKeySelectorTypes(TypeExtractor.java:246)

at org.apache.flink.streaming.api.datastream.KeyedStream.<init>(KeyedStream.java:87)

at org.apache.flink.streaming.api.datastream.DataStream.keyBy(DataStream.java:241)

Can someone confirm this? or is there a better way to do it?

Thanks!

Andrew

Confidentiality Notice: This e-mail transmission may contain confidential or legally privileged information that is intended only for the individual or entity named in the e-mail address. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or reliance upon the contents of this e-mail is strictly prohibited and may be unlawful. If you have received this e-mail in error, please notify the sender immediately by return e-mail and delete all copies of this message.

[Timo Walther]

I will assign this issue to me and fix it soon, if that's ok?

Regards,
Timo


On 30.03.2016 11:30, Stephan Ewen wrote:

Looks like something we should fix though. Probably just needs a case distinction in the TypeExtractor.

@Andrew, can you post the stack trace into the me linked issue?

We'll try to get at it until the next release...

On Wed, Mar 30, 2016 at 10:58 AM, Chesnay Schepler <[hidden email]> wrote:

based on https://issues.apache.org/jira/browse/FLINK-3138 this is not supported for non-static methods.

On 30.03.2016 10:33, Andrew Ge Wu wrote:

Hi,

This is not very obvious and looks like a bug.

I have a lambda expression to get key from objects in stream:

This works:

stream.keyBy(value -> value.getId())

This does not:

stream.keyBy(myClass::getId)

Exception:

Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: -1

at org.apache.flink.api.java.typeutils.TypeExtractor.getUnaryOperatorReturnType(TypeExtractor.java:294)

at org.apache.flink.api.java.typeutils.TypeExtractor.getKeySelectorTypes(TypeExtractor.java:253)

at org.apache.flink.api.java.typeutils.TypeExtractor.getKeySelectorTypes(TypeExtractor.java:246)

at org.apache.flink.streaming.api.datastream.KeyedStream.<init>(KeyedStream.java:87)

at org.apache.flink.streaming.api.datastream.DataStream.keyBy(DataStream.java:241)

Can someone confirm this? or is there a better way to do it?

Thanks!

Andrew

Confidentiality Notice: This e-mail transmission may contain confidential or legally privileged information that is intended only for the individual or entity named in the e-mail address. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or reliance upon the contents of this e-mail is strictly prohibited and may be unlawful. If you have received this e-mail in error, please notify the sender immediately by return e-mail and delete all copies of this message.

[Andrew Ge Wu]

Thanks for the quick reply

looks like Chesnay Schepler already found the ticket

https://issues.apache.org/jira/browse/FLINK-3138

with status:

Of course if it would be great if this can be fixed soon, but this can be a bit more obvious somewhere.

Thanks!

Andrew

On 30 Mar 2016, at 12:39, Timo Walther <[hidden email]> wrote:

I will assign this issue to me and fix it soon, if that's ok?

Regards,
Timo


On 30.03.2016 11:30, Stephan Ewen wrote:

Looks like something we should fix though. Probably just needs a case distinction in the TypeExtractor.

@Andrew, can you post the stack trace into the me linked issue?

We'll try to get at it until the next release...

On Wed, Mar 30, 2016 at 10:58 AM, Chesnay Schepler <[hidden email][hidden email]> wrote:

based on https://issues.apache.org/jira/browse/FLINK-3138 this is not supported for non-static methods.

On 30.03.2016 10:33, Andrew Ge Wu wrote:

Hi,

This is not very obvious and looks like a bug.

I have a lambda expression to get key from objects in stream:

This works:

stream.keyBy(value -> value.getId())

This does not:

stream.keyBy(myClass::getId)

Exception:

Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: -1

at org.apache.flink.api.java.typeutils.TypeExtractor.getUnaryOperatorReturnType(TypeExtractor.java:294)

at org.apache.flink.api.java.typeutils.TypeExtractor.getKeySelectorTypes(TypeExtractor.java:253)

at org.apache.flink.api.java.typeutils.TypeExtractor.getKeySelectorTypes(TypeExtractor.java:246)

at org.apache.flink.streaming.api.datastream.KeyedStream.<init>(KeyedStream.java:87)

at org.apache.flink.streaming.api.datastream.DataStream.keyBy(DataStream.java:241)

Can someone confirm this? or is there a better way to do it?

Thanks!

Andrew

Confidentiality Notice: This e-mail transmission may contain confidential or legally privileged information that is intended only for the individual or entity named in the e-mail address. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or reliance upon the contents of this e-mail is strictly prohibited and may be unlawful. If you have received this e-mail in error, please notify the sender immediately by return e-mail and delete all copies of this message.

Confidentiality Notice: This e-mail transmission may contain confidential or legally privileged information that is intended only for the individual or entity named in the e-mail address. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or reliance upon the contents of this e-mail is strictly prohibited and may be unlawful. If you have received this e-mail in error, please notify the sender immediately by return e-mail and delete all copies of this message.