Keyed function type erasure problem.

[yunfan123]

Just small change from PojoExample:
My problem is why the class like SelectorContainer can't as keyed function in flink
Full code please look up PojoExample.java
<http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/file/t921/PojoExample.java> 
public class PojoExample<KEY> {
        private SelectorContainer<KEY> selectorContainer;
        // KeySelector class like this can't be used in flink !!!
        public static class SelectorContainer<K> implements KeySelector<Word ,K> {
                public KeySelector<Word, K> keySelector;
                public SelectorContainer(KeySelector<Word, K> keySelector) {
                        this.keySelector = keySelector;
                }

                @Override
                public K getKey(Word value) throws Exception {
                        return keySelector.getKey(value);
                }
        }
        public void run() {
              dataStream.keyed(selectorContainer);
        }
       
}

This will cause:
Type of TypeVariable 'K' in 'class
org.apache.flink.streaming.examples.wordcount.PojoExample$SelectorContainer'
could not be determined. This is most likely a type erasure problem. The
type extraction currently supports types with generic variables only in
cases where all variables in the return type can be deduced from the input
type(s).

org.apache.flink.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:915)

org.apache.flink.api.java.typeutils.TypeExtractor.privateCreateTypeInfo(TypeExtractor.java:836)

org.apache.flink.api.java.typeutils.TypeExtractor.getUnaryOperatorReturnType(TypeExtractor.java:622)

org.apache.flink.api.java.typeutils.TypeExtractor.getKeySelectorTypes(TypeExtractor.java:443)

org.apache.flink.api.java.typeutils.TypeExtractor.getKeySelectorTypes(TypeExtractor.java:436)

org.apache.flink.streaming.api.datastream.KeyedStream.<init>(KeyedStream.java:108)

org.apache.flink.streaming.api.datastream.DataStream.keyBy(DataStream.java:263)

org.apache.flink.streaming.examples.wordcount.PojoExample.execute(PojoExample.java:86)

org.apache.flink.streaming.examples.wordcount.PojoExample.main(PojoExample.java:108)

It sames can't use a class contains the KeySelector ?



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[Fabian Hueske-2]

Hi,

The problem is that Flink cannot determine the key type produced by the KeySelector because the type is a generic.

The type information is lost at runtime due to Java's type erasure.

You can try to implement the ResultTypeQueryable interface with the KeySelector and tell Flink the key type directly.

Best,

Fabian

2017-09-11 12:38 GMT+02:00 yunfan123 <[hidden email]>:

Just small change from PojoExample:
PojoExample.java
<http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/file/t921/PojoExample.java>
public class PojoExample<KEY> {
        private SelectorContainer<KEY> selectorContainer;
        // KeySelector class like this can't be used in flink !!!
        public static class SelectorContainer<K> implements KeySelector<Word ,K> {
                public KeySelector<Word, K> keySelector;
                public SelectorContainer(KeySelector<Word, K> keySelector) {
                        this.keySelector = keySelector;
                }

                @Override
                public K getKey(Word value) throws Exception {
                        return keySelector.getKey(value);
                }
        }
        public void run() {
              dataStream.keyed(selectorContainer);
        }

}

This will cause:
Type of TypeVariable 'K' in 'class
org.apache.flink.streaming.examples.wordcount.PojoExample$SelectorContainer'
could not be determined. This is most likely a type erasure problem. The
type extraction currently supports types with generic variables only in
cases where all variables in the return type can be deduced from the input
type(s).

org.apache.flink.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:915)

org.apache.flink.api.java.typeutils.TypeExtractor.privateCreateTypeInfo(TypeExtractor.java:836)

org.apache.flink.api.java.typeutils.TypeExtractor.getUnaryOperatorReturnType(TypeExtractor.java:622)

org.apache.flink.api.java.typeutils.TypeExtractor.getKeySelectorTypes(TypeExtractor.java:443)

org.apache.flink.api.java.typeutils.TypeExtractor.getKeySelectorTypes(TypeExtractor.java:436)

org.apache.flink.streaming.api.datastream.KeyedStream.<init>(KeyedStream.java:108)

org.apache.flink.streaming.api.datastream.DataStream.keyBy(DataStream.java:263)

org.apache.flink.streaming.examples.wordcount.PojoExample.execute(PojoExample.java:86)

org.apache.flink.streaming.examples.wordcount.PojoExample.main(PojoExample.java:108)

It sames can't use a class contains the KeySelector ?



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[yunfan123]

Thanks



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/