Type of TypeVariable 'K' in 'class <> could not be determined

[otherwise777]

I get this error:

Exception in thread "main" org.apache.flink.api.common.functions.InvalidTypesException: Type of TypeVariable 'K' in 'class Tgraphs.SingleSourceShortestTemporalPathEAT3$InitVerticesMapper' could not be determined. This is most likely a type erasure problem. The type extraction currently supports types with generic variables only in cases where all variables in the return type can be deduced from the input type(s).

When i run my function:

tempgraphdoubles.run(new SingleSourceShortestTemporalPathEAT3<String,NullValue>("A",maxIterations)).print();

The function is here: http://paste.thezomg.com/19924/93864391/

I've narrowed it down to the Tuple2<K,Double> on line 44, apperently it doesn't like the K being there in a Tuple2<>

I couldn't find out why though, and how would i make a workaround for this when i a want to use this scenario?

[Timo Walther]

Hi,

Flink uses the signature of the class to determine the return types of a
function ("class MyClass implements MapFunction<IN, OUT>"). You always
have to keep in mind that every generic in a "new
XXXX<String,NullValue>()" gets erasured by Java to "new XXXX()". So
variable K has to be determined by the input type (which is always known).

It would be great if you could also share " TGraphAlgorithm" and all
classes till Function in the hierarchy. According to your comment in "
SingleSourceShortestTemporalPathEAT3" K is in the input type, so this
could be a bug. Have you tried the latest snapshot release? There were
some type inference improvements lately.

Regards,
Timo


Am 17/11/16 um 13:31 schrieb otherwise777:

> I get this error:
>
> *Exception in thread "main"
> org.apache.flink.api.common.functions.InvalidTypesException: Type of
> TypeVariable 'K' in 'class
> Tgraphs.SingleSourceShortestTemporalPathEAT3$InitVerticesMapper' could not
> be determined. This is most likely a type erasure problem. The type
> extraction currently supports types with generic variables only in cases
> where all variables in the return type can be deduced from the input
> type(s).*
>
> When i run my function:
>
> *tempgraphdoubles.run(new
> SingleSourceShortestTemporalPathEAT3<String,NullValue>("A",maxIterations)).print();*
>
> The function is here:  http://paste.thezomg.com/19924/93864391/
> <http://paste.thezomg.com/19924/93864391/>
>
> I've narrowed it down to the Tuple2<K,Double> on line 44, apperently it
> doesn't like the K being there in a Tuple2<>
>
> I couldn't find out why though, and how would i make a workaround for this
> when i a want to use this scenario?
>
>
>
> --
> View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Type-of-TypeVariable-K-in-class-could-not-be-determined-tp10173.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.

[otherwise777]

Hello timo,

the whole project is on github: https://github.com/otherwise777/Temporal_Graph_library
The Tgraphalgorithm is here: https://github.com/otherwise777/Temporal_Graph_library/blob/master/src/main/java/Tgraphs/TGraphAlgorithm.java

I just updated Flink and Gelly to 1.1.3 with Maven but the problem still occurs

[Vasiliki Kalavri]

Hi Wouter,

with InitVerticesMapper() are you trying to map the vertex value to a Tuple2<K, Double> or to a Double?

Your mapper is turning the vertex values into a Tuple2<> but your scatter-gather UDFs are defining Double vertex values.

-Vasia.

On 17 November 2016 at 14:03, otherwise777 <[hidden email]> wrote:

Hello timo,

the whole project is on github:
https://github.com/otherwise777/Temporal_Graph_library
<https://github.com/otherwise777/Temporal_Graph_library>
The Tgraphalgorithm is here:
https://github.com/otherwise777/Temporal_Graph_library/blob/master/src/main/java/Tgraphs/TGraphAlgorithm.java
<https://github.com/otherwise777/Temporal_Graph_library/blob/master/src/main/java/Tgraphs/TGraphAlgorithm.java>

I just updated Flink and Gelly to 1.1.3 with Maven but the problem still
occurs



--
View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Type-of-TypeVariable-K-in-class-could-not-be-determined-tp10173p10175.html

Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.

[otherwise777]

Hey Vasia,

I made this simple mapper to illustrate the problem, the file i'm working on is here:
https://github.com/otherwise777/Temporal_Graph_library/blob/master/src/main/java/Tgraphs/SingleSourceShortestTemporalPathEATBetweenness.java

Which uses a Tuple3<K,Double,ArrayList<K>>

Anyhow, you can ignore the scatter-gather UDFs from the file that i posted

[Timo Walther]

I checked out your repository. Which test do I have to run in order to
get the error?

Am 17/11/16 um 14:33 schrieb otherwise777:

> Hey Vasia,
>
> I made this simple mapper to illustrate the problem, the file i'm working on
> is here:
> https://github.com/otherwise777/Temporal_Graph_library/blob/master/src/main/java/Tgraphs/SingleSourceShortestTemporalPathEATBetweenness.java
> <https://github.com/otherwise777/Temporal_Graph_library/blob/master/src/main/java/Tgraphs/SingleSourceShortestTemporalPathEATBetweenness.java>
>
> Which uses a Tuple3<K,Double,ArrayList&lt;K>>
>
> Anyhow, you can ignore the scatter-gather UDFs from the file that i posted
>
>
>
> --
> View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Type-of-TypeVariable-K-in-class-could-not-be-determined-tp10173p10177.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.

--
Freundliche Grüße / Kind Regards

Timo Walther

Follow me: @twalthr
https://www.linkedin.com/in/twalthr

[otherwise777]

Sorry i already pushed a new update,
But in testclass.java if you change line 266 to:
tempgraphdoubles.run(new SingleSourceShortestTemporalPathEAT3<String,NullValue>(maxIterations)).print();

And then run the testclass.java You should get the error

[otherwise777]

The one that's currently in my github will give you the error,

In my other file i made a really ugly workaround by adding the element in an ArrayList<K> as a single item.

[Timo Walther]

I think I identified the problem. Input type inference can not be used
because of missing information.

@Vasia: Why is the TypeExtractor in Graph. mapVertices(mapper) called
without information about the input type? Isn't the input of the
MapFunction known at this point? ( vertices.getType())

Am 17/11/16 um 20:24 schrieb otherwise777:

> The one that's currently in my github will give you the error,
>
> In my other file i made a really ugly workaround by adding the element in an
> ArrayList<K> as a single item.
>
>
>
> --
> View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Type-of-TypeVariable-K-in-class-could-not-be-determined-tp10173p10184.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.

[Vasiliki Kalavri]

Hi Timo,

thanks for looking into this! Are you referring to the 4th argument in [1]?

Thanks,

-Vasia.

[1]: https://github.com/apache/flink/blob/master/flink-libraries/flink-gelly/src/main/java/org/apache/flink/graph/Graph.java#L506

On 18 November 2016 at 10:25, Timo Walther <[hidden email]> wrote:

I think I identified the problem. Input type inference can not be used because of missing information.

@Vasia: Why is the TypeExtractor in Graph. mapVertices(mapper) called without information about the input type? Isn't the input of the MapFunction known at this point? ( vertices.getType())

Am 17/11/16 um 20:24 schrieb otherwise777:

The one that's currently in my github will give you the error,

In my other file i made a really ugly workaround by adding the element in an
ArrayList<K> as a single item.



--
View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Type-of-TypeVariable-K-in-class-could-not-be-determined-tp10173p10184.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.

[Timo Walther]

Yes. I don't know if it solve this problem but in general if the input type is known it should be passed for input type inference.

Am 18/11/16 um 11:28 schrieb Vasiliki Kalavri:

Hi Timo,

thanks for looking into this! Are you referring to the 4th argument in [1]?

Thanks,

-Vasia.

[1]: https://github.com/apache/flink/blob/master/flink-libraries/flink-gelly/src/main/java/org/apache/flink/graph/Graph.java#L506

On 18 November 2016 at 10:25, Timo Walther <[hidden email]> wrote:

I think I identified the problem. Input type inference can not be used because of missing information.

@Vasia: Why is the TypeExtractor in Graph. mapVertices(mapper) called without information about the input type? Isn't the input of the MapFunction known at this point? ( vertices.getType())

Am 17/11/16 um 20:24 schrieb otherwise777:

The one that's currently in my github will give you the error,

In my other file i made a really ugly workaround by adding the element in an
ArrayList<K> as a single item.



--
View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Type-of-TypeVariable-K-in-class-could-not-be-determined-tp10173p10184.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.

[Vasiliki Kalavri]

Thanks Timo for the explanation and Wouter for reporting this. I've located two more instances of this in the Graph class and created FLINK5097. I'll ping you after I open the PR if that's OK.

-Vasia.

On 18 November 2016 at 12:22, Timo Walther <[hidden email]> wrote:

Yes. I don't know if it solve this problem but in general if the input type is known it should be passed for input type inference.

Am 18/11/16 um 11:28 schrieb Vasiliki Kalavri:

Hi Timo,

thanks for looking into this! Are you referring to the 4th argument in [1]?

Thanks,

-Vasia.

[1]: https://github.com/apache/flink/blob/master/flink-libraries/flink-gelly/src/main/java/org/apache/flink/graph/Graph.java#L506

On 18 November 2016 at 10:25, Timo Walther <[hidden email]> wrote:

I think I identified the problem. Input type inference can not be used because of missing information.

@Vasia: Why is the TypeExtractor in Graph. mapVertices(mapper) called without information about the input type? Isn't the input of the MapFunction known at this point? ( vertices.getType())

Am 17/11/16 um 20:24 schrieb otherwise777:

The one that's currently in my github will give you the error,

In my other file i made a really ugly workaround by adding the element in an
ArrayList<K> as a single item.



--
View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Type-of-TypeVariable-K-in-class-could-not-be-determined-tp10173p10184.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.