InvalidTypesException

[Rockstar Flo]

Hello :)

I get an InvalidTypesException while starting a stratosphere job. Could anyone help me?
The code is in the attachment of this mail. I used the Version "stratosphere-0.5.1".

I would welcome your response,
Florian Hönicke

xxx@xxx:~$ command time -o /home/hoenicke/stratosphere-0.5.1-hadoop2/zeitenKleinVernica.txt -a -f '%e' /home/hoenicke/stratosphere-0.5.1-hadoop2/bin/stratosphere run /home/hoenicke/stratosphere-0.5.1-hadoop2/jar/vernica.jar 0.9 \file:///home/hoenicke/stratosphere-0.5.1-hadoop2/input/klein file:///home/hoenicke/stratosphere-0.5.1-hadoop2/output -v
> Error: The main method caused an error.
> eu.stratosphere.client.program.ProgramInvocationException: The main method caused an error.
>         at eu.stratosphere.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:399)
>         at eu.stratosphere.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:302)
>         at eu.stratosphere.client.program.Client.run(Client.java:220)
>         at eu.stratosphere.client.CliFrontend.executeProgram(CliFrontend.java:327)
>         at eu.stratosphere.client.CliFrontend.run(CliFrontend.java:314)
>         at eu.stratosphere.client.CliFrontend.parseParameters(CliFrontend.java:927)
>         at eu.stratosphere.client.CliFrontend.main(CliFrontend.java:951)
> Caused by: eu.stratosphere.api.java.functions.InvalidTypesException: Interfaces and abstract classes are not valid types.
>         at eu.stratosphere.api.java.typeutils.TypeExtractor.getForClass(TypeExtractor.java:596)
>         at eu.stratosphere.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:508)
>         at eu.stratosphere.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:498)
>         at eu.stratosphere.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:455)
>         at eu.stratosphere.api.java.typeutils.TypeExtractor.createTypeInfo(TypeExtractor.java:150)
>         at eu.stratosphere.api.java.typeutils.TypeExtractor.getFlatMapReturnTypes(TypeExtractor.java:57)
>         at eu.stratosphere.api.java.operators.FlatMapOperator.<init>(FlatMapOperator.java:38)
>         at eu.stratosphere.api.java.DataSet.flatMap(DataSet.java:155)
>         at eu.stratosphere.quickstart.VernicaJob.main(VernicaJob.java:43)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at eu.stratosphere.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:384)
>         ... 6 more
> xxx@xxx:~$
>

[Stephan Ewen]

Hi!

Is is pretty much as the exception says. In the current state, Flink does (as Stratosphere did) support abstract types for the function signatures. The reason is that we need to do some type instruction and inference to set up our runtime utilities.

 Caused by: eu.stratosphere.api.java.functions.InvalidTypesException: Interfaces and abstract classes are not valid types

What you can do is use Type variables and bounds. This helps you with most cases:

class MyGenericFilter<T extends AbstractType> implements FilterFunction<T> {

    public boolean filter(T value) {

        // inside here, you can assume all that "AbstractType" has

    }

}

DataSet<ConcreteType> data = ...

data.filter( new MyGenericFilter<ConcreteType>() ).print();

Greetings,

Stephan

On Wed, Sep 17, 2014 at 8:18 PM, Rockstar Flo <[hidden email]> wrote:

Hello :)

I get an InvalidTypesException while starting a stratosphere job. Could anyone help me?
The code is in the attachment of this mail. I used the Version "stratosphere-0.5.1".

I would welcome your response,
Florian Hönicke

xxx@xxx:~$ command time -o /home/hoenicke/stratosphere-0.5.1-hadoop2/zeitenKleinVernica.txt -a -f '%e' /home/hoenicke/stratosphere-0.5.1-hadoop2/bin/stratosphere run /home/hoenicke/stratosphere-0.5.1-hadoop2/jar/vernica.jar 0.9 \file:///home/hoenicke/stratosphere-0.5.1-hadoop2/input/klein file:///home/hoenicke/stratosphere-0.5.1-hadoop2/output -v
> Error: The main method caused an error.
> eu.stratosphere.client.program.ProgramInvocationException: The main method caused an error.
>         at eu.stratosphere.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:399)
>         at eu.stratosphere.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:302)
>         at eu.stratosphere.client.program.Client.run(Client.java:220)
>         at eu.stratosphere.client.CliFrontend.executeProgram(CliFrontend.java:327)
>         at eu.stratosphere.client.CliFrontend.run(CliFrontend.java:314)
>         at eu.stratosphere.client.CliFrontend.parseParameters(CliFrontend.java:927)
>         at eu.stratosphere.client.CliFrontend.main(CliFrontend.java:951)
> Caused by: eu.stratosphere.api.java.functions.InvalidTypesException: Interfaces and abstract classes are not valid types.
>         at eu.stratosphere.api.java.typeutils.TypeExtractor.getForClass(TypeExtractor.java:596)
>         at eu.stratosphere.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:508)
>         at eu.stratosphere.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:498)
>         at eu.stratosphere.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:455)
>         at eu.stratosphere.api.java.typeutils.TypeExtractor.createTypeInfo(TypeExtractor.java:150)
>         at eu.stratosphere.api.java.typeutils.TypeExtractor.getFlatMapReturnTypes(TypeExtractor.java:57)
>         at eu.stratosphere.api.java.operators.FlatMapOperator.<init>(FlatMapOperator.java:38)
>         at eu.stratosphere.api.java.DataSet.flatMap(DataSet.java:155)
>         at eu.stratosphere.quickstart.VernicaJob.main(VernicaJob.java:43)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at eu.stratosphere.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:384)
>         ... 6 more
> xxx@xxx:~$
>

[Aljoscha Krettek]

This is type Signature of the FlatMap where the Exception is thrown:
public static class FlatMapPK extends FlatMapFunction<String, Tuple3<Integer, Integer, int[]>>

On the current Version it works. Did we maybe change anything and arrays didn't work yet in 0.5.x?

Aljoscha

On Wed, Sep 17, 2014 at 9:23 PM, Stephan Ewen <[hidden email]> wrote:

Hi!

Is is pretty much as the exception says. In the current state, Flink does (as Stratosphere did) support abstract types for the function signatures. The reason is that we need to do some type instruction and inference to set up our runtime utilities.

 Caused by: eu.stratosphere.api.java.functions.InvalidTypesException: Interfaces and abstract classes are not valid types

What you can do is use Type variables and bounds. This helps you with most cases:

class MyGenericFilter<T extends AbstractType> implements FilterFunction<T> {

    public boolean filter(T value) {

        // inside here, you can assume all that "AbstractType" has

    }

}

DataSet<ConcreteType> data = ...

data.filter( new MyGenericFilter<ConcreteType>() ).print();

Greetings,

Stephan

On Wed, Sep 17, 2014 at 8:18 PM, Rockstar Flo <[hidden email]> wrote:

Hello :)

I get an InvalidTypesException while starting a stratosphere job. Could anyone help me?
The code is in the attachment of this mail. I used the Version "stratosphere-0.5.1".

I would welcome your response,
Florian Hönicke

xxx@xxx:~$ command time -o /home/hoenicke/stratosphere-0.5.1-hadoop2/zeitenKleinVernica.txt -a -f '%e' /home/hoenicke/stratosphere-0.5.1-hadoop2/bin/stratosphere run /home/hoenicke/stratosphere-0.5.1-hadoop2/jar/vernica.jar 0.9 \file:///home/hoenicke/stratosphere-0.5.1-hadoop2/input/klein file:///home/hoenicke/stratosphere-0.5.1-hadoop2/output -v
> Error: The main method caused an error.
> eu.stratosphere.client.program.ProgramInvocationException: The main method caused an error.
>         at eu.stratosphere.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:399)
>         at eu.stratosphere.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:302)
>         at eu.stratosphere.client.program.Client.run(Client.java:220)
>         at eu.stratosphere.client.CliFrontend.executeProgram(CliFrontend.java:327)
>         at eu.stratosphere.client.CliFrontend.run(CliFrontend.java:314)
>         at eu.stratosphere.client.CliFrontend.parseParameters(CliFrontend.java:927)
>         at eu.stratosphere.client.CliFrontend.main(CliFrontend.java:951)
> Caused by: eu.stratosphere.api.java.functions.InvalidTypesException: Interfaces and abstract classes are not valid types.
>         at eu.stratosphere.api.java.typeutils.TypeExtractor.getForClass(TypeExtractor.java:596)
>         at eu.stratosphere.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:508)
>         at eu.stratosphere.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:498)
>         at eu.stratosphere.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:455)
>         at eu.stratosphere.api.java.typeutils.TypeExtractor.createTypeInfo(TypeExtractor.java:150)
>         at eu.stratosphere.api.java.typeutils.TypeExtractor.getFlatMapReturnTypes(TypeExtractor.java:57)
>         at eu.stratosphere.api.java.operators.FlatMapOperator.<init>(FlatMapOperator.java:38)
>         at eu.stratosphere.api.java.DataSet.flatMap(DataSet.java:155)
>         at eu.stratosphere.quickstart.VernicaJob.main(VernicaJob.java:43)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at eu.stratosphere.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:384)
>         ... 6 more
> xxx@xxx:~$
>

[Rockstar Flo]

Problem solved! :)

Thanks a lot. I changed "int[]" to "Integer[]" and now it works.

Florian

Am 18.09.2014 07:07, schrieb Aljoscha Krettek:

This is type Signature of the FlatMap where the Exception is thrown:
public static class FlatMapPK extends FlatMapFunction<String, Tuple3<Integer, Integer, int[]>>

On the current Version it works. Did we maybe change anything and arrays didn't work yet in 0.5.x?

Aljoscha

On Wed, Sep 17, 2014 at 9:23 PM, Stephan Ewen <[hidden email]> wrote:

Hi!

Is is pretty much as the exception says. In the current state, Flink does (as Stratosphere did) support abstract types for the function signatures. The reason is that we need to do some type instruction and inference to set up our runtime utilities.

 Caused by: eu.stratosphere.api.java.functions.InvalidTypesException: Interfaces and abstract classes are not valid types

What you can do is use Type variables and bounds. This helps you with most cases:

class MyGenericFilter<T extends AbstractType> implements FilterFunction<T> {

    public boolean filter(T value) {

        // inside here, you can assume all that "AbstractType" has

    }

}

DataSet<ConcreteType> data = ...

data.filter( new MyGenericFilter<ConcreteType>() ).print();

Greetings,

Stephan

On Wed, Sep 17, 2014 at 8:18 PM, Rockstar Flo <[hidden email]> wrote:

Hello :)

I get an InvalidTypesException while starting a stratosphere job. Could anyone help me?
The code is in the attachment of this mail. I used the Version "stratosphere-0.5.1".

I would welcome your response,
Florian Hönicke

xxx@xxx:~$ command time -o /home/hoenicke/stratosphere-0.5.1-hadoop2/zeitenKleinVernica.txt -a -f '%e' /home/hoenicke/stratosphere-0.5.1-hadoop2/bin/stratosphere run /home/hoenicke/stratosphere-0.5.1-hadoop2/jar/vernica.jar 0.9 \file:///home/hoenicke/stratosphere-0.5.1-hadoop2/input/klein file:///home/hoenicke/stratosphere-0.5.1-hadoop2/output -v
> Error: The main method caused an error.
> eu.stratosphere.client.program.ProgramInvocationException: The main method caused an error.
>         at eu.stratosphere.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:399)
>         at eu.stratosphere.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:302)
>         at eu.stratosphere.client.program.Client.run(Client.java:220)
>         at eu.stratosphere.client.CliFrontend.executeProgram(CliFrontend.java:327)
>         at eu.stratosphere.client.CliFrontend.run(CliFrontend.java:314)
>         at eu.stratosphere.client.CliFrontend.parseParameters(CliFrontend.java:927)
>         at eu.stratosphere.client.CliFrontend.main(CliFrontend.java:951)
> Caused by: eu.stratosphere.api.java.functions.InvalidTypesException: Interfaces and abstract classes are not valid types.
>         at eu.stratosphere.api.java.typeutils.TypeExtractor.getForClass(TypeExtractor.java:596)
>         at eu.stratosphere.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:508)
>         at eu.stratosphere.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:498)
>         at eu.stratosphere.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:455)
>         at eu.stratosphere.api.java.typeutils.TypeExtractor.createTypeInfo(TypeExtractor.java:150)
>         at eu.stratosphere.api.java.typeutils.TypeExtractor.getFlatMapReturnTypes(TypeExtractor.java:57)
>         at eu.stratosphere.api.java.operators.FlatMapOperator.<init>(FlatMapOperator.java:38)
>         at eu.stratosphere.api.java.DataSet.flatMap(DataSet.java:155)
>         at eu.stratosphere.quickstart.VernicaJob.main(VernicaJob.java:43)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at eu.stratosphere.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:384)
>         ... 6 more
> xxx@xxx:~$
>

[Aljoscha Krettek]

In newer versions it also works with int[]. :D

On Thu, Sep 18, 2014 at 12:28 PM, Florian Hönicke <[hidden email]> wrote:

Problem solved! :)

Thanks a lot. I changed "int[]" to "Integer[]" and now it works.

Florian

Am 18.09.2014 07:07, schrieb Aljoscha Krettek:

This is type Signature of the FlatMap where the Exception is thrown:
public static class FlatMapPK extends FlatMapFunction<String, Tuple3<Integer, Integer, int[]>>

On the current Version it works. Did we maybe change anything and arrays didn't work yet in 0.5.x?

Aljoscha

On Wed, Sep 17, 2014 at 9:23 PM, Stephan Ewen <[hidden email]> wrote:

Hi!

Is is pretty much as the exception says. In the current state, Flink does (as Stratosphere did) support abstract types for the function signatures. The reason is that we need to do some type instruction and inference to set up our runtime utilities.

 Caused by: eu.stratosphere.api.java.functions.InvalidTypesException: Interfaces and abstract classes are not valid types

What you can do is use Type variables and bounds. This helps you with most cases:

class MyGenericFilter<T extends AbstractType> implements FilterFunction<T> {

    public boolean filter(T value) {

        // inside here, you can assume all that "AbstractType" has

    }

}

DataSet<ConcreteType> data = ...

data.filter( new MyGenericFilter<ConcreteType>() ).print();

Greetings,

Stephan

On Wed, Sep 17, 2014 at 8:18 PM, Rockstar Flo <[hidden email]> wrote:

Hello :)

I get an InvalidTypesException while starting a stratosphere job. Could anyone help me?
The code is in the attachment of this mail. I used the Version "stratosphere-0.5.1".

I would welcome your response,
Florian Hönicke

xxx@xxx:~$ command time -o /home/hoenicke/stratosphere-0.5.1-hadoop2/zeitenKleinVernica.txt -a -f '%e' /home/hoenicke/stratosphere-0.5.1-hadoop2/bin/stratosphere run /home/hoenicke/stratosphere-0.5.1-hadoop2/jar/vernica.jar 0.9 \file:///home/hoenicke/stratosphere-0.5.1-hadoop2/input/klein file:///home/hoenicke/stratosphere-0.5.1-hadoop2/output -v
> Error: The main method caused an error.
> eu.stratosphere.client.program.ProgramInvocationException: The main method caused an error.
>         at eu.stratosphere.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:399)
>         at eu.stratosphere.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:302)
>         at eu.stratosphere.client.program.Client.run(Client.java:220)
>         at eu.stratosphere.client.CliFrontend.executeProgram(CliFrontend.java:327)
>         at eu.stratosphere.client.CliFrontend.run(CliFrontend.java:314)
>         at eu.stratosphere.client.CliFrontend.parseParameters(CliFrontend.java:927)
>         at eu.stratosphere.client.CliFrontend.main(CliFrontend.java:951)
> Caused by: eu.stratosphere.api.java.functions.InvalidTypesException: Interfaces and abstract classes are not valid types.
>         at eu.stratosphere.api.java.typeutils.TypeExtractor.getForClass(TypeExtractor.java:596)
>         at eu.stratosphere.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:508)
>         at eu.stratosphere.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:498)
>         at eu.stratosphere.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:455)
>         at eu.stratosphere.api.java.typeutils.TypeExtractor.createTypeInfo(TypeExtractor.java:150)
>         at eu.stratosphere.api.java.typeutils.TypeExtractor.getFlatMapReturnTypes(TypeExtractor.java:57)
>         at eu.stratosphere.api.java.operators.FlatMapOperator.<init>(FlatMapOperator.java:38)
>         at eu.stratosphere.api.java.DataSet.flatMap(DataSet.java:155)
>         at eu.stratosphere.quickstart.VernicaJob.main(VernicaJob.java:43)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at eu.stratosphere.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:384)
>         ... 6 more
> xxx@xxx:~$
>