Flink cannot compile my join code...

[Sebastian Schelter]

Hi,

I'm trying to write a simple join in flink-scala but for some reasons
flink fails to compile my code. I've tried several reformulations but
can't get rid of the error. Can you tell me how to fix this piece of
code? I'm using flink 0.8.0.

Sebastian

---------------------------------------------------------------------

Error:(24, 53) not enough arguments for method apply: (implicit
evidence$1:
org.apache.flink.api.common.typeinfo.TypeInformation[(String, Long)],
implicit evidence$2: scala.reflect.ClassTag[(String,
Long)])org.apache.flink.api.scala.DataSet[(String, Long)] in class
JoinDataSet.
Unspecified value parameters evidence$1, evidence$2.
       topTrackers.join(domains).where(0).equalTo(1) { (topTracker,
domain) => domain.annotation -> topTracker._2 }
                                                     ^
Error:(24, 53) could not find implicit value for evidence parameter of
type org.apache.flink.api.common.typeinfo.TypeInformation[(String, Long)]
       topTrackers.join(domains).where(0).equalTo(1) { (topTracker,
domain) => domain.annotation -> topTracker._2 }
                                                     ^
---------------------------------------------------------------------

def computeDistribution(trackingGraphFile: String,
     domainIndexFile: String) = {

   implicit val env = ExecutionEnvironment.getExecutionEnvironment

   val edges = GraphUtils.readEdges(trackingGraphFile)
   val domains = GraphUtils.readVertices(domainIndexFile)

   val trackersWithNumDomainsTracked =
     FlinkUtils.countByKey(edges, { edge: Edge => edge.src })

   val topTrackers =
     trackersWithNumDomainsTracked.filter { _._2 >= 10000 }

   val topTrackerDomains =
       topTrackers.join(domains).where(0).equalTo(1) {
         (topTracker, domain) => domain.annotation -> topTracker._2 }

   topTrackerDomains.print()

   env.execute()
}

[Aljoscha Krettek]

Hi,
do you have this import?

import org.apache.flink.api.scala._

You could also just import
org.apache.flink.api.scala.createTypeInformation but most of the other
stuff in the package is required anyways.

Cheers,
Aljoscha

On Tue, Feb 10, 2015 at 10:05 AM, Sebastian <[hidden email]> wrote:

> Hi,
>
> I'm trying to write a simple join in flink-scala but for some reasons flink
> fails to compile my code. I've tried several reformulations but can't get
> rid of the error. Can you tell me how to fix this piece of code? I'm using
> flink 0.8.0.
>
> Sebastian
>
> ---------------------------------------------------------------------
>
> Error:(24, 53) not enough arguments for method apply: (implicit evidence$1:
> org.apache.flink.api.common.typeinfo.TypeInformation[(String, Long)],
> implicit evidence$2: scala.reflect.ClassTag[(String,
> Long)])org.apache.flink.api.scala.DataSet[(String, Long)] in class
> JoinDataSet.
> Unspecified value parameters evidence$1, evidence$2.
>       topTrackers.join(domains).where(0).equalTo(1) { (topTracker, domain)
> => domain.annotation -> topTracker._2 }
>                                                     ^
> Error:(24, 53) could not find implicit value for evidence parameter of type
> org.apache.flink.api.common.typeinfo.TypeInformation[(String, Long)]
>       topTrackers.join(domains).where(0).equalTo(1) { (topTracker, domain)
> => domain.annotation -> topTracker._2 }
>                                                     ^
> ---------------------------------------------------------------------
>
> def computeDistribution(trackingGraphFile: String,
>     domainIndexFile: String) = {
>
>   implicit val env = ExecutionEnvironment.getExecutionEnvironment
>
>   val edges = GraphUtils.readEdges(trackingGraphFile)
>   val domains = GraphUtils.readVertices(domainIndexFile)
>
>   val trackersWithNumDomainsTracked =
>     FlinkUtils.countByKey(edges, { edge: Edge => edge.src })
>
>   val topTrackers =
>     trackersWithNumDomainsTracked.filter { _._2 >= 10000 }
>
>   val topTrackerDomains =
>       topTrackers.join(domains).where(0).equalTo(1) {
>         (topTracker, domain) => domain.annotation -> topTracker._2 }
>
>   topTrackerDomains.print()
>
>   env.execute()
> }
>

[Sebastian Schelter]

Yes, the import was missing thank you for the hint!

Now I'm getting the following error:

java.io.IOException: java.lang.NoSuchMethodError:
org.apache.flink.util.ClassUtils.isPrimitiveOrBoxedOrString(Ljava/lang/Class;)Z
        at org.apache.flink.runtime.ipc.RPC$Server.call(RPC.java:428)
        at org.apache.flink.runtime.ipc.Server$Handler.run(Server.java:949)



On 10.02.2015 10:19, Aljoscha Krettek wrote:
> import org.apache.flink.api.scala._

[rmetzger0]

Hi,

the NoSuchMethodError exception indicates that there is a mixup of Flink versions.

Are you running the job from within your IDE or are you submitting it to a cluster? Maybe the cluster has a different Flink version?

On Tue, Feb 10, 2015 at 10:51 AM, Sebastian <[hidden email]> wrote:

Yes, the import was missing thank you for the hint!

Now I'm getting the following error:

java.io.IOException: java.lang.NoSuchMethodError: org.apache.flink.util.ClassUtils.isPrimitiveOrBoxedOrString(Ljava/lang/Class;)Z
        at org.apache.flink.runtime.ipc.RPC$Server.call(RPC.java:428)
        at org.apache.flink.runtime.ipc.Server$Handler.run(Server.java:949)



On 10.02.2015 10:19, Aljoscha Krettek wrote:

import org.apache.flink.api.scala._

[Sebastian Schelter]

I'm running it from within my IDE...

On 10.02.2015 11:17, Robert Metzger wrote:

> Hi,
> the NoSuchMethodError exception indicates that there is a mixup of Flink
> versions.
> Are you running the job from within your IDE or are you submitting it to
> a cluster? Maybe the cluster has a different Flink version?
>
> On Tue, Feb 10, 2015 at 10:51 AM, Sebastian <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Yes, the import was missing thank you for the hint!
>
>     Now I'm getting the following error:
>
>     java.io.IOException: java.lang.NoSuchMethodError:
>     org.apache.flink.util.__ClassUtils.__isPrimitiveOrBoxedOrString(__Ljava/lang/Class;)Z
>              at org.apache.flink.runtime.ipc.__RPC$Server.call(RPC.java:428)
>              at
>     org.apache.flink.runtime.ipc.__Server$Handler.run(Server.__java:949)
>
>
>
>     On 10.02.2015 10:19, Aljoscha Krettek wrote:
>
>         import org.apache.flink.api.scala._
>
>

[Stephan Ewen]

This sounds very much like a version mixup. Can you check that you reference the same version for core, clients, runtime, ...?

The exception is from the 0.8 RPC service. Are you using 0.8 everywhere?

On Tue, Feb 10, 2015 at 12:09 PM, Sebastian <[hidden email]> wrote:

I'm running it from within my IDE...

On 10.02.2015 11:17, Robert Metzger wrote:

Hi,
the NoSuchMethodError exception indicates that there is a mixup of Flink
versions.
Are you running the job from within your IDE or are you submitting it to
a cluster? Maybe the cluster has a different Flink version?

On Tue, Feb 10, 2015 at 10:51 AM, Sebastian <[hidden email]
<mailto:[hidden email]>> wrote:

    Yes, the import was missing thank you for the hint!

    Now I'm getting the following error:

    java.io.IOException: java.lang.NoSuchMethodError:
    org.apache.flink.util.__ClassUtils.__isPrimitiveOrBoxedOrString(__Ljava/lang/Class;)Z
             at org.apache.flink.runtime.ipc.__RPC$Server.call(RPC.java:428)
             at
    org.apache.flink.runtime.ipc.__Server$Handler.run(Server.__java:949)



    On 10.02.2015 10:19, Aljoscha Krettek wrote:

        import org.apache.flink.api.scala._

[Sebastian Schelter]

Ah, I found an old jar in my classpath, thanks for the hint!

On 10.02.2015 12:12, Stephan Ewen wrote:

> This sounds very much like a version mixup. Can you check that you
> reference the same version for core, clients, runtime, ...?
>
> The exception is from the 0.8 RPC service. Are you using 0.8 everywhere?
>
> On Tue, Feb 10, 2015 at 12:09 PM, Sebastian <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     I'm running it from within my IDE...
>
>     On 10.02.2015 11:17, Robert Metzger wrote:
>
>         Hi,
>         the NoSuchMethodError exception indicates that there is a mixup
>         of Flink
>         versions.
>         Are you running the job from within your IDE or are you
>         submitting it to
>         a cluster? Maybe the cluster has a different Flink version?
>
>         On Tue, Feb 10, 2015 at 10:51 AM, Sebastian
>         <[hidden email] <mailto:[hidden email]>
>         <mailto:ssc.open@googlemail.__com
>         <mailto:[hidden email]>>> wrote:
>
>              Yes, the import was missing thank you for the hint!
>
>              Now I'm getting the following error:
>
>              java.io.IOException: java.lang.NoSuchMethodError:
>
>         org.apache.flink.util.____ClassUtils.____isPrimitiveOrBoxedOrString(____Ljava/lang/Class;)Z
>                       at
>         org.apache.flink.runtime.ipc.____RPC$Server.call(RPC.java:428)
>                       at
>
>         org.apache.flink.runtime.ipc.____Server$Handler.run(Server.____java:949)
>
>
>
>              On 10.02.2015 10:19, Aljoscha Krettek wrote:
>
>                  import org.apache.flink.api.scala._
>
>
>