Hi,
My Filter operator in Scala encounters that IDE indicates the data type error:
"overloaded method value filter with alternatives: (fun: ((String, Int)) ⇒ Boolean cannot be applied..."
Scala for me is quite new. I'm thinking the problem comes from the type doesn't match each other in map operator and filter operator:
val pldIndex = GraphUtils.readVertices(PLDIndexFile).map { vertex => Tuple2(vertex.annotation, vertex.id) }
class PruneToSampleVerticesFilter extends RichFilterFunction[AnnotatedVertex]
Did you have any suggestion that in which direction should I work on to solve this?
One thing makes me confused is the difference of "map" in the following and above(though there is still an error in filter operator):
val pldIndex = GraphUtils.readVertices(PLDIndexFile).map { vertex => AnnotatedVertex}
or
val pldIndex = GraphUtils.readVertices(PLDIndexFile).map { vertex => Tuple1<AnnotatedVertex>}
I'm thinking since the input of "filter" is AnnotatedVertex, would it be better to "map" to AnnotatedVertex?
Best regards,
Hung
----------------------------------------------------------------------------------------------------------
def compute(trackingGraphFile: String, PLDIndexFile: String, outputPath: String) = {
implicit val env = ExecutionEnvironment.getExecutionEnvironment
// Vertex => VertexName, VertexID
val pldIndex = GraphUtils.readVertices(PLDIndexFile).map { vertex => Tuple2(vertex.annotation, vertex.id) }
val trackingGraphEdges = GraphUtils.readEdges(trackingGraphFile)
val trackingGraphVertices = trackingGraphEdges.map { edge => Tuple1(edge.target) }
.distinct
// IDE complains this filter
val prunedPldIndex = pldIndex.filter(new PruneToSampleVerticesFilter)
.withBroadcastSet(trackingGraphVertices, "vertices")
prunedPldIndex.writeAsCsv(outputPath, fieldDelimiter = "\t", writeMode = WriteMode.OVERWRITE)
env.execute()
}
class PruneToSampleVerticesFilter extends RichFilterFunction[AnnotatedVertex] {
var vertices: Set[Int] = null
override def open(parameters: Configuration) = {
vertices = getRuntimeContext.getBroadcastVariable[Tuple1[Int]]("vertices").map { _._1 }
.toSet
}
override def filter(vertex: AnnotatedVertex): Boolean = {
vertices.contains(vertex.id)
}
}
------------------------------------------------------------------------------------------------------------------
// case class
case class AnnotatedVertex(annotation: String, id: Int)
case class Edge(src: Int, target: Int)
Locked
