http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/flink-one-transformation-end-the-next-transformation-start-tp12457.html
hi,all,
i run a job,it is :
---------------------------------------------------------
val data = env.readTextFile("hdfs:///....")//DataSet[(String,Array[String])]
val dataVec = computeDataVect(data)//DataSet[(String,Int,Array[(Int,Double)])]
val rescomm = computeCosSims (dataVec)//DataSet[(String,Array[(String,Double)])]
but when run on the yarn cluster,the result was error,the job can success;and run on the local,in eclipse on my computer,the result is correct.
so,i run twice,
first:
val data = env.readTextFile("hdfs:///....")//DataSet[(String,Array[String])]
val dataVec = computeDataVect(data)//DataSet[(String,Int,Array[(Int,Double)])]
dataVec.writeAsText("hdfs///vec")//the vector is correct,
second:
val readVec = env.readTextFile("hdfs:///vec").map(...)//DataSet[(String,Int,Array[(Int,Double)])]
val rescomm = computeCosSims (dataVec)//DataSet[(String,Array[(String,Double)])]
and the result is correct,is the same as on local,in eclispe.
----------------------------------
someone can solve the problem?