FlinkML - Fail to execute QuickStart example

[Thomas FOURNIER]

Hi,

Executing the following code (see QuickStart):

val env = ExecutionEnvironment.getExecutionEnvironment
val survival = env.readCsvFile[(String, String, String, String)]("src/main/resources/haberman.data", ",")


val survivalLV = survival
  .map { tuple =>
    val list = tuple.productIterator.toList
    val numList = list.map(_.asInstanceOf[String].toDouble)
    LabeledVector(numList(3), DenseVector(numList.take(3).toArray))
  }



val astroTrain = MLUtils.readLibSVM(env, "src/main/resources/svmguide1")
val astroTest = MLUtils.readLibSVM(env, "src/main/resources/svmguide1.t")


val svm = SVM()
  .setBlocks(env.getParallelism)
  .setIterations(100)
  .setRegularization(0.001)
  .setStepsize(0.1)
  .setSeed(42)

svm.fit(astroTrain)
svm.predict(astroTest)

I encounter the following error:

Exception in thread "main" java.lang.RuntimeException: There is no PredictOperation defined for org.apache.flink.ml.classification.SVM which takes a DataSet[org.apache.flink.ml.common.LabeledVector] as input.

Any idea ?

Thanks

Thomas

[Theodore Vasiloudis]

That is my bad, I must have been testing against a private branch when writing the guide, the SVM as it stands only has a predict operation for Vector not LabeledVector.

IMHO I would like to have a predict operator for LabeledVector for all predictors (that would just call the existing Vector prediction internally), but IIRC we decided to go with an Evaluate operator instead as written in the evaluation PR.

I'll make a PR to fix the guide, any chance you can create a JIRA for this?

Regards,

Theodore

On Mon, Oct 17, 2016 at 6:22 PM, Thomas FOURNIER <[hidden email]> wrote:

Hi,

Executing the following code (see QuickStart):

val env = ExecutionEnvironment.getExecutionEnvironment
val survival = env.readCsvFile[(String, String, String, String)]("src/main/resources/haberman.data", ",")


val survivalLV = survival
  .map { tuple =>
    val list = tuple.productIterator.toList
    val numList = list.map(_.asInstanceOf[String].toDouble)
    LabeledVector(numList(3), DenseVector(numList.take(3).toArray))
  }



val astroTrain = MLUtils.readLibSVM(env, "src/main/resources/svmguide1")
val astroTest = MLUtils.readLibSVM(env, "src/main/resources/svmguide1.t")


val svm = SVM()
  .setBlocks(env.getParallelism)
  .setIterations(100)
  .setRegularization(0.001)
  .setStepsize(0.1)
  .setSeed(42)

svm.fit(astroTrain)
svm.predict(astroTest)

I encounter the following error:

Exception in thread "main" java.lang.RuntimeException: There is no PredictOperation defined for org.apache.flink.ml.classification.SVM which takes a DataSet[org.apache.flink.ml.common.LabeledVector] as input.

Any idea ?

Thanks

Thomas

[Thomas FOURNIER]

Hi,

No problem I'm going to create a JIRA.

Regards

Thomas

2016-10-17 21:34 GMT+02:00 Theodore Vasiloudis <[hidden email]>:

That is my bad, I must have been testing against a private branch when writing the guide, the SVM as it stands only has a predict operation for Vector not LabeledVector.

IMHO I would like to have a predict operator for LabeledVector for all predictors (that would just call the existing Vector prediction internally), but IIRC we decided to go with an Evaluate operator instead as written in the evaluation PR.

I'll make a PR to fix the guide, any chance you can create a JIRA for this?

Regards,

Theodore

On Mon, Oct 17, 2016 at 6:22 PM, Thomas FOURNIER <[hidden email]> wrote:

Hi,

Executing the following code (see QuickStart):

val env = ExecutionEnvironment.getExecutionEnvironment
val survival = env.readCsvFile[(String, String, String, String)]("src/main/resources/haberman.data", ",")


val survivalLV = survival
  .map { tuple =>
    val list = tuple.productIterator.toList
    val numList = list.map(_.asInstanceOf[String].toDouble)
    LabeledVector(numList(3), DenseVector(numList.take(3).toArray))
  }



val astroTrain = MLUtils.readLibSVM(env, "src/main/resources/svmguide1")
val astroTest = MLUtils.readLibSVM(env, "src/main/resources/svmguide1.t")


val svm = SVM()
  .setBlocks(env.getParallelism)
  .setIterations(100)
  .setRegularization(0.001)
  .setStepsize(0.1)
  .setSeed(42)

svm.fit(astroTrain)
svm.predict(astroTest)

I encounter the following error:

Exception in thread "main" java.lang.RuntimeException: There is no PredictOperation defined for org.apache.flink.ml.classification.SVM which takes a DataSet[org.apache.flink.ml.common.LabeledVector] as input.

Any idea ?

Thanks

Thomas