env.fromElements produces TypeInformation error

[Dan Drewes]

Hi,

compiling the code:

def minimize(f: DF, init: T): T = {

  //create execution environment
  val env = ExecutionEnvironment.getExecutionEnvironment

  val initialstate = initialState(f, init)

  val iterativestate= env.fromElements(initialstate).iterate(10000) {
    iterationInput: DataSet[State] =>
      val result = iterationInput.map {
        oldstate => computeNextState(adjustedFun, oldstate)
      }
      result
  }
}

object IterationsFirstOrderMinimizer {

  case class State[+T, +ConvergenceInfo, +History] (x: T,
                                                   value: Double, grad: T,
                                                   adjustedValue: Double, adjustedGradient: T,
                                                   iter: Int,
                                                   initialAdjVal: Double,
                                                   history: History,
                                                   convergenceInfo: ConvergenceInfo,
                                                   searchFailed: Boolean = false) {
  }

... fails with the error: could not find implicit value for evidence parameter of type org.apache.flink.api.common.typeinfo.TypeInformation[IterationsFirstOrderMinimizer.this.State]     val iterativestate= env.fromElements(initialState(f, init)).iterate(10000) {                                         ^ Google only told me to import org.apache.flink.api.scala._     which i do. The other suggested solution for generic methods (https://ci.apache.org/projects/flink/flink-docs-master/internals/types_serialization.html) does also not apply because it's the fromElements call that produces the error. I am very new to Flink and Scala and because I don't know if the code I posted above is enough to say what I'm doing wrong, you can find the complete scala file attached. Thanks in advance for any ideas and hints. Best, Dan

Virenfrei. www.avast.com

[Simone Robutti]

I'm not sure if this is the solution and I don't have the possibility to try right now, but you should move the case class "State" definition outside the abstract class.

2016-06-04 17:34 GMT+02:00 Dan Drewes <[hidden email]>:

Hi,

compiling the code:

def minimize(f: DF, init: T): T = {

  //create execution environment
  val env = ExecutionEnvironment.getExecutionEnvironment

  val initialstate = initialState(f, init)

  val iterativestate= env.fromElements(initialstate).iterate(10000) {
    iterationInput: DataSet[State] =>
      val result = iterationInput.map {
        oldstate => computeNextState(adjustedFun, oldstate)
      }
      result
  }
}

object IterationsFirstOrderMinimizer {

  case class State[+T, +ConvergenceInfo, +History] (x: T,
                                                   value: Double, grad: T,
                                                   adjustedValue: Double, adjustedGradient: T,
                                                   iter: Int,
                                                   initialAdjVal: Double,
                                                   history: History,
                                                   convergenceInfo: ConvergenceInfo,
                                                   searchFailed: Boolean = false) {
  }

... fails with the error: could not find implicit value for evidence parameter of type org.apache.flink.api.common.typeinfo.TypeInformation[IterationsFirstOrderMinimizer.this.State]     val iterativestate= env.fromElements(initialState(f, init)).iterate(10000) {                                         ^ Google only told me to import org.apache.flink.api.scala._     which i do. The other suggested solution for generic methods (https://ci.apache.org/projects/flink/flink-docs-master/internals/types_serialization.html) does also not apply because it's the fromElements call that produces the error. I am very new to Flink and Scala and because I don't know if the code I posted above is enough to say what I'm doing wrong, you can find the complete scala file attached. Thanks in advance for any ideas and hints. Best, Dan

Virenfrei. www.avast.com

[Dan Drewes]

I've tested it, but unfortunately it does not solve the problem.
The error message remains the same.

Am 04.06.2016 um 19:38 schrieb Simone Robutti:

I'm not sure if this is the solution and I don't have the possibility to try right now, but you should move the case class "State" definition outside the abstract class.

2016-06-04 17:34 GMT+02:00 Dan Drewes <[hidden email]>:

Hi,

compiling the code:

def minimize(f: DF, init: T): T = {

  //create execution environment
  val env = ExecutionEnvironment.getExecutionEnvironment

  val initialstate = initialState(f, init)

  val iterativestate= env.fromElements(initialstate).iterate(10000) {
    iterationInput: DataSet[State] =>
      val result = iterationInput.map {
        oldstate => computeNextState(adjustedFun, oldstate)
      }
      result
  }
}

object IterationsFirstOrderMinimizer {

  case class State[+T, +ConvergenceInfo, +History] (x: T,
                                                   value: Double, grad: T,
                                                   adjustedValue: Double, adjustedGradient: T,
                                                   iter: Int,
                                                   initialAdjVal: Double,
                                                   history: History,
                                                   convergenceInfo: ConvergenceInfo,
                                                   searchFailed: Boolean = false) {
  }

... fails with the error: could not find implicit value for evidence parameter of type org.apache.flink.api.common.typeinfo.TypeInformation[IterationsFirstOrderMinimizer.this.State]     val iterativestate= env.fromElements(initialState(f, init)).iterate(10000) {                                         ^ Google only told me to import org.apache.flink.api.scala._     which i do. The other suggested solution for generic methods (https://ci.apache.org/projects/flink/flink-docs-master/internals/types_serialization.html) does also not apply because it's the fromElements call that produces the error. I am very new to Flink and Scala and because I don't know if the code I posted above is enough to say what I'm doing wrong, you can find the complete scala file attached. Thanks in advance for any ideas and hints. Best, Dan

Virenfrei. www.avast.com

Virenfrei. www.avast.com

[Aljoscha Krettek]

Hi,

I think the problem is that the case class has generic parameters. You can try making TypeInformation for those parameters implicitly available at the call site, i.e:

implicit val typeT = createTypeInformation[T] // where you insert the specific type for T and do the same for the other generic parameters

Maybe this helps.

Cheers,

Aljoscha 

On Sat, 4 Jun 2016 at 20:04 Dan Drewes <[hidden email]> wrote:

I've tested it, but unfortunately it does not solve the problem.
The error message remains the same.

Am 04.06.2016 um 19:38 schrieb Simone Robutti:

I'm not sure if this is the solution and I don't have the possibility to try right now, but you should move the case class "State" definition outside the abstract class.

2016-06-04 17:34 GMT+02:00 Dan Drewes <[hidden email]>:

Hi,

compiling the code:

def minimize(f: DF, init: T): T = {

  //create execution environment
  val env = ExecutionEnvironment.getExecutionEnvironment

  val initialstate = initialState(f, init)

  val iterativestate= env.fromElements(initialstate).iterate(10000) {
    iterationInput: DataSet[State] =>
      val result = iterationInput.map {
        oldstate => computeNextState(adjustedFun, oldstate)
      }
      result
  }
}

object IterationsFirstOrderMinimizer {

  case class State[+T, +ConvergenceInfo, +History] (x: T,
                                                   value: Double, grad: T,
                                                   adjustedValue: Double, adjustedGradient: T,
                                                   iter: Int,
                                                   initialAdjVal: Double,
                                                   history: History,
                                                   convergenceInfo: ConvergenceInfo,
                                                   searchFailed: Boolean = false) {
  }

... fails with the error: could not find implicit value for evidence parameter of type org.apache.flink.api.common.typeinfo.TypeInformation[IterationsFirstOrderMinimizer.this.State]     val iterativestate= env.fromElements(initialState(f, init)).iterate(10000) {                                         ^ Google only told me to import org.apache.flink.api.scala._     which i do. The other suggested solution for generic methods (https://ci.apache.org/projects/flink/flink-docs-master/internals/types_serialization.html) does also not apply because it's the fromElements call that produces the error. I am very new to Flink and Scala and because I don't know if the code I posted above is enough to say what I'm doing wrong, you can find the complete scala file attached. Thanks in advance for any ideas and hints. Best, Dan

Virenfrei. www.avast.com

Virenfrei. www.avast.com