map/flatMapWithState and checkpoints

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

map/flatMapWithState and checkpoints

hkmaki
Hi,

If I have enabled checkpointing in my Flink program with env.enableCheckpointing(checkpointIntervalMillis) and I'm using mapWithState or flatMapWithState function, for example

inputData.keyBy(0).flatMapWithState((keyAndCount: (String, Int), count: Option[Int]) =>
      count match {
        case None => (Iterator((keyAndCount._1, 1)), Some(1))
        case Some(c) => (Iterator((keyAndCount._1, c+1)), Some(c+1))
      }).print

is the state automatically checkpointed?

In general I'm a bit confused about what needs to be explicitly checkpointed (for example, a UDF that implements the Checkpointed interface https://ci.apache.org/projects/flink/flink-docs-release-1.1/apis/streaming/state.html#checkpointing-instance-fields) and what doesn't.