MapState Initializion

[Boris Lublinsky]

Any response to this?

List State is created using

checkpointedState = context.getOperatorStateStore.getListState (checkPointDescriptor)

But there is no similar APIs for MapState

Boris Lublinsky
FDP Architect
[hidden email]
https://www.lightbend.com/

Begin forwarded message:

From: Boris Lublinsky <[hidden email]>

Subject: Re: Questions about managed operator state

Date: January 14, 2018 at 7:39:00 PM CST

To: Fabian Hueske <[hidden email]>

Cc: user <[hidden email]>

Thanks Fabian,

After I switched to var it compiles, but its not initialized. 

@transient private var currentModels : MapState[String, Model] = _

Assignes null to MapState.

Do I create an empty hashMap there?

Boris Lublinsky
FDP Architect
[hidden email]
https://www.lightbend.com/

On Jan 14, 2018, at 11:09 AM, Fabian Hueske <[hidden email]> wrote:

Hi Boris,

the CheckpointedRestoring interface was removed in Flink 1.4.0 (and deprecated in an earlier version). Unfortunately, the docs have not been updated accordingly. I'll open a JIRA to fix this.

The replacements for CheckpointedRestoring are the CheckpointedFunction or ListCheckpointed interfaces (see [1]).

I think the compile error is caused because you define newModels as val and not as var.

Best, Fabian

[1] https://github.com/apache/flink/blob/release-1.3/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/checkpoint/CheckpointedRestoring.java

2018-01-14 2:39 GMT+01:00 Boris Lublinsky <[hidden email]>:

Documentation https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/stream/state/state.html#using-managed-operator-state

Refers to CheckpointedRestoring interface.

Which jar defines this interface - can’t find it

Also documentation refers to MapState<UK, UV>. Do you have any example of creation of Map State.

For value State in Scala, I can do just do

private var sum: ValueState[(Long, Long)] = _

But it does not work for MapState -

Error:(44, 53) unbound placeholder parameter

  private val newModels : MapState[String, Model] = _

Boris Lublinsky
FDP Architect
[hidden email]
https://www.lightbend.com/

Begin forwarded message:

From: Boris Lublinsky <[hidden email]>

Subject: Questions about managed operator state

Date: January 13, 2018 at 7:39:09 PM CST

To: user <[hidden email]>

Documentation https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/stream/state/state.html#using-managed-operator-state

Refers to CheckpointedRestoring interface.

Which jar defines this interface - can’t find it

Also documentation refers to MapState<UK, UV>. Do you have any example of creation of Map State.

For value State in Scala, I can do just do

private var sum: ValueState[(Long, Long)] = _

But it does not work for MapState -

Error:(44, 53) unbound placeholder parameter

  private val newModels : MapState[String, Model] = _

Boris Lublinsky
FDP Architect
[hidden email]
https://www.lightbend.com/

[Fabian Hueske-2]

Sorry for the late response.

MapState is currently only support as keyed state but not as operator state.

If you want to create a keyed MapState the object should be created using a MapStateDescriptor in the open() method via the RuntimeContext.

2018-01-16 1:54 GMT+01:00 Boris Lublinsky <[hidden email]>:

Any response to this?

List State is created using

checkpointedState = context.getOperatorStateStore.getListState (checkPointDescriptor)

But there is no similar APIs for MapState

Boris Lublinsky
FDP Architect
[hidden email]
https://www.lightbend.com/

Begin forwarded message:

From: Boris Lublinsky <[hidden email]>

Subject: Re: Questions about managed operator state

Date: January 14, 2018 at 7:39:00 PM CST

To: Fabian Hueske <[hidden email]>

Cc: user <[hidden email]>

Thanks Fabian,

After I switched to var it compiles, but its not initialized. 

@transient private var currentModels : MapState[String, Model] = _

Assignes null to MapState.

Do I create an empty hashMap there?

Boris Lublinsky
FDP Architect
[hidden email]
https://www.lightbend.com/

On Jan 14, 2018, at 11:09 AM, Fabian Hueske <[hidden email]> wrote:

Hi Boris,

the CheckpointedRestoring interface was removed in Flink 1.4.0 (and deprecated in an earlier version). Unfortunately, the docs have not been updated accordingly. I'll open a JIRA to fix this.

The replacements for CheckpointedRestoring are the CheckpointedFunction or ListCheckpointed interfaces (see [1]).

I think the compile error is caused because you define newModels as val and not as var.

Best, Fabian

[1] https://github.com/apache/flink/blob/release-1.3/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/checkpoint/CheckpointedRestoring.java

2018-01-14 2:39 GMT+01:00 Boris Lublinsky <[hidden email]>:

Documentation https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/stream/state/state.html#using-managed-operator-state

Refers to CheckpointedRestoring interface.

Which jar defines this interface - can’t find it

Also documentation refers to MapState<UK, UV>. Do you have any example of creation of Map State.

For value State in Scala, I can do just do

private var sum: ValueState[(Long, Long)] = _

But it does not work for MapState -

Error:(44, 53) unbound placeholder parameter

  private val newModels : MapState[String, Model] = _

Boris Lublinsky
FDP Architect
[hidden email]
https://www.lightbend.com/

Begin forwarded message:

From: Boris Lublinsky <[hidden email]>

Subject: Questions about managed operator state

Date: January 13, 2018 at 7:39:09 PM CST

To: user <[hidden email]>

Documentation https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/stream/state/state.html#using-managed-operator-state

Refers to CheckpointedRestoring interface.

Which jar defines this interface - can’t find it

Also documentation refers to MapState<UK, UV>. Do you have any example of creation of Map State.

For value State in Scala, I can do just do

private var sum: ValueState[(Long, Long)] = _

But it does not work for MapState -

Error:(44, 53) unbound placeholder parameter

  private val newModels : MapState[String, Model] = _

Boris Lublinsky
FDP Architect
[hidden email]
https://www.lightbend.com/

[Fabian Hueske-2]

Alternatively, you can also create a keyed MapState as context.getKeyedStateStore().getMapState() in CheckpointedFunction.initializeState().

2018-01-16 9:58 GMT+01:00 Fabian Hueske <[hidden email]>:

Sorry for the late response.

MapState is currently only support as keyed state but not as operator state.

If you want to create a keyed MapState the object should be created using a MapStateDescriptor in the open() method via the RuntimeContext.

2018-01-16 1:54 GMT+01:00 Boris Lublinsky <[hidden email]>:

Any response to this?

List State is created using

checkpointedState = context.getOperatorStateStore.getListState (checkPointDescriptor)

But there is no similar APIs for MapState

Boris Lublinsky
FDP Architect
[hidden email]
https://www.lightbend.com/

Begin forwarded message:

From: Boris Lublinsky <[hidden email]>

Subject: Re: Questions about managed operator state

Date: January 14, 2018 at 7:39:00 PM CST

To: Fabian Hueske <[hidden email]>

Cc: user <[hidden email]>

Thanks Fabian,

After I switched to var it compiles, but its not initialized. 

@transient private var currentModels : MapState[String, Model] = _

Assignes null to MapState.

Do I create an empty hashMap there?

Boris Lublinsky
FDP Architect
[hidden email]
https://www.lightbend.com/

On Jan 14, 2018, at 11:09 AM, Fabian Hueske <[hidden email]> wrote:

Hi Boris,

the CheckpointedRestoring interface was removed in Flink 1.4.0 (and deprecated in an earlier version). Unfortunately, the docs have not been updated accordingly. I'll open a JIRA to fix this.

The replacements for CheckpointedRestoring are the CheckpointedFunction or ListCheckpointed interfaces (see [1]).

I think the compile error is caused because you define newModels as val and not as var.

Best, Fabian

[1] https://github.com/apache/flink/blob/release-1.3/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/checkpoint/CheckpointedRestoring.java

2018-01-14 2:39 GMT+01:00 Boris Lublinsky <[hidden email]>:

Documentation https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/stream/state/state.html#using-managed-operator-state

Refers to CheckpointedRestoring interface.

Which jar defines this interface - can’t find it

Also documentation refers to MapState<UK, UV>. Do you have any example of creation of Map State.

For value State in Scala, I can do just do

private var sum: ValueState[(Long, Long)] = _

But it does not work for MapState -

Error:(44, 53) unbound placeholder parameter

  private val newModels : MapState[String, Model] = _

Boris Lublinsky
FDP Architect
[hidden email]
https://www.lightbend.com/

Begin forwarded message:

From: Boris Lublinsky <[hidden email]>

Subject: Questions about managed operator state

Date: January 13, 2018 at 7:39:09 PM CST

To: user <[hidden email]>

Documentation https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/stream/state/state.html#using-managed-operator-state

Refers to CheckpointedRestoring interface.

Which jar defines this interface - can’t find it

Also documentation refers to MapState<UK, UV>. Do you have any example of creation of Map State.

For value State in Scala, I can do just do

private var sum: ValueState[(Long, Long)] = _

But it does not work for MapState -

Error:(44, 53) unbound placeholder parameter

  private val newModels : MapState[String, Model] = _

Boris Lublinsky
FDP Architect
[hidden email]
https://www.lightbend.com/