"myuid" in snapshot.readingstate

[Abdullah bin Omar]

Hi,

when we readstate of of savepooint, we use, "myuid" as a argument of the function. For example,

DataSet<KeyedState> keyedState = savepoint.readKeyedState("my-uid", new ReaderFunction());

Question 1:

In [1] (line no 79), we get the "uid" with datastream. Then in [2] (line no 45), how can we use the "uid" that we have got from in [1]? Because in [2], there is no declaration of using the "uid" from [1]. 

Question 2:

what does it mean by "uid" in the datastream of [1]? is it something a unique user identification for each stream or for each state of the datastream?

[1] https://github.com/ververica/flink-training/blob/master/rides-and-fares/src/solution/java/org/apache/flink/training/solutions/ridesandfares/RidesAndFaresSolution.java

[2] https://github.com/ververica/flink-training/blob/master/state-processor/src/main/java/com/ververica/flink/training/exercises/ReadRidesAndFaresSnapshot.java

Thank you

[David Anderson-4]

You can read about assigning unique IDs to stateful operators in the docs [1][2]. What the uid() method does is to establish a stable and unique identifier for a stateful operator. Then as you evolve your application, this helps ensure that future versions of your job will be able to restore savepoints taken by earlier versions and find the state they need, despite changes to the topology.

The two uids specified in those two independent jobs (the streaming RidesAndFaresSolution job and the batch ReadRidesAndFaresSnapshot job) must match -- in other words, the strings must be the same. But there's nothing in that example that ensures this will be the case.

Regards,

David

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.12/ops/production_ready.html#set-uuids-for-all-operators

[2] https://ci.apache.org/projects/flink/flink-docs-release-1.12/ops/state/savepoints.html#assigning-operator-ids

On Fri, Apr 30, 2021 at 4:52 PM Abdullah bin Omar <[hidden email]> wrote:

Hi,

when we readstate of of savepooint, we use, "myuid" as a argument of the function. For example,

DataSet<KeyedState> keyedState = savepoint.readKeyedState("my-uid", new ReaderFunction());

Question 1:

In [1] (line no 79), we get the "uid" with datastream. Then in [2] (line no 45), how can we use the "uid" that we have got from in [1]? Because in [2], there is no declaration of using the "uid" from [1]. 

Question 2:

what does it mean by "uid" in the datastream of [1]? is it something a unique user identification for each stream or for each state of the datastream?

[1] https://github.com/ververica/flink-training/blob/master/rides-and-fares/src/solution/java/org/apache/flink/training/solutions/ridesandfares/RidesAndFaresSolution.java

[2] https://github.com/ververica/flink-training/blob/master/state-processor/src/main/java/com/ververica/flink/training/exercises/ReadRidesAndFaresSnapshot.java

Thank you

[Abu Bakar Siddiqur Rahman Rocky]

Hi David,

A quick question more 

I am trying to import org.apache.flink.training.exercises.common.sources.TaxiFareGenerator; However, it can not resolve.

What is dependency (in pom.xml) for the org.apache.flink.training?

Thank you

On Fri, Apr 30, 2021 at 10:12 AM David Anderson <[hidden email]> wrote:

You can read about assigning unique IDs to stateful operators in the docs [1][2]. What the uid() method does is to establish a stable and unique identifier for a stateful operator. Then as you evolve your application, this helps ensure that future versions of your job will be able to restore savepoints taken by earlier versions and find the state they need, despite changes to the topology.

The two uids specified in those two independent jobs (the streaming RidesAndFaresSolution job and the batch ReadRidesAndFaresSnapshot job) must match -- in other words, the strings must be the same. But there's nothing in that example that ensures this will be the case.

Regards,

David

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.12/ops/production_ready.html#set-uuids-for-all-operators

[2] https://ci.apache.org/projects/flink/flink-docs-release-1.12/ops/state/savepoints.html#assigning-operator-ids

On Fri, Apr 30, 2021 at 4:52 PM Abdullah bin Omar <[hidden email]> wrote:

Hi,

when we readstate of of savepooint, we use, "myuid" as a argument of the function. For example,

DataSet<KeyedState> keyedState = savepoint.readKeyedState("my-uid", new ReaderFunction());

Question 1:

In [1] (line no 79), we get the "uid" with datastream. Then in [2] (line no 45), how can we use the "uid" that we have got from in [1]? Because in [2], there is no declaration of using the "uid" from [1]. 

Question 2:

what does it mean by "uid" in the datastream of [1]? is it something a unique user identification for each stream or for each state of the datastream?

[1] https://github.com/ververica/flink-training/blob/master/rides-and-fares/src/solution/java/org/apache/flink/training/solutions/ridesandfares/RidesAndFaresSolution.java

[2] https://github.com/ververica/flink-training/blob/master/state-processor/src/main/java/com/ververica/flink/training/exercises/ReadRidesAndFaresSnapshot.java

Thank you

--

Regards,

Abu Bakar Siddiqur Rahman

[Abu Bakar Siddiqur Rahman Rocky]

I am asking on behalf of my colleagues the previous question. Though we have a couple of questions that were in another email

On Fri, Apr 30, 2021 at 10:18 AM Abu Bakar Siddiqur Rahman Rocky <[hidden email]> wrote:

Hi David,

A quick question more 

I am trying to import org.apache.flink.training.exercises.common.sources.TaxiFareGenerator; However, it can not resolve.

What is dependency (in pom.xml) for the org.apache.flink.training?

Thank you

On Fri, Apr 30, 2021 at 10:12 AM David Anderson <[hidden email]> wrote:

You can read about assigning unique IDs to stateful operators in the docs [1][2]. What the uid() method does is to establish a stable and unique identifier for a stateful operator. Then as you evolve your application, this helps ensure that future versions of your job will be able to restore savepoints taken by earlier versions and find the state they need, despite changes to the topology.

The two uids specified in those two independent jobs (the streaming RidesAndFaresSolution job and the batch ReadRidesAndFaresSnapshot job) must match -- in other words, the strings must be the same. But there's nothing in that example that ensures this will be the case.

Regards,

David

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.12/ops/production_ready.html#set-uuids-for-all-operators

[2] https://ci.apache.org/projects/flink/flink-docs-release-1.12/ops/state/savepoints.html#assigning-operator-ids

On Fri, Apr 30, 2021 at 4:52 PM Abdullah bin Omar <[hidden email]> wrote:

Hi,

when we readstate of of savepooint, we use, "myuid" as a argument of the function. For example,

DataSet<KeyedState> keyedState = savepoint.readKeyedState("my-uid", new ReaderFunction());

Question 1:

In [1] (line no 79), we get the "uid" with datastream. Then in [2] (line no 45), how can we use the "uid" that we have got from in [1]? Because in [2], there is no declaration of using the "uid" from [1]. 

Question 2:

what does it mean by "uid" in the datastream of [1]? is it something a unique user identification for each stream or for each state of the datastream?

[1] https://github.com/ververica/flink-training/blob/master/rides-and-fares/src/solution/java/org/apache/flink/training/solutions/ridesandfares/RidesAndFaresSolution.java

[2] https://github.com/ververica/flink-training/blob/master/state-processor/src/main/java/com/ververica/flink/training/exercises/ReadRidesAndFaresSnapshot.java

Thank you

--

Regards,

Abu Bakar Siddiqur Rahman

--

[David Anderson-4]

What is dependency (in pom.xml) for the org.apache.flink.training?

We don't publish artifacts for this repository. 

David

On Fri, Apr 30, 2021 at 5:19 PM Abu Bakar Siddiqur Rahman Rocky <[hidden email]> wrote:

Hi David,

A quick question more 

I am trying to import org.apache.flink.training.exercises.common.sources.TaxiFareGenerator; However, it can not resolve.

What is dependency (in pom.xml) for the org.apache.flink.training?

Thank you

On Fri, Apr 30, 2021 at 10:12 AM David Anderson <[hidden email]> wrote:

You can read about assigning unique IDs to stateful operators in the docs [1][2]. What the uid() method does is to establish a stable and unique identifier for a stateful operator. Then as you evolve your application, this helps ensure that future versions of your job will be able to restore savepoints taken by earlier versions and find the state they need, despite changes to the topology.

The two uids specified in those two independent jobs (the streaming RidesAndFaresSolution job and the batch ReadRidesAndFaresSnapshot job) must match -- in other words, the strings must be the same. But there's nothing in that example that ensures this will be the case.

Regards,

David

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.12/ops/production_ready.html#set-uuids-for-all-operators

[2] https://ci.apache.org/projects/flink/flink-docs-release-1.12/ops/state/savepoints.html#assigning-operator-ids

On Fri, Apr 30, 2021 at 4:52 PM Abdullah bin Omar <[hidden email]> wrote:

Hi,

when we readstate of of savepooint, we use, "myuid" as a argument of the function. For example,

DataSet<KeyedState> keyedState = savepoint.readKeyedState("my-uid", new ReaderFunction());

Question 1:

In [1] (line no 79), we get the "uid" with datastream. Then in [2] (line no 45), how can we use the "uid" that we have got from in [1]? Because in [2], there is no declaration of using the "uid" from [1]. 

Question 2:

what does it mean by "uid" in the datastream of [1]? is it something a unique user identification for each stream or for each state of the datastream?

[1] https://github.com/ververica/flink-training/blob/master/rides-and-fares/src/solution/java/org/apache/flink/training/solutions/ridesandfares/RidesAndFaresSolution.java

[2] https://github.com/ververica/flink-training/blob/master/state-processor/src/main/java/com/ververica/flink/training/exercises/ReadRidesAndFaresSnapshot.java

Thank you

--

Regards,

Abu Bakar Siddiqur Rahman

[Abu Bakar Siddiqur Rahman Rocky]

Thank you, David. 

On Fri, Apr 30, 2021 at 10:31 AM David Anderson <[hidden email]> wrote:

What is dependency (in pom.xml) for the org.apache.flink.training?

We don't publish artifacts for this repository. 

David

On Fri, Apr 30, 2021 at 5:19 PM Abu Bakar Siddiqur Rahman Rocky <[hidden email]> wrote:

Hi David,

A quick question more 

I am trying to import org.apache.flink.training.exercises.common.sources.TaxiFareGenerator; However, it can not resolve.

What is dependency (in pom.xml) for the org.apache.flink.training?

Thank you

On Fri, Apr 30, 2021 at 10:12 AM David Anderson <[hidden email]> wrote:

You can read about assigning unique IDs to stateful operators in the docs [1][2]. What the uid() method does is to establish a stable and unique identifier for a stateful operator. Then as you evolve your application, this helps ensure that future versions of your job will be able to restore savepoints taken by earlier versions and find the state they need, despite changes to the topology.

The two uids specified in those two independent jobs (the streaming RidesAndFaresSolution job and the batch ReadRidesAndFaresSnapshot job) must match -- in other words, the strings must be the same. But there's nothing in that example that ensures this will be the case.

Regards,

David

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.12/ops/production_ready.html#set-uuids-for-all-operators

[2] https://ci.apache.org/projects/flink/flink-docs-release-1.12/ops/state/savepoints.html#assigning-operator-ids

On Fri, Apr 30, 2021 at 4:52 PM Abdullah bin Omar <[hidden email]> wrote:

Hi,

when we readstate of of savepooint, we use, "myuid" as a argument of the function. For example,

DataSet<KeyedState> keyedState = savepoint.readKeyedState("my-uid", new ReaderFunction());

Question 1:

In [1] (line no 79), we get the "uid" with datastream. Then in [2] (line no 45), how can we use the "uid" that we have got from in [1]? Because in [2], there is no declaration of using the "uid" from [1]. 

Question 2:

what does it mean by "uid" in the datastream of [1]? is it something a unique user identification for each stream or for each state of the datastream?

[1] https://github.com/ververica/flink-training/blob/master/rides-and-fares/src/solution/java/org/apache/flink/training/solutions/ridesandfares/RidesAndFaresSolution.java

[2] https://github.com/ververica/flink-training/blob/master/state-processor/src/main/java/com/ververica/flink/training/exercises/ReadRidesAndFaresSnapshot.java

Thank you

--

Regards,

Abu Bakar Siddiqur Rahman

--