update the existing Keyed value state

Hi Users,

We want to have a real time aggregation (KPI) .

we are maintaining aggregation counters in the keyed value  state  . 

key could be customer activation date and type.

Lot of counters are maintained against that key.

If we want to add one more counter for the existing keys which is in the state backend.  

1.compute the new counter value using database data .

2. group the counter value based on the key   

How do we update the new computed counter to all the existing keyed state?.

[Piotr Nowojski-3]

Hi,

This might be tricky. There are some on-going efforts [1] and 3rd party project [2] that allow you to read save point, modify it and write back the new modified save point from which you can restore.

Besides those, you might be able to modify the code of your aggregators, to initialise their value for the first time either from something hardcoded or by some call to external system/lookup in a database/read from file.

[1] https://github.com/sjwiesman/flink/commits/savepoint-connector

[2] https://github.com/king/bravo

Piotr Nowojski

On 3 May 2019, at 11:14, Selvaraj chennappan <[hidden email]> wrote:

Hi Users,

We want to have a real time aggregation (KPI) .

we are maintaining aggregation counters in the keyed value  state  . 

key could be customer activation date and type.

Lot of counters are maintained against that key.

If we want to add one more counter for the existing keys which is in the state backend.  

1.compute the new counter value using database data .

2. group the counter value based on the key   

How do we update the new computed counter to all the existing keyed state?.