What's the best way to clean up the expired rocksdb state

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

What's the best way to clean up the expired rocksdb state

sundy
hi:

my streaming application always do Key by the some keys with event timestamp, such as  keyBy( “qps_1520777430”), so the expired keys(1 hours ago) are useless.

And I use rocksdb to store the state,  I want to know What's the best way to clean up the expired rocksdb state, should I must implement a ProcessFunction and set a time ticker to do that?
Reply | Threaded
Open this post in threaded view
|

Re: What's the best way to clean up the expired rocksdb state

Chesnay Schepler
Hello,

yes, i think you'll need to use a ProcessFunction and clean up the state
manually.

On 11.03.2018 15:13, sundy wrote:
> hi:
>
> my streaming application always do Key by the some keys with event timestamp, such as  keyBy( “qps_1520777430”), so the expired keys(1 hours ago) are useless.
>
> And I use rocksdb to store the state,  I want to know What's the best way to clean up the expired rocksdb state, should I must implement a ProcessFunction and set a time ticker to do that?