Is there a way to clean up the state based on custom/business logic?

[kant kodali]

Hi All,

I read through the doc below and I am wondering if I can clean up the state based on custom logic rather min and max retention time?

For example, I want to say clean up all the state where the key = foo or say the value = bar. so until the keys reach a particular value just keep accumulating(retain) and when it does reach a particular value then write to some sink and clean up! I am looking for something along those lines. please let me know.

Thanks 

https://ci.apache.org/projects/flink/flink-docs-stable/dev/table/streaming/query_configuration.html

[Dawid Wysakowicz-2]

Hi,

What you are asking is certainly possible with the DataStream API. Table
API/SQL is a declarative API, where say what to want to compute not how.

As a rule of thumb I would say whenever you need to manually handle your
state or timers the DataStream API and ProcessFunction[1] will be a
better fit for you.

Best,

Dawid

[1]
https://ci.apache.org/projects/flink/flink-docs-release-1.9/dev/stream/operators/process_function.html

On 14/01/2020 11:33, kant kodali wrote:

> Hi All,
>
> I read through the doc below and I am wondering if I can clean up the
> state based on custom logic rather min and max retention time?
>
> For example, I want to say clean up all the state where the key = foo
> or say the value = bar. so until the keys reach a particular value
> just keep accumulating(retain) and when it does reach a particular
> value then write to some sink and clean up! I am looking for something
> along those lines. please let me know.
>
> Thanks 
>
>
> https://ci.apache.org/projects/flink/flink-docs-stable/dev/table/streaming/query_configuration.html