questions on custom state with flink window

[saiprasad mishra]

Hi All

I have few questions on understanding state with flink

1) Is it advisable to create custom state within a RichWindowFunction function.

I am able to create it but

2) If I create state in the window function then does the state remain for ever as I want the state to stay for ever

3) Also one thing I am noticing is the custom valuestate is not queryable from the query client and it results in the following exception

org.apache.flink.runtime.query.UnknownKvStateLocation: No KvStateLocation

4) is it possible to list all the state stores for a job Id, I did not see it in the client api

Regards

Sai

[rmetzger0]

Hi Sai,

1) I think its okay to keep state in a RichWindowFunction.

2) I think it stays forever, yes

3) I'm including Nico, he can probably help you with the queryable state question.

4) I guess that's a queryable state question too.

On Fri, Mar 10, 2017 at 1:04 AM, saiprasad mishra <[hidden email]> wrote:

Hi All

I have few questions on understanding state with flink

1) Is it advisable to create custom state within a RichWindowFunction function.

I am able to create it but

2) If I create state in the window function then does the state remain for ever as I want the state to stay for ever

3) Also one thing I am noticing is the custom valuestate is not queryable from the query client and it results in the following exception

org.apache.flink.runtime.query.UnknownKvStateLocation: No KvStateLocation

4) is it possible to list all the state stores for a job Id, I did not see it in the client api

Regards

Sai

[Nico Kruber]

Hi Sai,

3) If you want to make "Managed Keyed State" queryable, you have to set it as
queryable through the API, e.g.:

        final ValueStateDescriptor<Long> query1State =
                        new ValueStateDescriptor<>("stateName", Long.class);
        query1State.setQueryable("queryName");

Then, you can query for the given "queryName", given a jobId and the key you
want to query for.
Please refer to https://ci.apache.org/projects/flink/flink-docs-release-1.2/dev/
stream/queryable_state.html for more details and examples.

Non-keyed state is currently not queryable.

4) If you're asking for a list of all states set as queryable, there's nothing
like that but since you actively set state to be queryable, you should know.
Also there's more than the name if you want to work with it, its data type,
for example.


Nico

On Friday, 10 March 2017 14:46:37 CET Robert Metzger wrote:

> Hi Sai,
>
> 1) I think its okay to keep state in a RichWindowFunction.
> 2) I think it stays forever, yes
> 3) I'm including Nico, he can probably help you with the queryable state
> question.
> 4) I guess that's a queryable state question too.
>
>
> On Fri, Mar 10, 2017 at 1:04 AM, saiprasad mishra <[hidden email]
> > wrote:
> >
> > Hi All
> > I have few questions on understanding state with flink
> >
> > 1) Is it advisable to create custom state within a RichWindowFunction
> > function.
> > I am able to create it but
> >
> > 2) If I create state in the window function then does the state remain for
> > ever as I want the state to stay for ever
> >
> > 3) Also one thing I am noticing is the custom valuestate is not queryable
> > from the query client and it results in the following exception
> >
> > org.apache.flink.runtime.query.UnknownKvStateLocation: No KvStateLocation
> > 4) is it possible to list all the state stores for a job Id, I did not see
> > it in the client api
> >
> > Regards
> > Sai