Stateful Functions - accessing the state aside of normal processing
Locked
Stateful Functions - accessing the state aside of normal processing
 
|
Hi, We are trying to use Statefuns for our tool and it seems to be a good fit. I already adopted it and it works quite well. However, we have millions of different states (all the same FunctionType but different ids) and
each state consists of several @Persisted values (values and tables). We want to build an administration tool for examining the crowd of states (count, histogram, etc.) and each state in detail (the persisted-tables and -values). Additionally we need some kind of dig-down functionality for finding those individual states. For example some of those persisted values can be used to categorize the crowd of states. My question now is how to achieve this. Is there a way to browse and examine statefuns in a read-only fashion (their ids, their persisted values)? How can one achieve this without duplicating status in e.g. a relational
database? Thanks, Stephan PS: I have another questions but I will send them in separate mails to avoid mixing up topics. |
Re: Stateful Functions - accessing the state aside of normal processing
|
|
Hi Stephan, Great to hear about your experience with StateFun so far! I think what you are looking for is a way to read StateFun checkpoints, which are basically an immutable consistent point-in-time snapshot of all the states across all your functions, and run some computation or simply to explore the state values. StateFun checkpoints are essentially adopted from Flink, so you can find more detail about that here [1]. Currently, StateFun does provide a means for state "bootstrapping": running a batch offline job to write and compose a StateFun checkpoint [2]. What is still missing is the "reading / analysis" side of things, to do exactly what you described: running a separate batch offline job for reading and processing an existing StateFun checkpoint. Before we dive into details on how that may look like, do you think that is what you would need? Although I don't think we would be able to support such a feature yet since we're currently focused on reworking the SDKs and request-reply protocol, in any case it would be interesting to discuss if this feature would be important for multiple users already. Cheers, Gordon [1] https://ci.apache.org/projects/flink/flink-docs-master/concepts/stateful-stream-processing.html#checkpointing [2] https://ci.apache.org/projects/flink/flink-statefun-docs-release-2.2/deployment-and-operations/state-bootstrap.html On Wed, Jan 27, 2021 at 11:41 PM Stephan Pelikan <[hidden email]> wrote:
|
AW: Stateful Functions - accessing the state aside of normal processing
|
|
Hi Gordon, If operating on checkpoints instead of savepoints this might be OK. But since this is not in the current scope I digged into Flink docs and found the "queryable state" (https://ci.apache.org/projects/flink/flink-docs-release-1.12/dev/stream/state/queryable_state.html#querying-state).
This sounds good and seems to be a possibility to read the state of a specific function by id. This would solve the first part of my challange (examining the current state). Additionally
there is remote client what makes things easy. As far as I understand its only necessary to enable this for statefuns. If the types like PersistedValue also takes a queryable-name like ValueStateDescriptor it could be passed through
in places like
https://github.com/apache/flink-statefun/blob/master/statefun-flink/statefun-flink-core/src/main/java/org/apache/flink/statefun/flink/core/state/FlinkState.java#L65. Then the state of single jobs could be retrieved if I'm right. But I can only query states
of a specific statefun by id. Not the total crowd of states. To get a solution in the "near" future I could send "state changes" egress messages and stream them into an ElasticSearch sink. Then I could search that ES index the way I like. I only
have to check if that works in terms of amount of data and throughput. Additionally I'll have to consider how to structure those "state changes" events in the ES to be able to query as I need. As a give-away I would get historical data of states outdated or
cleared. This sounds like a feasible solution. What do you think? Cheers, Stephan Von: Tzu-Li (Gordon) Tai <[hidden email]>
Hi Stephan, On Wed, Jan 27, 2021 at 11:41 PM Stephan Pelikan <[hidden email]> wrote:
|
Re: Stateful Functions - accessing the state aside of normal processing
|
|
Hi Stephan, I think that what you are trying to achieve is very interesting, and possibly other users might find that useful as well and we will definitely add that to our roadmap. I think that Gordon's suggestion of using the state processor API to examine a savepoint, makes a lot of sense in this case. A savepoint can be analyzed off-band with the DataSet API and sliced and diced however you'd like. I think that This approach will also be a major part of the solution we will go with. If you are interested to work on a contribution here, me and Gordon would be more than happy to guide you toward that. Meanwhile, your suggestion to stream the changes to a temporary storage that is optimized for drill down exploration might also work. I'd like to suggest an alternative/complementing approach - how about implementing these requirements in StateFun itself? First I'd like to refer you to a blog post at [2] and in particular the image at the bottom, perhaps it can also work in your case. In your case, you will need an additional function type that will hold the aggregation data (possibly sharded to reduce hotspots). The aggregation function can periodically emit the aggregations elsewhere. To implement a drill down functionality you would have to implement sort-of an RPC mechanism with your functions (see here for example [3][4]) to ask a specific function instance for its state. Kind regards, Igal. On Thu, Jan 28, 2021 at 7:45 PM Stephan Pelikan <[hidden email]> wrote:
|
| Free forum by Nabble | Edit this page |