Need a map-like state in an operator state

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Need a map-like state in an operator state

xsheng
This post was updated on .
Hi All,

The Flink app we are trying to build is as such: read from kafka, sort the
messages according to some dependency rules, and only send messages that
have satisfied all dependency requirements. I will need set-like and
map-like states to look up which messages have been published. The problem
is, it's not a keyed stream where map-like states are applicable. It seems
like we are stuck with operator states which seem to only support list-like
states. I guess I can use a ListState[Map[A,B]] but I don't think it's
possible to merge the maps in the ListState to allow for quick look-ups.

Can anyone help please?

Xianghai



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
Reply | Threaded
Open this post in threaded view
|

Re: Need a map-like state in an operator state

xsheng
Solved it by using a key selector that returns a constant, so that creates a
"pseudo" keyedStream with only one partition.



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/