Keeping latest data point in a data stream variable

[Biplob Biswas]

Hi,

I want to keep the latest data point which is processed in a datastream variable. So technically I need just one value in the variable and discard all the older ones.

Can this be done somehow? I was thinking about using filters but i don't think i can use it for this scenario.
Any ideas as to how to do something like this is really appreciated.

Thanks a lot
Biplob

[rmetzger0]

Hi Biplob,

would you like to send the last value somewhere?

is there a way of detecting when the stream ends? (Something like a marker element, or could you use a timeout?)

Anyways, what you can do is use a flatMap() function, always store the current element. Once the stream is over, you emit the last element.

On Tue, Jun 21, 2016 at 11:45 AM, Biplob Biswas <[hidden email]> wrote:

Hi,

I want to keep the latest data point which is processed in a datastream
variable. So technically I need just one value in the variable and discard
all the older ones.

Can this be done somehow? I was thinking about using filters but i don't
think i can use it for this scenario.
Any ideas as to how to do something like this is really appreciated.

Thanks a lot
Biplob



--
View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Keeping-latest-data-point-in-a-data-stream-variable-tp7642.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.