Login  Register

Fire and Purge with Idle State

classic Classic list List threaded Threaded
3 messages Options Options
Embed post
Permalink
Reply | Threaded
Open this post in threaded view
| More
Print post
Permalink

Fire and Purge with Idle State

shkob1
Hey

Say im aggregating an event stream by sessionId in SQL and im emitting the
results once the session is "over", i guess i should be using Fire and Purge
- i dont expect to need to session data once over. How should i treat the
Idle state retention time - is it needed at all if im using purge? will it
become relevant only if a session is both never-ending AND never has more
records?

Thanks!



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
Reply | Threaded
Open this post in threaded view
| More
Print post
Permalink

Re: Fire and Purge with Idle State

Hequn Cheng
Hi shkob1,

Currently, the idle state retention time is only used for unbounded operators in sql/table-api. The unbounded operators include non-window group by, non-window join, unbounded over, etc. The retention time affects neither sql/table-api window operators nor DataStream operators.

Best, Hequn

On Sat, Oct 13, 2018 at 2:40 AM shkob1 <[hidden email]> wrote:
Hey

Say im aggregating an event stream by sessionId in SQL and im emitting the
results once the session is "over", i guess i should be using Fire and Purge
- i dont expect to need to session data once over. How should i treat the
Idle state retention time - is it needed at all if im using purge? will it
become relevant only if a session is both never-ending AND never has more
records?

Thanks!



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
Reply | Threaded
Open this post in threaded view
| More
Print post
Permalink

Re: Fire and Purge with Idle State

shkob1
Thanks!

On Fri, Oct 12, 2018 at 9:29 PM Hequn Cheng <[hidden email]> wrote:
Hi shkob1,

Currently, the idle state retention time is only used for unbounded operators in sql/table-api. The unbounded operators include non-window group by, non-window join, unbounded over, etc. The retention time affects neither sql/table-api window operators nor DataStream operators.

Best, Hequn

On Sat, Oct 13, 2018 at 2:40 AM shkob1 <[hidden email]> wrote:
Hey

Say im aggregating an event stream by sessionId in SQL and im emitting the
results once the session is "over", i guess i should be using Fire and Purge
- i dont expect to need to session data once over. How should i treat the
Idle state retention time - is it needed at all if im using purge? will it
become relevant only if a session is both never-ending AND never has more
records?

Thanks!



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/