Fire and Purge with Idle State

[shkob1]

Hey

Say im aggregating an event stream by sessionId in SQL and im emitting the
results once the session is "over", i guess i should be using Fire and Purge
- i dont expect to need to session data once over. How should i treat the
Idle state retention time - is it needed at all if im using purge? will it
become relevant only if a session is both never-ending AND never has more
records?

Thanks!



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[Hequn Cheng]

Hi shkob1,

Currently, the idle state retention time is only used for unbounded operators in sql/table-api. The unbounded operators include non-window group by, non-window join, unbounded over, etc. The retention time affects neither sql/table-api window operators nor DataStream operators.

Best, Hequn

On Sat, Oct 13, 2018 at 2:40 AM shkob1 <[hidden email]> wrote:

Hey

Say im aggregating an event stream by sessionId in SQL and im emitting the
results once the session is "over", i guess i should be using Fire and Purge
- i dont expect to need to session data once over. How should i treat the
Idle state retention time - is it needed at all if im using purge? will it
become relevant only if a session is both never-ending AND never has more
records?

Thanks!



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[shkob1]

Thanks!

On Fri, Oct 12, 2018 at 9:29 PM Hequn Cheng <[hidden email]> wrote:

Hi shkob1,

Currently, the idle state retention time is only used for unbounded operators in sql/table-api. The unbounded operators include non-window group by, non-window join, unbounded over, etc. The retention time affects neither sql/table-api window operators nor DataStream operators.

Best, Hequn

On Sat, Oct 13, 2018 at 2:40 AM shkob1 <[hidden email]> wrote:

Hey

Say im aggregating an event stream by sessionId in SQL and im emitting the
results once the session is "over", i guess i should be using Fire and Purge
- i dont expect to need to session data once over. How should i treat the
Idle state retention time - is it needed at all if im using purge? will it
become relevant only if a session is both never-ending AND never has more
records?

Thanks!



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/