Table SQL and updating rows

classic Classic list List threaded Threaded
1 message Options
Dan
Reply | Threaded
Open this post in threaded view
|

Table SQL and updating rows

Dan
Context
I'm working on a user event logging system using Flink.   I'm tracking some info that belongs to the user's current session (e.g. location, device, experiment info).  I don't have a v1 requirement to support mutability but I want to plan for it.  I think the most likely reason for mutation will be that different parts of the system update session info (e.g. location might not be accessible until later if the user does not give permission for it initially).

Ideas
1) Introduce an aggregation step.  E.g. key by sessionId and run a TableAggregateFunction.
2) Split the session into smaller parts that can only be inserted once.  Do a join to create a combined session.

How have other people solved this with Flink Table SQL?  If I was using DataStream directly, I'd group by and have a map function that uses key state to keep track of the latest values.