Qustion about Flink Upsert Dynamic Kafka Table unlimited expansion

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Qustion about Flink Upsert Dynamic Kafka Table unlimited expansion

vtygoss


Hi Community,


Recently i am working on building realtime data warehouse at medical field. Using Flink and Upsert-Kafka Dynamic Table, but the historical data must not be expired and the changelog stream in kafka is unlimited expanding, i have met a problem with unlimited expanding data scale.


How to solve this problem? Any suggestions, thanks for your reply!