Re: How to stream intermediate data that is stored in external storage?

Posted by kant kodali on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/How-to-stream-intermediate-data-that-is-stored-in-external-storage-tp30748p30799.html

Hi Averell,

I want to write intermediate results (A join B) incrementally and in real-time to some external storage so I can query it using SQL.

I am new to Flink so I am trying to find out if 1) such mechanism exists? 2) If not, what are the alternatives?

Thanks

On Thu, Oct 31, 2019 at 1:42 AM Averell <[hidden email]> wrote:

Hi Kant,

I wonder why you need to "source" your intermediate state from files? Why
not "source" it from the previous operator? I.e. instead of (A join B) ->
State -> files -> (C), why not do (A join B) -> State -> (files + C)?



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/