How to load data in flink memory?

[dhirajpraj]

I want to load some data in flink memory more than the size of individual
task manager memory. Is this possible with flink? I want to use this loaded
data to process other incoming events.



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[miki haiat]

If you  want  to use  earlier or other   data  and process it.

You can   use process function  process_function 
Also  loading data in to memory isn't a good idea , you can crash the task manager .

Using state backend like rocks-db will  be very useful  , working-with-state

On Sun, Jul 8, 2018 at 4:47 PM dhirajpraj <[hidden email]> wrote:

I want to load some data in flink memory more than the size of individual
task manager memory. Is this possible with flink? I want to use this loaded
data to process other incoming events.



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[dhirajpraj]

Isn't the data loaded in memory distributed among the task managers?



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/