Re: using updating shared data

Posted by miki haiat on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/using-updating-shared-data-tp25278p25279.html

Im trying to understand  your  use case.

What is the source  of the data ? FS ,KAFKA else ?

On Tue, Jan 1, 2019 at 6:29 PM Avi Levi <[hidden email]> wrote:

Hi,

I have a list (couple of thousands text lines) that I need to use in my map function. I read this article about broadcasting variables  or using distributed cache however I need to update this list from time to time, and if I understood correctly it is not possible on broadcast or cache without restarting the job. Is there idiomatic way to achieve this? A db seems to be an overkill for that and I do want to be cheap on io/network calls as much as possible.

Cheers 

Avi