Concurrency on fields within operators

[Flavio Pompermaier]

Hi to all,

I have a simple question: are transient fields inside a Rich-Function  (like a RichMapPartition) thread-safe or not?

Best,

Flavio

[rmetzger0]

Hi Flavio,

tl;dr: they are thread safe.

making a field transient means that it is ignored when serializing the class. This is for example useful when you have a non-serializable field in your function (you have to initialize it in the open() method then).

So making it transient or not doesn't change the fields thread safety. Flink doesn't call the user defined functions concurrently (=from multiple threads), so you don't need to worry about thread safety.

The only case you need to worry about thread safety in your (Rich-)Function is when you're starting threads yourself inside the Function.

On Thu, Apr 16, 2015 at 9:09 AM, Flavio Pompermaier <[hidden email]> wrote:

Hi to all,

I have a simple question: are transient fields inside a Rich-Function  (like a RichMapPartition) thread-safe or not?

Best,

Flavio

[Flavio Pompermaier]

Ok thanks a lot for the confirmation! Thread-safety within operators is absolutely helpful :)

On Thu, Apr 16, 2015 at 9:20 AM, Robert Metzger <[hidden email]> wrote:

Hi Flavio,

tl;dr: they are thread safe.

making a field transient means that it is ignored when serializing the class. This is for example useful when you have a non-serializable field in your function (you have to initialize it in the open() method then).

So making it transient or not doesn't change the fields thread safety. Flink doesn't call the user defined functions concurrently (=from multiple threads), so you don't need to worry about thread safety.

The only case you need to worry about thread safety in your (Rich-)Function is when you're starting threads yourself inside the Function.

On Thu, Apr 16, 2015 at 9:09 AM, Flavio Pompermaier <[hidden email]> wrote:

Hi to all,

I have a simple question: are transient fields inside a Rich-Function  (like a RichMapPartition) thread-safe or not?

Best,

Flavio