Collector and null tuple values

[Flavio Pompermaier]

Hi to all,

why it is possible to collect (in a flatMap function for example) a Tuple with a null String but it's not possible with a null Boolean value? Is that normal?

Best,

Flavio

[rmetzger0]

Are you talking about a Tuple2<String, Boolean> t for example?

So t.f0 == null doesn't cause any issues, but t.f1 == null is failing?

What's the error message?

On Wed, Sep 23, 2015 at 3:31 PM, Flavio Pompermaier <[hidden email]> wrote:

Hi to all,

why it is possible to collect (in a flatMap function for example) a Tuple with a null String but it's not possible with a null Boolean value? Is that normal?

Best,

Flavio

[Stephan Ewen]

In general, Tuples may not have null fields - the  tuples have no handling for null.

It is just by chance that String actually handles itself with null. 

In general the unspecified (but effective) behavior is that if the types themselves support null handling, then you can use them. But I would not rely on that feature.

Tuples should be used with non-null elements. I would use an "Option" type (in Scala) for null elements, or a POJO in Java.

On Wed, Sep 23, 2015 at 3:34 PM, Robert Metzger <[hidden email]> wrote:

Are you talking about a Tuple2<String, Boolean> t for example?

So t.f0 == null doesn't cause any issues, but t.f1 == null is failing?

What's the error message?

On Wed, Sep 23, 2015 at 3:31 PM, Flavio Pompermaier <[hidden email]> wrote:

Hi to all,

why it is possible to collect (in a flatMap function for example) a Tuple with a null String but it's not possible with a null Boolean value? Is that normal?

Best,

Flavio

[Fabian Hueske-2]

Flink's tuple data types do not support null fields.
A program with null-valued tuple fields will fail as soon as the tuple is serialized (written into memory, to disk, or shipped over the network).

If you need support for null values, you can for example implement a custom POJO data type.

Best, Fabian

2015-09-23 15:34 GMT+02:00 Robert Metzger <[hidden email]>:

Are you talking about a Tuple2<String, Boolean> t for example?

So t.f0 == null doesn't cause any issues, but t.f1 == null is failing?

What's the error message?

On Wed, Sep 23, 2015 at 3:31 PM, Flavio Pompermaier <[hidden email]> wrote:

Hi to all,

why it is possible to collect (in a flatMap function for example) a Tuple with a null String but it's not possible with a null Boolean value? Is that normal?

Best,

Flavio

[Flavio Pompermaier]

I could but I heard about the fact that the Tuple model is much faster than the Pojo one, isn't it?

How big is such difference? Is there any blog post about that?

Thanks for the support,

Flavio

On Wed, Sep 23, 2015 at 4:00 PM, Fabian Hueske <[hidden email]> wrote:

Flink's tuple data types do not support null fields.
A program with null-valued tuple fields will fail as soon as the tuple is serialized (written into memory, to disk, or shipped over the network).

If you need support for null values, you can for example implement a custom POJO data type.

Best, Fabian

2015-09-23 15:34 GMT+02:00 Robert Metzger <[hidden email]>:

Are you talking about a Tuple2<String, Boolean> t for example?

So t.f0 == null doesn't cause any issues, but t.f1 == null is failing?

What's the error message?

On Wed, Sep 23, 2015 at 3:31 PM, Flavio Pompermaier <[hidden email]> wrote:

Hi to all,

why it is possible to collect (in a flatMap function for example) a Tuple with a null String but it's not possible with a null Boolean value? Is that normal?

Best,

Flavio

[Stephan Ewen]

One part why the tuples are faster is actually the simpler serialization format without null values...

On Wed, Sep 23, 2015 at 5:24 PM, Flavio Pompermaier <[hidden email]> wrote:

I could but I heard about the fact that the Tuple model is much faster than the Pojo one, isn't it?

How big is such difference? Is there any blog post about that?

Thanks for the support,

Flavio

On Wed, Sep 23, 2015 at 4:00 PM, Fabian Hueske <[hidden email]> wrote:

Flink's tuple data types do not support null fields.
A program with null-valued tuple fields will fail as soon as the tuple is serialized (written into memory, to disk, or shipped over the network).

If you need support for null values, you can for example implement a custom POJO data type.

Best, Fabian

2015-09-23 15:34 GMT+02:00 Robert Metzger <[hidden email]>:

Are you talking about a Tuple2<String, Boolean> t for example?

So t.f0 == null doesn't cause any issues, but t.f1 == null is failing?

What's the error message?

On Wed, Sep 23, 2015 at 3:31 PM, Flavio Pompermaier <[hidden email]> wrote:

Hi to all,

why it is possible to collect (in a flatMap function for example) a Tuple with a null String but it's not possible with a null Boolean value? Is that normal?

Best,

Flavio

[Stephan Ewen]

I was just thinking that it would be nice to have an "Option" or "Nullable" type in the Java API that the type extractor understands and that can be used for exactly that case...

On Wed, Sep 23, 2015 at 5:26 PM, Stephan Ewen <[hidden email]> wrote:

One part why the tuples are faster is actually the simpler serialization format without null values...

On Wed, Sep 23, 2015 at 5:24 PM, Flavio Pompermaier <[hidden email]> wrote:

I could but I heard about the fact that the Tuple model is much faster than the Pojo one, isn't it?

How big is such difference? Is there any blog post about that?

Thanks for the support,

Flavio

On Wed, Sep 23, 2015 at 4:00 PM, Fabian Hueske <[hidden email]> wrote:

Flink's tuple data types do not support null fields.
A program with null-valued tuple fields will fail as soon as the tuple is serialized (written into memory, to disk, or shipped over the network).

If you need support for null values, you can for example implement a custom POJO data type.

Best, Fabian

2015-09-23 15:34 GMT+02:00 Robert Metzger <[hidden email]>:

Are you talking about a Tuple2<String, Boolean> t for example?

So t.f0 == null doesn't cause any issues, but t.f1 == null is failing?

What's the error message?

On Wed, Sep 23, 2015 at 3:31 PM, Flavio Pompermaier <[hidden email]> wrote:

Hi to all,

why it is possible to collect (in a flatMap function for example) a Tuple with a null String but it's not possible with a null Boolean value? Is that normal?

Best,

Flavio