Quick question about enableObjectReuse()

[LINZ, Arnaud]

Hi,

 

I just want to be sure : when I set enableObjectReuse, I don’t need to create copies of objects that I get as input and return as output but which I don’t keep inside my user function ?

For instance, if I want to join Tuple2(A,B) with C into Tuple3(A,B,C) using a Join function, I can write something like :

 

public Tuple3 join(Tuple2 first, Object second) {

                return Tuple3.of(first.f0, first.f1, second);

}

And not               return Tuple3.of(first.f0.clone(), first.f1.clone(), second.clone()) ?

 

 

Best regards,

Arnaud

 

 

---

L'intégrité de ce message n'étant pas assurée sur internet, la société expéditrice ne peut être tenue responsable de son contenu ni de ses pièces jointes. Toute utilisation ou diffusion non autorisée est interdite. Si vous n'êtes pas destinataire de ce message, merci de le détruire et d'avertir l'expéditeur.

The integrity of this message cannot be guaranteed on the Internet. The company that sent this message cannot therefore be held liable for its content nor attachments. Any unauthorized use or dissemination is prohibited. If you are not the intended recipient of this message, then please delete it and notify the sender.

[Till Rohrmann]

Yes, you're right Arnaud.

Cheers,

Till

On Tue, Feb 9, 2016 at 10:42 AM, LINZ, Arnaud <[hidden email]> wrote:

Hi,

 

I just want to be sure : when I set enableObjectReuse, I don’t need to create copies of objects that I get as input and return as output but which I don’t keep inside my user function ?

For instance, if I want to join Tuple2(A,B) with C into Tuple3(A,B,C) using a Join function, I can write something like :

 

public Tuple3 join(Tuple2 first, Object second) {

                return Tuple3.of(first.f0, first.f1, second);

}

And not               return Tuple3.of(first.f0.clone(), first.f1.clone(), second.clone()) ?

 

 

Best regards,

Arnaud

 

 

---

L'intégrité de ce message n'étant pas assurée sur internet, la société expéditrice ne peut être tenue responsable de son contenu ni de ses pièces jointes. Toute utilisation ou diffusion non autorisée est interdite. Si vous n'êtes pas destinataire de ce message, merci de le détruire et d'avertir l'expéditeur.

The integrity of this message cannot be guaranteed on the Internet. The company that sent this message cannot therefore be held liable for its content nor attachments. Any unauthorized use or dissemination is prohibited. If you are not the intended recipient of this message, then please delete it and notify the sender.

[Stephan Ewen]

The only thing you need to be aware of (in the batch API) is that you cannot simply gather elements in a list any more.

The following does not work when enabling object reuse:

class MyReducer implements GroupReduceFunction<In, Out> {

    public void reduceGroup(Iterable<In> values, Collector<Out> out) {

        List<In> allValues = new List<In>();

        for (In i : values) {

            allValues.add(i);

        }

    }

}

In general, object reuse is a feature that makes sense if you want to optimize GC performance. If you don't need it, all the better. It is fairly easy to accidentally implement the user functions to hold onto reused object references, and compute wrong results that way.

Greetings,
Stephan

On Tue, Feb 9, 2016 at 10:48 AM, Till Rohrmann <[hidden email]> wrote:

Yes, you're right Arnaud.

Cheers,

Till

On Tue, Feb 9, 2016 at 10:42 AM, LINZ, Arnaud <[hidden email]> wrote:

Hi,

 

I just want to be sure : when I set enableObjectReuse, I don’t need to create copies of objects that I get as input and return as output but which I don’t keep inside my user function ?

For instance, if I want to join Tuple2(A,B) with C into Tuple3(A,B,C) using a Join function, I can write something like :

 

public Tuple3 join(Tuple2 first, Object second) {

                return Tuple3.of(first.f0, first.f1, second);

}

And not               return Tuple3.of(first.f0.clone(), first.f1.clone(), second.clone()) ?

 

 

Best regards,

Arnaud

 

 

---

L'intégrité de ce message n'étant pas assurée sur internet, la société expéditrice ne peut être tenue responsable de son contenu ni de ses pièces jointes. Toute utilisation ou diffusion non autorisée est interdite. Si vous n'êtes pas destinataire de ce message, merci de le détruire et d'avertir l'expéditeur.

The integrity of this message cannot be guaranteed on the Internet. The company that sent this message cannot therefore be held liable for its content nor attachments. Any unauthorized use or dissemination is prohibited. If you are not the intended recipient of this message, then please delete it and notify the sender.