wait until BulkIteration finishes

[Lydia Ickler]

Hi all,

is there a way to tell the program that it should wait until the BulkIteration finishes before the rest of the program is executed?

Best regards,
Lydia

[Till Rohrmann]

Hi Lydia,

all downstream operators which depend on the bulk iteration will wait implicitly until data from the iteration operator is available.

Cheers,

Till

On Thu, Mar 31, 2016 at 9:39 AM, Lydia Ickler <[hidden email]> wrote:

Hi all,

is there a way to tell the program that it should wait until the BulkIteration finishes before the rest of the program is executed?

Best regards,
Lydia

[Lydia Ickler]

Hi Till, 

thanks for your reply!

Is there a way to store intermediate results of the bulk iteration to use then in the next iteration except the data set one sends already by default?

Best regards, 

Lydia

Am 31.03.2016 um 12:01 schrieb Till Rohrmann <[hidden email]>:

Hi Lydia,

all downstream operators which depend on the bulk iteration will wait implicitly until data from the iteration operator is available.

Cheers,

Till

On Thu, Mar 31, 2016 at 9:39 AM, Lydia Ickler <[hidden email]> wrote:

Hi all,

is there a way to tell the program that it should wait until the BulkIteration finishes before the rest of the program is executed?

Best regards,
Lydia

[Till Rohrmann]

I think I don't completely understand your question.

On Thu, Mar 31, 2016 at 4:40 PM, Lydia Ickler <[hidden email]> wrote:

Hi Till, 

thanks for your reply!

Is there a way to store intermediate results of the bulk iteration to use then in the next iteration except the data set one sends already by default?

Best regards, 

Lydia

Am 31.03.2016 um 12:01 schrieb Till Rohrmann <[hidden email]>:

Hi Lydia,

all downstream operators which depend on the bulk iteration will wait implicitly until data from the iteration operator is available.

Cheers,

Till

On Thu, Mar 31, 2016 at 9:39 AM, Lydia Ickler <[hidden email]> wrote:

Hi all,

is there a way to tell the program that it should wait until the BulkIteration finishes before the rest of the program is executed?

Best regards,
Lydia

[Stephan Ewen]

Hi Lydia!

The same function instances (for example MapFunction objects) are used across all supersteps. If, for example, you store something in a HashMap inside some MapFunction, you can access that in the next iteration superstep.

You can figure out when a superstep finished and when the next superstep starts, by overriding "open()" and "close()" from the RichFunction interface.

Stephan

On Thu, Mar 31, 2016 at 4:45 PM, Till Rohrmann <[hidden email]> wrote:

I think I don't completely understand your question.

On Thu, Mar 31, 2016 at 4:40 PM, Lydia Ickler <[hidden email]> wrote:

Hi Till, 

thanks for your reply!

Is there a way to store intermediate results of the bulk iteration to use then in the next iteration except the data set one sends already by default?

Best regards, 

Lydia

Am 31.03.2016 um 12:01 schrieb Till Rohrmann <[hidden email]>:

Hi Lydia,

all downstream operators which depend on the bulk iteration will wait implicitly until data from the iteration operator is available.

Cheers,

Till

On Thu, Mar 31, 2016 at 9:39 AM, Lydia Ickler <[hidden email]> wrote:

Hi all,

is there a way to tell the program that it should wait until the BulkIteration finishes before the rest of the program is executed?

Best regards,
Lydia

[Lydia Ickler]

Hi,

thanks for your reply!

Could you please give me an example for the close() step? I can’t find an example online only for open().

There I can „save“ my new result?

Best regards,

Lydia

Am 31.03.2016 um 18:16 schrieb Stephan Ewen <[hidden email]>:

Hi Lydia!

The same function instances (for example MapFunction objects) are used across all supersteps. If, for example, you store something in a HashMap inside some MapFunction, you can access that in the next iteration superstep.

You can figure out when a superstep finished and when the next superstep starts, by overriding "open()" and "close()" from the RichFunction interface.

Stephan

On Thu, Mar 31, 2016 at 4:45 PM, Till Rohrmann <[hidden email]> wrote:

I think I don't completely understand your question.

On Thu, Mar 31, 2016 at 4:40 PM, Lydia Ickler <[hidden email]> wrote:

Hi Till, 

thanks for your reply!

Is there a way to store intermediate results of the bulk iteration to use then in the next iteration except the data set one sends already by default?

Best regards, 

Lydia

Am 31.03.2016 um 12:01 schrieb Till Rohrmann <[hidden email]>:

Hi Lydia,

all downstream operators which depend on the bulk iteration will wait implicitly until data from the iteration operator is available.

Cheers,

Till

On Thu, Mar 31, 2016 at 9:39 AM, Lydia Ickler <[hidden email]> wrote:

Hi all,

is there a way to tell the program that it should wait until the BulkIteration finishes before the rest of the program is executed?

Best regards,
Lydia