How is Collector<T> out element processed?

[son]

Hi all,

when I create new classes extending ProcessFunction or implementing WindowFunction, there is a Collector<T> out for output.

How is this output processed in the next stage, for example a Sink or another WindowAssigner? Is it processed immediately by the next operator by push mechanism, or is the collector checked regularly by the next operator to see if an element exists?

Thanks,

Son

[tison]

Hi Son,

As from Collector's document, it collects a record and forwards it.

The collector is the "push" counterpart of the Iterator which "pulls" data in.

Son Mai <[hidden email]> 于2019年4月4日周四 上午10:15写道：

Hi all,

when I create new classes extending ProcessFunction or implementing WindowFunction, there is a Collector<T> out for output.

How is this output processed in the next stage, for example a Sink or another WindowAssigner? Is it processed immediately by the next operator by push mechanism, or is the collector checked regularly by the next operator to see if an element exists?

Thanks,

Son

[son]

Hi Tison,

so are you saying that the output will be iterated on when the next operator that called them? and they are not processed in push but pull mechanism by the next operator like sink?

Thanks,

On Thu, Apr 4, 2019 at 9:46 AM Zili Chen <[hidden email]> wrote:

Hi Son,

As from Collector's document, it collects a record and forwards it.

The collector is the "push" counterpart of the Iterator which "pulls" data in.

Son Mai <[hidden email]> 于2019年4月4日周四 上午10:15写道：

Hi all,

when I create new classes extending ProcessFunction or implementing WindowFunction, there is a Collector<T> out for output.

How is this output processed in the next stage, for example a Sink or another WindowAssigner? Is it processed immediately by the next operator by push mechanism, or is the collector checked regularly by the next operator to see if an element exists?

Thanks,

Son

[tison]

Collector should follow the "push" mechanism.

Son Mai <[hidden email]> 于2019年4月4日周四 下午12:11写道：

Hi Tison,

so are you saying that the output will be iterated on when the next operator that called them? and they are not processed in push but pull mechanism by the next operator like sink?

Thanks,

On Thu, Apr 4, 2019 at 9:46 AM Zili Chen <[hidden email]> wrote:

Hi Son,

As from Collector's document, it collects a record and forwards it.

The collector is the "push" counterpart of the Iterator which "pulls" data in.

Son Mai <[hidden email]> 于2019年4月4日周四 上午10:15写道：

Hi all,

when I create new classes extending ProcessFunction or implementing WindowFunction, there is a Collector<T> out for output.

How is this output processed in the next stage, for example a Sink or another WindowAssigner? Is it processed immediately by the next operator by push mechanism, or is the collector checked regularly by the next operator to see if an element exists?

Thanks,

Son