What are channels mapped to?

[Chris Miller]

Hi,

in the OutputEmitter, the output channel can be selected in different manner. 

eg. OutputEmitter#hashPartitionDefault()

What are the channels mapped to? Do they map to one IP Address or Port?

 

Thanks.

 

Chris

 

 

[Zhijiang(wangzhijiang999)]

The channels are mapped to the subpartition index which would be consumed by specific downstream task parallelism.

For example, if there are three reduce tasks parallelism, every map task would generate three subpartitions. If one record is hashed to the first channel, that means this record will be consumed by the first reduce task.

Best,

Zhijiang

------------------------------------------------------------------

发件人：Chris Miller <[hidden email]>

发送时间：2018年10月11日(星期四) 16:54

收件人：user <[hidden email]>

主　题：What are channels mapped to?

Hi,

in the OutputEmitter, the output channel can be selected in different manner. 

eg. OutputEmitter#hashPartitionDefault()

What are the channels mapped to? Do they map to one IP Address or Port?

 

Thanks.

 

Chris