multiple consumer of intermediate data set

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

multiple consumer of intermediate data set

yoon

Hi All,

 

figure1
https://ci.apache.org/projects/flink/flink-docs-release-1.2/fig/job_and_execution_graph.svg

 

as we can see in figure1, JobVertex(B) has two consumer( JobVertex(C) and JobVertex(D) )

and accordingly Intermediate Data Set of JobVertex(B) has two consumer( JobVertex(C) and JobVertex(D) )
but in case of JobVertex(A), though it has two consumer( JobVertex(B) and JobVertex(D) ) same as JobVertex(B)

it has two separate intermediates data set and each intermediate data set has one consumer.
i couldn't understand why... for me it looks same case but why one has one Intermediate Data Set and another has two?
could anyone explain what is difference between JobVertex(A) and JobVertex(B)? 

Reply | Threaded
Open this post in threaded view
|

Re: multiple consumer of intermediate data set

Aljoscha Krettek

I think Ufuk (cc-ed) could know more about this>


On Mon, Mar 13, 2017, at 05:42, 윤형덕 wrote:

Hi All,

 

figure1

 

as we can see in figure1, JobVertex(B) has two consumer( JobVertex(C) and JobVertex(D) )

and accordingly Intermediate Data Set of JobVertex(B) has two consumer( JobVertex(C) and JobVertex(D) )
but in case of JobVertex(A), though it has two consumer( JobVertex(B) and JobVertex(D) ) same as JobVertex(B)

it has two separate intermediates data set and each intermediate data set has one consumer.
i couldn't understand why... for me it looks same case but why one has one Intermediate Data Set and another has two?
could anyone explain what is difference between JobVertex(A) and JobVertex(B)?