Flink Streaming Data Source Node

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Flink Streaming Data Source Node

Adrienne Kole
Hi,

I recognized that, if  the number of data input sources are less than or equal to number of slots in one node, they (input source operators) are all deployed in the same node. What is the logic behind this? Can't this be a bottleneck for throughput and distribution of input sources?

Thanks
Adrienne
Reply | Threaded
Open this post in threaded view
|

Re: Flink Streaming Data Source Node

rmetzger0
Hi,
I'm not sure if I fully understood your question. The number of input sources is always less or equal to the number of slots in one node.

Usually source instances are equally distributed among the parallel workers (TaskManagers). Maybe this document describing the deployment model is also helpful: https://ci.apache.org/projects/flink/flink-docs-release-1.2/concepts/index.html#workers-slots-resources



On Tue, Nov 22, 2016 at 10:14 AM, Adrienne Kole <[hidden email]> wrote:
Hi,

I recognized that, if  the number of data input sources are less than or equal to number of slots in one node, they (input source operators) are all deployed in the same node. What is the logic behind this? Can't this be a bottleneck for throughput and distribution of input sources?

Thanks
Adrienne