Programmatic way to determine required number of slots for a job?

[Jared Stehler]

Is there a way to determine via an API the required number of task manager slots for a particular job graph? I’ve looked at "flink.getStreamGraph().getJobGraph().getNumberOfVertices()” but it doesn’t seem to correspond with what the job actually ends up using.

--
Jared Stehler
Chief Architect - Intellify Learning
o: 617.701.6330 x703

[Jared Stehler]

I found in the “Flip6LocalStreamEnvironment” the following code, but with the caveat that it doesn’t include slot reuse?

int slotsCount = 0;

for (org.apache.flink.runtime.jobgraph.JobVertex jobVertex : flink.getStreamGraph().getJobGraph().getVertices()) {

slotsCount += jobVertex.getParallelism();

}

--

Jared Stehler
Chief Architect - Intellify Learning

o: 617.701.6330 x703

On Oct 27, 2017, at 2:30 PM, Jared Stehler <[hidden email]> wrote:

Is there a way to determine via an API the required number of task manager slots for a particular job graph? I’ve looked at "flink.getStreamGraph().getJobGraph().getNumberOfVertices()” but it doesn’t seem to correspond with what the job actually ends up using.

--
Jared Stehler
Chief Architect - Intellify Learning
o: 617.701.6330 x703

[Till Rohrmann]

Hi Jared,

I think by adding up the parallelism of the operator with the maximum degree of parallelism in each slot sharing group, you will end up with the correct number of required slots (given that all operators have to run concurrently).

Cheers,

Till

On Fri, Oct 27, 2017 at 8:34 PM, Jared Stehler <[hidden email]> wrote:

I found in the “Flip6LocalStreamEnvironment” the following code, but with the caveat that it doesn’t include slot reuse?

int slotsCount = 0;

for (org.apache.flink.runtime.jobgraph.JobVertex jobVertex : flink.getStreamGraph().getJobGraph().getVertices()) {

slotsCount += jobVertex.getParallelism();

}

--

Jared Stehler
Chief Architect - Intellify Learning

o: <a href="tel:(617)%20701-6330" value="[phone]" target="_blank">617.701.6330 x703

On Oct 27, 2017, at 2:30 PM, Jared Stehler <[hidden email]> wrote:

Is there a way to determine via an API the required number of task manager slots for a particular job graph? I’ve looked at "flink.getStreamGraph().getJobGraph().getNumberOfVertices()” but it doesn’t seem to correspond with what the job actually ends up using.

--
Jared Stehler
Chief Architect - Intellify Learning
o: <a href="tel:(617)%20701-6330" value="[phone]" target="_blank">617.701.6330 x703