Flink job parallelism

[Vishwas Siravara]

Hi guys,

I have a flink job which I want to run with a parallelism of 2. 

I run it from command line like : flink run -p 2 -C file:///home/was/classpathconfig/ -c com.visa.flink.cli.Main flink-job-assembly-0.1-SNAPSHOT.jar flink druid

My cluster has two task managers with only 1 task slot each. 

However when I look at the Web UI for my job , I see that one of the task managers is still available. But when I submit with the web UI , both the task managers are used for this job and I get a parallelism of 2. 

Can you help me with understanding as to why this happens ? 

Thank you

Vishwas 

[Biao Liu]

Hi Vishwas,

Regardless of the available task manager, what's your job really look like? Is it with parallelism 2 or 1?

It's hard to say what happened based on your description. Could you reproduce the scenario?

If the answer is yes, then could you provide more details? Like a screenshot, logs, codes of your example.

On Thu, 15 Aug 2019 at 23:54, Vishwas Siravara <[hidden email]> wrote:

Hi guys,

I have a flink job which I want to run with a parallelism of 2. 

I run it from command line like : flink run -p 2 -C file:///home/was/classpathconfig/ -c com.visa.flink.cli.Main flink-job-assembly-0.1-SNAPSHOT.jar flink druid

My cluster has two task managers with only 1 task slot each. 

However when I look at the Web UI for my job , I see that one of the task managers is still available. But when I submit with the web UI , both the task managers are used for this job and I get a parallelism of 2. 

Can you help me with understanding as to why this happens ? 

Thank you

Vishwas