Job Distribution Strategy On Cluster.

[srikanth flink]

Hi there,

I'm running Flink with 3 node cluster.

While running my jobs(both SQL client and jar submission), the jobs are being assigned to single machine instead of distribution among the cluster. How could I achieve the job distribution to make use of the computation power?

Thanks

Srikanth

[vino yang]

Hi srikanth,

What's your job's parallelism?

In some scenes, many operators are chained with each other. if it's parallelism is 1, it would just use a single slot.

Best,

Vino

srikanth flink <[hidden email]> 于2019年11月6日周三 下午10:03写道：

Hi there,

I'm running Flink with 3 node cluster.

While running my jobs(both SQL client and jar submission), the jobs are being assigned to single machine instead of distribution among the cluster. How could I achieve the job distribution to make use of the computation power?

Thanks

Srikanth

[Zhu Zhu]

Hi Srikanth,

Is this issue what you encounter? FLINK-12122: a job would tend to fill one TM before using another.

If it is, you may need to wait for the release 1.9.2 or 1.10, since it is just fixed.

Thanks,

Zhu Zhu 

vino yang <[hidden email]> 于2019年11月11日周一 下午5:48写道：

Hi srikanth,

What's your job's parallelism?

In some scenes, many operators are chained with each other. if it's parallelism is 1, it would just use a single slot.

Best,

Vino

srikanth flink <[hidden email]> 于2019年11月6日周三 下午10:03写道：

Hi there,

I'm running Flink with 3 node cluster.

While running my jobs(both SQL client and jar submission), the jobs are being assigned to single machine instead of distribution among the cluster. How could I achieve the job distribution to make use of the computation power?

Thanks

Srikanth

[srikanth flink]

Vina,

I've set parallelism as 6 while max parallelism is 128.

Thanks

Srikanth

On Mon, Nov 11, 2019 at 3:18 PM vino yang <[hidden email]> wrote:

Hi srikanth,

What's your job's parallelism?

In some scenes, many operators are chained with each other. if it's parallelism is 1, it would just use a single slot.

Best,

Vino

srikanth flink <[hidden email]> 于2019年11月6日周三 下午10:03写道：

Hi there,

I'm running Flink with 3 node cluster.

While running my jobs(both SQL client and jar submission), the jobs are being assigned to single machine instead of distribution among the cluster. How could I achieve the job distribution to make use of the computation power?

Thanks

Srikanth

[srikanth flink]

Zhu Zhu,

That's awesome and is what I'm looking for.

Any update on when would be the next release date?

Thanks

Srikanth

On Mon, Nov 11, 2019 at 3:40 PM Zhu Zhu <[hidden email]> wrote:

Hi Srikanth,

Is this issue what you encounter? FLINK-12122: a job would tend to fill one TM before using another.

If it is, you may need to wait for the release 1.9.2 or 1.10, since it is just fixed.

Thanks,

Zhu Zhu 

vino yang <[hidden email]> 于2019年11月11日周一 下午5:48写道：

Hi srikanth,

What's your job's parallelism?

In some scenes, many operators are chained with each other. if it's parallelism is 1, it would just use a single slot.

Best,

Vino

srikanth flink <[hidden email]> 于2019年11月6日周三 下午10:03写道：

Hi there,

I'm running Flink with 3 node cluster.

While running my jobs(both SQL client and jar submission), the jobs are being assigned to single machine instead of distribution among the cluster. How could I achieve the job distribution to make use of the computation power?

Thanks

Srikanth

[Zhu Zhu]

There is no plan for release 1.9.2 yet.

Flink 1.10.0 is planned to be released in early January.

Thanks,

Zhu Zhu

srikanth flink <[hidden email]> 于2019年11月11日周一 下午9:53写道：

Zhu Zhu,

That's awesome and is what I'm looking for.

Any update on when would be the next release date?

Thanks

Srikanth

On Mon, Nov 11, 2019 at 3:40 PM Zhu Zhu <[hidden email]> wrote:

Hi Srikanth,

Is this issue what you encounter? FLINK-12122: a job would tend to fill one TM before using another.

If it is, you may need to wait for the release 1.9.2 or 1.10, since it is just fixed.

Thanks,

Zhu Zhu 

vino yang <[hidden email]> 于2019年11月11日周一 下午5:48写道：

Hi srikanth,

What's your job's parallelism?

In some scenes, many operators are chained with each other. if it's parallelism is 1, it would just use a single slot.

Best,

Vino

srikanth flink <[hidden email]> 于2019年11月6日周三 下午10:03写道：

Hi there,

I'm running Flink with 3 node cluster.

While running my jobs(both SQL client and jar submission), the jobs are being assigned to single machine instead of distribution among the cluster. How could I achieve the job distribution to make use of the computation power?

Thanks

Srikanth

[srikanth flink]

Great, thanks for the update.

On Tue, Nov 12, 2019 at 8:51 AM Zhu Zhu <[hidden email]> wrote:

There is no plan for release 1.9.2 yet.

Flink 1.10.0 is planned to be released in early January.

Thanks,

Zhu Zhu

srikanth flink <[hidden email]> 于2019年11月11日周一 下午9:53写道：

Zhu Zhu,

That's awesome and is what I'm looking for.

Any update on when would be the next release date?

Thanks

Srikanth

On Mon, Nov 11, 2019 at 3:40 PM Zhu Zhu <[hidden email]> wrote:

Hi Srikanth,

Is this issue what you encounter? FLINK-12122: a job would tend to fill one TM before using another.

If it is, you may need to wait for the release 1.9.2 or 1.10, since it is just fixed.

Thanks,

Zhu Zhu 

vino yang <[hidden email]> 于2019年11月11日周一 下午5:48写道：

Hi srikanth,

What's your job's parallelism?

In some scenes, many operators are chained with each other. if it's parallelism is 1, it would just use a single slot.

Best,

Vino

srikanth flink <[hidden email]> 于2019年11月6日周三 下午10:03写道：

Hi there,

I'm running Flink with 3 node cluster.

While running my jobs(both SQL client and jar submission), the jobs are being assigned to single machine instead of distribution among the cluster. How could I achieve the job distribution to make use of the computation power?

Thanks

Srikanth