The rpc invocation size 13478509 exceeds the maximum akka framesize

[faaron zheng]

Hi all,

I find this exception when I run q71.sql in 10 T tpcds data with 400 parallelism. The flink version is 1.10, I try to increase the value of akka. framesize to 20485760b and it not works. flink will set sink parallelism with 1 when handle select query.  I'm not sure if that's the cause of the problem, but it works fine when I increase the sink parallelism to 400.

[Zhu Zhu]

Hi Zheng,

Would you check whether the option "akka.framesize" is properly set?

In your mail I saw a whitespace between "akka." and "framesize". Not sure if it is a mistake in the mail or in the config?

You can also check the printed config options in jobmaster log to see whether your configuration takes effect.

Thanks,

Zhu

zheng faaron <[hidden email]> 于2020年9月14日周一 上午11:57写道：

Hi all,

I find this exception when I run q71.sql in 10 T tpcds data with 400 parallelism. The flink version is 1.10, I try to increase the value of akka. framesize to 20485760b and it not works. flink will set sink parallelism with 1 when handle select query.  I'm not sure if that's the cause of the problem, but it works fine when I increase the sink parallelism to 400.

[faaron zheng]

Hi Zhu,

It's just a mistake in mail. It seems increase akka.framesize not works in this scenario.

Best,

Faaron Zheng

---

From: Zhu Zhu <[hidden email]>
Sent: Tuesday, September 15, 2020 11:04:06 AM
To: zheng faaron <[hidden email]>
Cc: user <[hidden email]>
Subject: Re: The rpc invocation size 13478509 exceeds the maximum akka framesize

 

Hi Zheng,

Would you check whether the option "akka.framesize" is properly set?

In your mail I saw a whitespace between "akka." and "framesize". Not sure if it is a mistake in the mail or in the config?

You can also check the printed config options in jobmaster log to see whether your configuration takes effect.

Thanks,

Zhu

zheng faaron <[hidden email]> 于2020年9月14日周一 上午11:57写道：

Hi all,

I find this exception when I run q71.sql in 10 T tpcds data with 400 parallelism. The flink version is 1.10, I try to increase the value of akka. framesize to 20485760b and it not works. flink will set sink parallelism with 1 when handle select query.  I'm not sure if that's the cause of the problem, but it works fine when I increase the sink parallelism to 400.

[Jake]

Hi zheng,

It seem’s data is large. Resizing the framesize of akka will not working.

You can increase the parallelism.

Jake.

On Sep 15, 2020, at 5:58 PM, zheng faaron <[hidden email]> wrote:

Hi Zhu,

It's just a mistake in mail. It seems increase akka.framesize not works in this scenario.

Best,

Faaron Zheng

---

From: Zhu Zhu <[hidden email]>
Sent: Tuesday, September 15, 2020 11:04:06 AM
To: zheng faaron <[hidden email]>
Cc: user <[hidden email]>
Subject: Re: The rpc invocation size 13478509 exceeds the maximum akka framesize

 

Hi Zheng,

Would you check whether the option "akka.framesize" is properly set?

In your mail I saw a whitespace between "akka." and "framesize". Not sure if it is a mistake in the mail or in the config?

You can also check the printed config options in jobmaster log to see whether your configuration takes effect.

Thanks,

Zhu

zheng faaron <[hidden email]> 于2020年9月14日周一 上午11:57写道：

Hi all,

I find this exception when I run q71.sql in 10 T tpcds data with 400 parallelism. The flink version is 1.10, I try to increase the value of akka. framesize to 20485760b and it not works. flink will set sink parallelism with 1 when handle select query.  I'm not sure if that's the cause of the problem, but it works fine when I increase the sink parallelism to 400.

<20200914-111346.png><20200914-111353.png>

[Zhu Zhu]

That's weird then. I checked the code and did some testing on 

the config parsing. It works as expected.

And looks to me the error you presented would happen only if 

Flink does not see your configuration "akka.framesize: 20485760b".

Would you share the printed config options in jobmaster log?

Thanks,

Zhu

zheng faaron <[hidden email]> 于2020年9月15日周二 下午5:58写道：

Hi Zhu,

It's just a mistake in mail. It seems increase akka.framesize not works in this scenario.

Best,

Faaron Zheng

---

From: Zhu Zhu <[hidden email]>
Sent: Tuesday, September 15, 2020 11:04:06 AM
To: zheng faaron <[hidden email]>
Cc: user <[hidden email]>
Subject: Re: The rpc invocation size 13478509 exceeds the maximum akka framesize

 

Hi Zheng,

Would you check whether the option "akka.framesize" is properly set?

In your mail I saw a whitespace between "akka." and "framesize". Not sure if it is a mistake in the mail or in the config?

You can also check the printed config options in jobmaster log to see whether your configuration takes effect.

Thanks,

Zhu

zheng faaron <[hidden email]> 于2020年9月14日周一 上午11:57写道：

Hi all,

I find this exception when I run q71.sql in 10 T tpcds data with 400 parallelism. The flink version is 1.10, I try to increase the value of akka. framesize to 20485760b and it not works. flink will set sink parallelism with 1 when handle select query.  I'm not sure if that's the cause of the problem, but it works fine when I increase the sink parallelism to 400.

[faaron zheng]

Thanks, the problem has been solved. I make another mistake. It works by increasing akka.framesize.

Best,

Faaron Zheng

---

From: Zhu Zhu <[hidden email]>
Sent: Tuesday, September 15, 2020 7:12:40 PM
To: zheng faaron <[hidden email]>
Cc: user <[hidden email]>
Subject: Re: The rpc invocation size 13478509 exceeds the maximum akka framesize

 

That's weird then. I checked the code and did some testing on 

the config parsing. It works as expected.

And looks to me the error you presented would happen only if 

Flink does not see your configuration "akka.framesize: 20485760b".

Would you share the printed config options in jobmaster log?

Thanks,

Zhu

zheng faaron <[hidden email]> 于2020年9月15日周二 下午5:58写道：

Hi Zhu,

It's just a mistake in mail. It seems increase akka.framesize not works in this scenario.

Best,

Faaron Zheng

---

From: Zhu Zhu <[hidden email]>
Sent: Tuesday, September 15, 2020 11:04:06 AM
To: zheng faaron <[hidden email]>
Cc: user <[hidden email]>
Subject: Re: The rpc invocation size 13478509 exceeds the maximum akka framesize

 

Hi Zheng,

Would you check whether the option "akka.framesize" is properly set?

In your mail I saw a whitespace between "akka." and "framesize". Not sure if it is a mistake in the mail or in the config?

You can also check the printed config options in jobmaster log to see whether your configuration takes effect.

Thanks,

Zhu

zheng faaron <[hidden email]> 于2020年9月14日周一 上午11:57写道：

Hi all,

I find this exception when I run q71.sql in 10 T tpcds data with 400 parallelism. The flink version is 1.10, I try to increase the value of akka. framesize to 20485760b and it not works. flink will set sink parallelism with 1 when handle select query.  I'm not sure if that's the cause of the problem, but it works fine when I increase the sink parallelism to 400.