akka.remote.OversizedPayloadException

[Alex Soto]

Hello,

I am using Flink version 1.4.2.  When I try to run my application on a Yarn cluster, I get this error:

2018-04-23 14:08:13,346 ERROR akka.remote.EndpointWriter                                    - Transient association error (association remains live)
akka.remote.OversizedPayloadException: Discarding oversized payload sent to Actor[<a href="akka.tcp://flink@XXXX:37877/user/jobmanager#1227317945]:" class="">akka.tcp://flink@XXXX:37877/user/jobmanager#1227317945]: max allowed size 10485760 bytes, actual size of encoded class org.apache.flink.runtime.messages.JobManagerMessages$LeaderSessionMessage was 213691138 bytes.

I am trying to override the default limit in my flink-conf.yaml file:

env.java.opts: -Dakka.framesize=314572800b -Dakka.client.timeout=10min

I can verify this setting shows up in the Job Manager Web UI:

Key Value

env.java.opts -Dakka.framesize=314572800b -Dakka.client.timeout=10min

I am submitting the Job with the command line client:

flink run -m …

No matter what I do, I always get the same error message with the same default limit.  Please help, I am not finding any information on how to override the default limit.

Best regards,
Alex soto

[Ted Yu]

I noticed trailing 'b' in your setting.

Have you tried without the trailing 'b' ?

Cheers

On Mon, Apr 23, 2018 at 11:48 AM, Alex Soto <[hidden email]> wrote:

Hello,

I am using Flink version 1.4.2.  When I try to run my application on a Yarn cluster, I get this error:

2018-04-23 14:08:13,346 ERROR akka.remote.EndpointWriter                                    - Transient association error (association remains live)
akka.remote.OversizedPayloadException: Discarding oversized payload sent to Actor[akka.tcp://flink@XXXX:37877/user/jobmanager#1227317945]: max allowed size 10485760 bytes, actual size of encoded class org.apache.flink.runtime.messages.JobManagerMessages$LeaderSessionMessage was 213691138 bytes.

I am trying to override the default limit in my flink-conf.yaml file:

env.java.opts: -Dakka.framesize=314572800b -Dakka.client.timeout=10min

I can verify this setting shows up in the Job Manager Web UI:

Key Value

env.java.opts -Dakka.framesize=314572800b -Dakka.client.timeout=10min

I am submitting the Job with the command line client:

flink run -m …

No matter what I do, I always get the same error message with the same default limit.  Please help, I am not finding any information on how to override the default limit.

Best regards,
Alex soto

[Gary Yao-2]

Hi Alex,

Can you try add the following two lines to your flink-conf.yaml?

  akka.framesize: 314572800b
  akka.client.timeout: 10min

AFAIK it is not needed to use Java system properties here.

Best,
Gary

On Mon, Apr 23, 2018 at 8:48 PM, Alex Soto <[hidden email]> wrote:

Hello,

I am using Flink version 1.4.2.  When I try to run my application on a Yarn cluster, I get this error:

2018-04-23 14:08:13,346 ERROR akka.remote.EndpointWriter                                    - Transient association error (association remains live)
akka.remote.OversizedPayloadException: Discarding oversized payload sent to Actor[akka.tcp://flink@XXXX:37877/user/jobmanager#1227317945]: max allowed size 10485760 bytes, actual size of encoded class org.apache.flink.runtime.messages.JobManagerMessages$LeaderSessionMessage was 213691138 bytes.

I am trying to override the default limit in my flink-conf.yaml file:

env.java.opts: -Dakka.framesize=314572800b -Dakka.client.timeout=10min

I can verify this setting shows up in the Job Manager Web UI:

Key Value

env.java.opts -Dakka.framesize=314572800b -Dakka.client.timeout=10min

I am submitting the Job with the command line client:

flink run -m …

No matter what I do, I always get the same error message with the same default limit.  Please help, I am not finding any information on how to override the default limit.

Best regards,
Alex soto