Question about akka configuration for FLIP-6

[Tony Wei]

Hi,

I'm going to migrate my flink cluster from 1.4.0 to 1.5.3, and I have been trying to map config file

to the latest version. I used to use these three configuration. Are they still needed in FLIP-6 mode?

Moreover, is any akka config still needed in FLIP-6 mode? Since I had a impression that FLIP-6

tried to get rid of akka and use its own rpc interface. Please correct me if I misunderstood. Thanks.

akka.watch.heartbeat.interval

akka.watch.heartbeat.pause

taskmanager.exit-on-fatal-akka-error

Best Regards,

Tony Wei

[Gary Yao-2]

Hi Tony,

You are right that with FLIP-6 Akka is abstracted away. If you want custom
heartbeat settings, you can configure the options below [1]:

    heatbeat.interval
    heartbeat.timeout

The config option taskmanager.exit-on-fatal-akka-error is also not relevant
anymore. I closest I can think of is taskmanager.registration.timeout [2].

Best,
Gary


[1] https://ci.apache.org/projects/flink/flink-docs-release-1.5/ops/config.html#heartbeat-manager
[2] https://ci.apache.org/projects/flink/flink-docs-release-1.5/ops/config.html#taskmanager-registration-timeout

On Mon, Sep 10, 2018 at 4:24 AM, Tony Wei <[hidden email]> wrote:

Hi,

I'm going to migrate my flink cluster from 1.4.0 to 1.5.3, and I have been trying to map config file

to the latest version. I used to use these three configuration. Are they still needed in FLIP-6 mode?

Moreover, is any akka config still needed in FLIP-6 mode? Since I had a impression that FLIP-6

tried to get rid of akka and use its own rpc interface. Please correct me if I misunderstood. Thanks.

akka.watch.heartbeat.interval

akka.watch.heartbeat.pause

taskmanager.exit-on-fatal-akka-error

Best Regards,

Tony Wei

[Gary Yao-2]

I should add that in FLIP-6 mode we are not relying on Akka's DeathWatch but
because Flink's RPC framework uses Akka, you are still able to configure the
other Akka config options [1].

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.5/ops/config.html#distributed-coordination-via-akka

On Mon, Sep 10, 2018 at 7:38 AM, Gary Yao <[hidden email]> wrote:

Hi Tony,

You are right that with FLIP-6 Akka is abstracted away. If you want custom
heartbeat settings, you can configure the options below [1]:

    heatbeat.interval
    heartbeat.timeout

The config option taskmanager.exit-on-fatal-akka-error is also not relevant
anymore. I closest I can think of is taskmanager.registration.timeout [2].

Best,
Gary


[1] https://ci.apache.org/projects/flink/flink-docs-release-1.5/ops/config.html#heartbeat-manager
[2] https://ci.apache.org/projects/flink/flink-docs-release-1.5/ops/config.html#taskmanager-registration-timeout

On Mon, Sep 10, 2018 at 4:24 AM, Tony Wei <[hidden email]> wrote:

Hi,

I'm going to migrate my flink cluster from 1.4.0 to 1.5.3, and I have been trying to map config file

to the latest version. I used to use these three configuration. Are they still needed in FLIP-6 mode?

Moreover, is any akka config still needed in FLIP-6 mode? Since I had a impression that FLIP-6

tried to get rid of akka and use its own rpc interface. Please correct me if I misunderstood. Thanks.

akka.watch.heartbeat.interval

akka.watch.heartbeat.pause

taskmanager.exit-on-fatal-akka-error

Best Regards,

Tony Wei

[tison]

Hi Gray,

Thanks for your useful information! Here I wonder if the following configs still valid on FLIP-6 mode.

1. akka.transport.heartbeat.interval

2. akka.transport.heartbeat.pause

It seems they are different from HeartbeatServices and possibly still valid.

Gary Yao <[hidden email]> 于2018年9月10日周一 下午1:50写道：

I should add that in FLIP-6 mode we are not relying on Akka's DeathWatch but
because Flink's RPC framework uses Akka, you are still able to configure the
other Akka config options [1].

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.5/ops/config.html#distributed-coordination-via-akka

On Mon, Sep 10, 2018 at 7:38 AM, Gary Yao <[hidden email]> wrote:

Hi Tony,

You are right that with FLIP-6 Akka is abstracted away. If you want custom
heartbeat settings, you can configure the options below [1]:

    heatbeat.interval
    heartbeat.timeout

The config option taskmanager.exit-on-fatal-akka-error is also not relevant
anymore. I closest I can think of is taskmanager.registration.timeout [2].

Best,
Gary


[1] https://ci.apache.org/projects/flink/flink-docs-release-1.5/ops/config.html#heartbeat-manager
[2] https://ci.apache.org/projects/flink/flink-docs-release-1.5/ops/config.html#taskmanager-registration-timeout

On Mon, Sep 10, 2018 at 4:24 AM, Tony Wei <[hidden email]> wrote:

Hi,

I'm going to migrate my flink cluster from 1.4.0 to 1.5.3, and I have been trying to map config file

to the latest version. I used to use these three configuration. Are they still needed in FLIP-6 mode?

Moreover, is any akka config still needed in FLIP-6 mode? Since I had a impression that FLIP-6

tried to get rid of akka and use its own rpc interface. Please correct me if I misunderstood. Thanks.

akka.watch.heartbeat.interval

akka.watch.heartbeat.pause

taskmanager.exit-on-fatal-akka-error

Best Regards,

Tony Wei

[Gary Yao-2]

Hi Tison,

These can be still be set but judging from the documentation [1], they have
never been very relevant in Flink:

    Heartbeat interval for Akka's transport failure detector. Since Flink uses
    TCP, the detector is not necessary. Therefore, the detector is disabled by
    setting the interval to a very high value. In case you should need the
    transport failure detector, set the interval to some reasonable value. The
    interval value requires a time-unit specifier (ms/s/min/h/d).

If you have a good reason to use the transport failure detector, I would be
interested in it.

Best,
Gary

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.5/ops/config.html#akka-transport-heartbeat-interval

On Mon, Sep 10, 2018 at 8:20 AM, 陈梓立 <[hidden email]> wrote:

Hi Gray,

Thanks for your useful information! Here I wonder if the following configs still valid on FLIP-6 mode.

1. akka.transport.heartbeat.interval

2. akka.transport.heartbeat.pause

It seems they are different from HeartbeatServices and possibly still valid.

Gary Yao <[hidden email]> 于2018年9月10日周一 下午1:50写道：

I should add that in FLIP-6 mode we are not relying on Akka's DeathWatch but
because Flink's RPC framework uses Akka, you are still able to configure the
other Akka config options [1].

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.5/ops/config.html#distributed-coordination-via-akka

On Mon, Sep 10, 2018 at 7:38 AM, Gary Yao <[hidden email]> wrote:

Hi Tony,

You are right that with FLIP-6 Akka is abstracted away. If you want custom
heartbeat settings, you can configure the options below [1]:

    heatbeat.interval
    heartbeat.timeout

The config option taskmanager.exit-on-fatal-akka-error is also not relevant
anymore. I closest I can think of is taskmanager.registration.timeout [2].

Best,
Gary


[1] https://ci.apache.org/projects/flink/flink-docs-release-1.5/ops/config.html#heartbeat-manager
[2] https://ci.apache.org/projects/flink/flink-docs-release-1.5/ops/config.html#taskmanager-registration-timeout

On Mon, Sep 10, 2018 at 4:24 AM, Tony Wei <[hidden email]> wrote:

Hi,

I'm going to migrate my flink cluster from 1.4.0 to 1.5.3, and I have been trying to map config file

to the latest version. I used to use these three configuration. Are they still needed in FLIP-6 mode?

Moreover, is any akka config still needed in FLIP-6 mode? Since I had a impression that FLIP-6

tried to get rid of akka and use its own rpc interface. Please correct me if I misunderstood. Thanks.

akka.watch.heartbeat.interval

akka.watch.heartbeat.pause

taskmanager.exit-on-fatal-akka-error

Best Regards,

Tony Wei