Flink issue while setting up in Kubernetes
Locked
Flink issue while setting up in Kubernetes
 
|
We are trying to setup a single node Kubernetes cluster. 1 Job Manager and 1 Task Manager. Before we were getting an error, and we followed this thread. After following the above mentioned archive , we have used the following commands to startup the Flink services : ${FLINK_HOME}/bin/jobmanager.sh start-foreground ${FLINK_HOME}/bin/taskmanager.sh start-foreground Previously jobmanager was being started as : ${FLINK_HOME}/bin/jobmanager.sh start-foreground
cluster ${FLINK_HOME}/bin/taskmanager.sh start-foreground It removed that error and now we are getting this error as shown below. We searched all archives and have a dead end. We have set up all ports correctly. Flink version used is 1.6.2.
Thanks in advance. 2018-12-08 06:52:38,959 WARN akka.remote.ReliableDeliverySupervisor - Association with remote system [akka.tcp://flink@flink-taskmanager-6cf55db87b-5x9sd:6122] has failed, address
is now gated for [50] ms. Reason: [Disassociated] 2018-12-08 06:52:41,863 INFO org.apache.flink.runtime.resourcemanager.StandaloneResourceManager - Registering TaskManager with ResourceID 037d1c33ec0406598f2ce30472f97e65
(akka.tcp://flink@flink-taskmanager-6cf55db87b-5x9sd:6122/user/taskmanager_0) at ResourceManager
2018-12-08 06:53:23,619 INFO org.apache.flink.runtime.resourcemanager.StandaloneResourceManager - The heartbeat of TaskManager with id e0383ee248832f639659082c70a2f4e9
timed out. 2018-12-08 06:53:23,619 INFO org.apache.flink.runtime.resourcemanager.StandaloneResourceManager - Closing TaskExecutor connection e0383ee248832f639659082c70a2f4e9
because: The heartbeat of TaskManager with id e0383ee248832f639659082c70a2f4e9 timed out.
2018-12-08 06:53:48,961 WARN akka.remote.ReliableDeliverySupervisor - Association with remote system [akka.tcp://flink@flink-taskmanager-6cf55db87b-5x9sd:6122] has
failed, address is now gated for [50] ms. Reason: [Disassociated] 2018-12-08 06:53:53,615 WARN akka.remote.ReliableDeliverySupervisor - Association with remote system [akka.tcp://flink@flink-taskmanager-6cf55db87b-5x9sd:6122] has
failed, address is now gated for [50] ms. Reason: [Association failed with [akka.tcp://flink@flink-taskmanager-6cf55db87b-5x9sd:6122]] Caused by: [flink-taskmanager-6cf55db87b-5x9sd: Name or service not known]
2018-12-08 06:54:03,601 WARN akka.remote.ReliableDeliverySupervisor - Association with remote system [akka.tcp://flink@flink-taskmanager-6cf55db87b-5x9sd:6122] has
failed, address is now gated for [50] ms. Reason: [Association failed with [akka.tcp://flink@flink-taskmanager-6cf55db87b-5x9sd:6122]] Caused by: [flink-taskmanager-6cf55db87b-5x9sd]
2018-12-08 06:54:13,605 WARN akka.remote.ReliableDeliverySupervisor - Association with remote system [akka.tcp://flink@flink-taskmanager-6cf55db87b-5x9sd:6122] has
failed, address is now gated for [50] ms. Reason: [Association failed with [akka.tcp://flink@flink-taskmanager-6cf55db87b-5x9sd:6122]] Caused by: [flink-taskmanager-6cf55db87b-5x9sd: Name or service not known]
2018-12-08 06:54:23,613 WARN akka.remote.ReliableDeliverySupervisor - Association with remote system [akka.tcp://flink@flink-taskmanager-6cf55db87b-5x9sd:6122] has
failed, address is now gated for [50] ms. Reason: [Association failed with [akka.tcp://flink@flink-taskmanager-6cf55db87b-5x9sd:6122]] Caused by: [flink-taskmanager-6cf55db87b-5x9sd: Name or service not known]
2018-12-08 06:54:33,601 WARN akka.remote.ReliableDeliverySupervisor - Association with remote system [akka.tcp://flink@flink-taskmanager-6cf55db87b-5x9sd:6122] has
failed, address is now gated for [50] ms. Reason: [Association failed with [akka.tcp://flink@flink-taskmanager-6cf55db87b-5x9sd:6122]] Caused by: [flink-taskmanager-6cf55db87b-5x9sd]
2018-12-08 06:54:33,619 INFO org.apache.flink.runtime.resourcemanager.StandaloneResourceManager - The heartbeat of TaskManager with id 037d1c33ec0406598f2ce30472f97e65
timed out. 2018-12-08 06:54:33,619 INFO org.apache.flink.runtime.resourcemanager.StandaloneResourceManager - Closing TaskExecutor connection 037d1c33ec0406598f2ce30472f97e65
because: The heartbeat of TaskManager with id 037d1c33ec0406598f2ce30472f97e65 timed out.
2018-12-08 06:59:04,160 INFO org.apache.flink.runtime.resourcemanager.StandaloneResourceManager - Registering TaskManager with ResourceID 0c12c8035663c0b7bc94d520b3c6ff0a
(akka.tcp://flink@flink-taskmanager-6cf55db87b-5x9sd:6122/user/taskmanager_0) at ResourceManager
2018-12-08 07:00:08,983 WARN akka.remote.ReliableDeliverySupervisor - Association with remote system [akka.tcp://flink@flink-taskmanager-6cf55db87b-5x9sd:6122] has
failed, address is now gated for [50] ms. Reason: [Disassociated] 2018-12-08 07:00:12,040 INFO org.apache.flink.runtime.resourcemanager.StandaloneResourceManager - Registering TaskManager with ResourceID 8949a9d85ec65b207654b28bf60ba10c
(akka.tcp://flink@flink-taskmanager-6cf55db87b-5x9sd:6122/user/taskmanager_0) at ResourceManager
2018-12-08 07:00:53,619 INFO org.apache.flink.runtime.resourcemanager.StandaloneResourceManager - The heartbeat of TaskManager with id 0c12c8035663c0b7bc94d520b3c6ff0a
timed out. 2018-12-08 07:00:53,619 INFO org.apache.flink.runtime.resourcemanager.StandaloneResourceManager - Closing TaskExecutor connection 0c12c8035663c0b7bc94d520b3c6ff0a
because: The heartbeat of TaskManager with id 0c12c8035663c0b7bc94d520b3c6ff0a timed out.
2018-12-08 07:01:18,969 WARN akka.remote.ReliableDeliverySupervisor - Association with remote system [akka.tcp://flink@flink-taskmanager-6cf55db87b-5x9sd:6122] has
failed, address is now gated for [50] ms. Reason: [Disassociated] 2018-12-08 07:01:23,604 WARN akka.remote.ReliableDeliverySupervisor - Association with remote system [akka.tcp://flink@flink-taskmanager-6cf55db87b-5x9sd:6122] has
failed, address is now gated for [50] ms. Reason: [Association failed with [akka.tcp://flink@flink-taskmanager-6cf55db87b-5x9sd:6122]] Caused by: [flink-taskmanager-6cf55db87b-5x9sd: Name or service not known]
2018-12-08 07:01:33,601 WARN akka.remote.ReliableDeliverySupervisor - Association with remote system [akka.tcp://flink@flink-taskmanager-6cf55db87b-5x9sd:6122] has
failed, address is now gated for [50] ms. Reason: [Association failed with [akka.tcp://flink@flink-taskmanager-6cf55db87b-5x9sd:6122]] Caused by: [flink-taskmanager-6cf55db87b-5x9sd]
2018-12-08 07:01:43,615 WARN akka.remote.ReliableDeliverySupervisor - Association with remote system [akka.tcp://flink@flink-taskmanager-6cf55db87b-5x9sd:6122] has
failed, address is now gated for [50] ms. Reason: [Association failed with [akka.tcp://flink@flink-taskmanager-6cf55db87b-5x9sd:6122]] Caused by: [flink-taskmanager-6cf55db87b-5x9sd: Name or service not known]
2018-12-08 07:01:53,601 WARN akka.remote.ReliableDeliverySupervisor - Association with remote system [akka.tcp://flink@flink-taskmanager-6cf55db87b-5x9sd:6122] has
failed, address is now gated for [50] ms. Reason: [Association failed with [akka.tcp://flink@flink-taskmanager-6cf55db87b-5x9sd:6122]] Caused by: [flink-taskmanager-6cf55db87b-5x9sd]
2018-12-08 07:02:03,614 WARN akka.remote.ReliableDeliverySupervisor - Association with remote system [akka.tcp://flink@flink-taskmanager-6cf55db87b-5x9sd:6122] has
failed, address is now gated for [50] ms. Reason: [Association failed with [akka.tcp://flink@flink-taskmanager-6cf55db87b-5x9sd:6122]] Caused by: [flink-taskmanager-6cf55db87b-5x9sd: Name or service not known]
2018-12-08 07:02:03,619 INFO org.apache.flink.runtime.resourcemanager.StandaloneResourceManager - The heartbeat of TaskManager with id 8949a9d85ec65b207654b28bf60ba10c
timed out. 2018-12-08 07:02:03,620 INFO org.apache.flink.runtime.resourcemanager.StandaloneResourceManager - Closing TaskExecutor connection 8949a9d85ec65b207654b28bf60ba10c
because: The heartbeat of TaskManager with id 8949a9d85ec65b207654b28bf60ba10c timed out.
2018-12-08 07:06:22,248 INFO org.apache.flink.runtime.resourcemanager.StandaloneResourceManager - Registering TaskManager with ResourceID 79676d2993f4c61c8a17e37555d50474
(akka.tcp://flink@flink-taskmanager-6cf55db87b-5x9sd:6122/user/taskmanager_0) at ResourceManager
2018-12-08 07:07:28,960 WARN akka.remote.ReliableDeliverySupervisor - Association with remote system [akka.tcp://flink@flink-taskmanager-6cf55db87b-5x9sd:6122] has
failed, address is now gated for [50] ms. Reason: [Disassociated] 2018-12-08 07:07:31,900 INFO org.apache.flink.runtime.resourcemanager.StandaloneResourceManager - Registering TaskManager with ResourceID 231a4d51d784b397cc4c5f49559b92bf
(akka.tcp://flink@flink-taskmanager-6cf55db87b-5x9sd:6122/user/taskmanager_0) at ResourceManager
2018-12-08 07:08:13,639 INFO org.apache.flink.runtime.resourcemanager.StandaloneResourceManager - The heartbeat of TaskManager with id 79676d2993f4c61c8a17e37555d50474
timed out. 2018-12-08 07:08:13,639 INFO org.apache.flink.runtime.resourcemanager.StandaloneResourceManager - Closing TaskExecutor connection 79676d2993f4c61c8a17e37555d50474
because: The heartbeat of TaskManager with id 79676d2993f4c61c8a17e37555d50474 timed out.
2018-12-08 07:08:38,968 WARN akka.remote.ReliableDeliverySupervisor - Association with remote system [akka.tcp://flink@flink-taskmanager-6cf55db87b-5x9sd:6122] has
failed, address is now gated for [50] ms. Reason: [Disassociated] 2018-12-08 07:08:43,611 WARN akka.remote.ReliableDeliverySupervisor - Association with remote system [akka.tcp://flink@flink-taskmanager-6cf55db87b-5x9sd:6122] has
failed, address is now gated for [50] ms. Reason: [Association failed with [akka.tcp://flink@flink-taskmanager-6cf55db87b-5x9sd:6122]] Caused by: [flink-taskmanager-6cf55db87b-5x9sd: Name or service not known]
2018-12-08 07:08:53,601 WARN akka.remote.ReliableDeliverySupervisor - Association with remote system [akka.tcp://flink@flink-taskmanager-6cf55db87b-5x9sd:6122] has
failed, address is now gated for [50] ms. Reason: [Association failed with [akka.tcp://flink@flink-taskmanager-6cf55db87b-5x9sd:6122]] Caused by: [flink-taskmanager-6cf55db87b-5x9sd]
2018-12-08 07:09:03,604 WARN akka.remote.ReliableDeliverySupervisor - Association with remote system [akka.tcp://flink@flink-taskmanager-6cf55db87b-5x9sd:6122] has
failed, address is now gated for [50] ms. Reason: [Association failed with [akka.tcp://flink@flink-taskmanager-6cf55db87b-5x9sd:6122]] Caused by: [flink-taskmanager-6cf55db87b-5x9sd: Name or service not known]
2018-12-08 07:09:13,601 WARN akka.remote.ReliableDeliverySupervisor - Association with remote system [akka.tcp://flink@flink-taskmanager-6cf55db87b-5x9sd:6122] has
failed, address is now gated for [50] ms. Reason: [Association failed with [akka.tcp://flink@flink-taskmanager-6cf55db87b-5x9sd:6122]] Caused by: [flink-taskmanager-6cf55db87b-5x9sd]
2018-12-08 07:09:23,605 WARN akka.remote.ReliableDeliverySupervisor - Association with remote system [akka.tcp://flink@flink-taskmanager-6cf55db87b-5x9sd:6122] has
failed, address is now gated for [50] ms. Reason: [Association failed with [akka.tcp://flink@flink-taskmanager-6cf55db87b-5x9sd:6122]] Caused by: [flink-taskmanager-6cf55db87b-5x9sd: Name or service not known]
2018-12-08 07:09:23,629 INFO org.apache.flink.runtime.resourcemanager.StandaloneResourceManager - The heartbeat of TaskManager with id 231a4d51d784b397cc4c5f49559b92bf
timed out. Regards, Abhi Thakur |
Re: Flink issue while setting up in Kubernetes
|
|
Hi Abhi Thakur, We need more information to help you. What docker images are you using? Can you share the kubernetes resource definitions? Can you share the complete logs of the JM and TMs? Did you follow the steps outlined in the Flink documentation [1]? Best, Gary [1] https://ci.apache.org/projects/flink/flink-docs-master/ops/deployment/kubernetes.html On Mon, Dec 10, 2018 at 7:29 AM Thakur, Abhi <[hidden email]> wrote:
|
| Free forum by Nabble | Edit this page |