jobmanager rpc inside kubernetes

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

jobmanager rpc inside kubernetes

Chris Latko
when trying to submit a job to my k8s cluster using `flink run -m
localhost:6123 app.jar` i get the following error in jobmanager log:

-----
dropping message [class akka.actor.ActorSelectionMessage] for
non-local recipient [Actor[akka.tcp://flink@localhost:6123/]] arriving
at [akka.tcp://flink@localhost:6123] inbound addresses are
[akka.tcp://[hidden email]:6123]
-----

there are several workarounds but i'm absolutely not going to bundle
my jar into a taskmanager pod just so i can deploy. i would rather
have the jar on ceph and deploy from tm or jm:
- kubectl exec -ti pod-replica-version bash
- flink run blah blah blah...

i could also run akka in unsafe mode, what are the majority of people
doing regarding this issue?

i have an up to date helm chart that is nearing production readiness
that i'm willing to share if people are interested (and my company
lets me).
Reply | Threaded
Open this post in threaded view
|

Re: jobmanager rpc inside kubernetes

Sampath Bhat
It would be helpful if you provide the complete CLI logs. Because even I'm using flink run command to submit jobs to flink jobmanager running on K8s and its working fine. For remote execution using flink CLI you should provide flink-conf.yaml file which contains job manager address, port and SSL/HA related config if the same is enabled in your flink cluster to which the job had to be submitted.

On Fri, Apr 27, 2018 at 7:24 AM, Chris Latko <[hidden email]> wrote:
when trying to submit a job to my k8s cluster using `flink run -m
localhost:6123 app.jar` i get the following error in jobmanager log:

-----
dropping message [class akka.actor.ActorSelectionMessage] for
non-local recipient [Actor[akka.tcp://flink@localhost:6123/]] arriving
at [akka.tcp://flink@localhost:6123] inbound addresses are
[akka.tcp://flink@sis-flink-cluster-jobmanager.platform.svc.k8s.local:6123]
-----

there are several workarounds but i'm absolutely not going to bundle
my jar into a taskmanager pod just so i can deploy. i would rather
have the jar on ceph and deploy from tm or jm:
- kubectl exec -ti pod-replica-version bash
- flink run blah blah blah...

i could also run akka in unsafe mode, what are the majority of people
doing regarding this issue?

i have an up to date helm chart that is nearing production readiness
that i'm willing to share if people are interested (and my company
lets me).