Hi guys,
I want to use some client lib that integrated with akka system
Flink aslo already created akka system job manager and those task manager can use akka by its akka.tcp addess
My architecture stucture currently is:
EMR (flink 1.8)
1 core node
1 master node
Flink job on Yarn
I have on task manager and one job manager:
On my task manager I got this log:
container_1560142756526_0001_01_000002/taskmanager.log:2019-06-10 06:32:52,171 INFO akka.remote.Remoting - Remoting started; listening on addresses :[akka.tcp://
[hidden email]:46267]
container_1560142756526_0001_01_000002/taskmanager.log:2019-06-10 06:32:52,183 INFO org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils - Actor system started at akka.tcp://
[hidden email]:46267
container_1560142756526_0001_01_000002/taskmanager.log:2019-06-10 06:32:52,284 INFO org.apache.flink.runtime.taskexecutor.TaskManagerRunner - Actor system started at akka.tcp://
[hidden email]:36753
container_1560142756526_0001_01_000002/taskmanager.log:2019-06-10 06:32:52,731 INFO org.apache.flink.runtime.rpc.akka.AkkaRpcService - Starting RPC endpoint for org.apache.flink.runtime.taskexecutor.TaskExecutor at akka://flink/user/taskmanager_0 .
container_1560142756526_0001_01_000002/taskmanager.log:2019-06-10 06:32:52,755 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor - Connecting to ResourceManager akka.tcp://
[hidden email]:33547/user/resourcemanager(00000000000000000000000000000000).
container_1560142756526_0001_01_000002/taskmanager.log:2019-06-10 06:32:53,153 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor - Successful registration at resource manager akka.tcp://
[hidden email]:33547/user/resourcemanager under registration id 31415730214d58ecaf8396835ccce613.
On my job manager log:
container_1560142756526_0001_01_000001/jobmanager.out:2019-06-10 06:32:43,473 INFO org.apache.flink.yarn.YarnResourceManager - Registering job manager
[hidden email]://flink@ip-10-
0-5-203.ap-southeast-1.compute.internal:33547/user/jobmanager_0 for job 3111b27f8e4a873ad4301bbe2f7e7069.
container_1560142756526_0001_01_000001/jobmanager.out:2019-06-10 06:32:43,484 INFO org.apache.flink.yarn.YarnResourceManager - Registered job manager
[hidden email]://flink@ip-10-0
-5-203.ap-southeast-1.compute.internal:33547/user/jobmanager_0 for job 3111b27f8e4a873ad4301bbe2f7e7069.
I want to access to this actor system for my client lib (like akka http, redisscala…). Those need to declare akka.tcp address too.
implicit val akkaSystem = akka.actor.ActorSystem(“akka.tcp”)
How do I find those address dynamically so that I can use them in AWS emr?
Thanks
Locked
