(DEPRECATED) Apache Flink User Mailing List archive.

Flink & Mesos don't launch Job and Task managers

classic Classic list List threaded Threaded
♦
♦
 2 messages Options
Felipe Gutierrez
Reply | Threaded
Open this post in threaded view
♦
♦
|

Flink & Mesos don't launch Job and Task managers

Felipe Gutierrez
Hi,

I am trying to run Flink with Mesos but without DC/OS [1]. My mesos instance is working and when I start flink by the script mesos-appmaster.sh, Flink starts, however without any Job Manager or Task Manager. What could be the problem?

I receive no errors on the log file, only some warnings:
2019-09-05 17:15:32,087 WARN  org.apache.flink.configuration.Configuration                  - Config uses deprecated configuration key 'taskmanager.tmp.dirs' instead of proper key 'io.tmp.dirs'
2019-09-05 17:15:32,439 WARN  org.apache.flink.runtime.clusterframework.overlays.HadoopConfOverlay  - Unable to locate a Hadoop configuration; HDFS will use defaults.
2019-09-05 17:15:32,483 WARN  org.apache.flink.configuration.Configuration                  - Config uses deprecated configuration key 'jobmanager.web.port' instead of proper key 'rest.bind-port'
2019-09-05 17:15:32,899 INFO  org.apache.flink.mesos.runtime.clusterframework.MesosResourceManager  - Registering as new framework.
2019-09-05 17:15:32,899 INFO  org.apache.flink.mesos.runtime.clusterframework.MesosResourceManager  - --------------------------------------------------------------------------------
2019-09-05 17:15:32,899 INFO  org.apache.flink.mesos.runtime.clusterframework.MesosResourceManager  -  Mesos Info:
2019-09-05 17:15:32,899 INFO  org.apache.flink.mesos.runtime.clusterframework.MesosResourceManager  -     Master URL: XXX.XXX.XXX.XXX:5050
2019-09-05 17:15:32,899 INFO  org.apache.flink.mesos.runtime.clusterframework.MesosResourceManager  -  Framework Info:
2019-09-05 17:15:32,899 INFO  org.apache.flink.mesos.runtime.clusterframework.MesosResourceManager  -     ID: (none)
2019-09-05 17:15:32,899 INFO  org.apache.flink.mesos.runtime.clusterframework.MesosResourceManager  -     Name: Flink
2019-09-05 17:15:32,899 INFO  org.apache.flink.mesos.runtime.clusterframework.MesosResourceManager  -     Failover Timeout (secs): 604800.0
2019-09-05 17:15:32,900 INFO  org.apache.flink.mesos.runtime.clusterframework.MesosResourceManager  -     Role: *
2019-09-05 17:15:32,900 INFO  org.apache.flink.mesos.runtime.clusterframework.MesosResourceManager  -     Capabilities: (none)
2019-09-05 17:15:32,900 INFO  org.apache.flink.mesos.runtime.clusterframework.MesosResourceManager  -     Principal: (none)
2019-09-05 17:15:32,900 INFO  org.apache.flink.mesos.runtime.clusterframework.MesosResourceManager  -     Host: XXX.XXX.XXX.XXX
2019-09-05 17:15:32,900 INFO  org.apache.flink.mesos.runtime.clusterframework.MesosResourceManager  -     Web UI: http://XXX.XXX.XXX.XXX:8081
2019-09-05 17:15:32,900 INFO  org.apache.flink.mesos.runtime.clusterframework.MesosResourceManager  - --------------------------------------------------------------------------------


Here is my flink-conf.yaml file

mesos.master: XXX.XXX.XXX.XXX:5050
mesos.initial-tasks: 10
mesos.resourcemanager.tasks.container.type: mesos
jobmanager.heap.mb: 1024
jobmanager.rpc.port: 6123
jobmanager.web.port: 8081
mesos.resourcemanager.tasks.mem: 4096
taskmanager.heap.mb: 3500



Thanks,
Felipe

--
-- Felipe Gutierrez
-- skype: felipe.o.gutierrez
Felipe Gutierrez
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Flink & Mesos don't launch Job and Task managers

Felipe Gutierrez
my bad. Flink allocates task managers dynamically.
--
-- Felipe Gutierrez
-- skype: felipe.o.gutierrez


On Thu, Sep 5, 2019 at 5:24 PM Felipe Gutierrez <[hidden email]> wrote:
Hi,

I am trying to run Flink with Mesos but without DC/OS [1]. My mesos instance is working and when I start flink by the script mesos-appmaster.sh, Flink starts, however without any Job Manager or Task Manager. What could be the problem?

I receive no errors on the log file, only some warnings:
2019-09-05 17:15:32,087 WARN  org.apache.flink.configuration.Configuration                  - Config uses deprecated configuration key 'taskmanager.tmp.dirs' instead of proper key 'io.tmp.dirs'
2019-09-05 17:15:32,439 WARN  org.apache.flink.runtime.clusterframework.overlays.HadoopConfOverlay  - Unable to locate a Hadoop configuration; HDFS will use defaults.
2019-09-05 17:15:32,483 WARN  org.apache.flink.configuration.Configuration                  - Config uses deprecated configuration key 'jobmanager.web.port' instead of proper key 'rest.bind-port'
2019-09-05 17:15:32,899 INFO  org.apache.flink.mesos.runtime.clusterframework.MesosResourceManager  - Registering as new framework.
2019-09-05 17:15:32,899 INFO  org.apache.flink.mesos.runtime.clusterframework.MesosResourceManager  - --------------------------------------------------------------------------------
2019-09-05 17:15:32,899 INFO  org.apache.flink.mesos.runtime.clusterframework.MesosResourceManager  -  Mesos Info:
2019-09-05 17:15:32,899 INFO  org.apache.flink.mesos.runtime.clusterframework.MesosResourceManager  -     Master URL: XXX.XXX.XXX.XXX:5050
2019-09-05 17:15:32,899 INFO  org.apache.flink.mesos.runtime.clusterframework.MesosResourceManager  -  Framework Info:
2019-09-05 17:15:32,899 INFO  org.apache.flink.mesos.runtime.clusterframework.MesosResourceManager  -     ID: (none)
2019-09-05 17:15:32,899 INFO  org.apache.flink.mesos.runtime.clusterframework.MesosResourceManager  -     Name: Flink
2019-09-05 17:15:32,899 INFO  org.apache.flink.mesos.runtime.clusterframework.MesosResourceManager  -     Failover Timeout (secs): 604800.0
2019-09-05 17:15:32,900 INFO  org.apache.flink.mesos.runtime.clusterframework.MesosResourceManager  -     Role: *
2019-09-05 17:15:32,900 INFO  org.apache.flink.mesos.runtime.clusterframework.MesosResourceManager  -     Capabilities: (none)
2019-09-05 17:15:32,900 INFO  org.apache.flink.mesos.runtime.clusterframework.MesosResourceManager  -     Principal: (none)
2019-09-05 17:15:32,900 INFO  org.apache.flink.mesos.runtime.clusterframework.MesosResourceManager  -     Host: XXX.XXX.XXX.XXX
2019-09-05 17:15:32,900 INFO  org.apache.flink.mesos.runtime.clusterframework.MesosResourceManager  -     Web UI: http://XXX.XXX.XXX.XXX:8081
2019-09-05 17:15:32,900 INFO  org.apache.flink.mesos.runtime.clusterframework.MesosResourceManager  - --------------------------------------------------------------------------------


Here is my flink-conf.yaml file

mesos.master: XXX.XXX.XXX.XXX:5050
mesos.initial-tasks: 10
mesos.resourcemanager.tasks.container.type: mesos
jobmanager.heap.mb: 1024
jobmanager.rpc.port: 6123
jobmanager.web.port: 8081
mesos.resourcemanager.tasks.mem: 4096
taskmanager.heap.mb: 3500



Thanks,
Felipe

--
-- Felipe Gutierrez
-- skype: felipe.o.gutierrez
Free forum by Nabble Edit this page