Flink configuration on Docker deployment

[Soheil Pourbafrani]

Hi,

I need to set up a Flink cluster using the docker(and not using the docker-compose). I successfully could strat the jobmanager and taskmanager but the problem is I have no idea how to change the default configuration for them. For example in the case of giving 8 slots to the taskmanager or change the memory size of both jobmanager and taskmanager.

It will be appreciated if somebody tells me how to change the Flink parameters on docker

Thanks

[Yang Wang]

Hi Soheil,

Since you are not using any container orchestration framework(e.g. docker-compose, Kubernetes,

mesos), so you need to manually update the flink-conf.yaml in your docker images. Usually, it is

located in the path "/opt/flink/conf". 

Docker volume also could be used to override the flink configuration when you start the jobmanager

and taskmanager containers[1].

Best,

Yang

[1]. https://docs.docker.com/storage/volumes/

Soheil Pourbafrani <[hidden email]> 于2020年1月21日周二 下午7:46写道：

Hi,

I need to set up a Flink cluster using the docker(and not using the docker-compose). I successfully could strat the jobmanager and taskmanager but the problem is I have no idea how to change the default configuration for them. For example in the case of giving 8 slots to the taskmanager or change the memory size of both jobmanager and taskmanager.

It will be appreciated if somebody tells me how to change the Flink parameters on docker

Thanks

[Soheil Pourbafrani]

Thanks a lot! 

On Wed, Jan 22, 2020 at 3:58 AM Yang Wang <[hidden email]> wrote:

Hi Soheil,

Since you are not using any container orchestration framework(e.g. docker-compose, Kubernetes,

mesos), so you need to manually update the flink-conf.yaml in your docker images. Usually, it is

located in the path "/opt/flink/conf". 

Docker volume also could be used to override the flink configuration when you start the jobmanager

and taskmanager containers[1].

Best,

Yang

[1]. https://docs.docker.com/storage/volumes/

Soheil Pourbafrani <[hidden email]> 于2020年1月21日周二 下午7:46写道：

Hi,

I need to set up a Flink cluster using the docker(and not using the docker-compose). I successfully could strat the jobmanager and taskmanager but the problem is I have no idea how to change the default configuration for them. For example in the case of giving 8 slots to the taskmanager or change the memory size of both jobmanager and taskmanager.

It will be appreciated if somebody tells me how to change the Flink parameters on docker

Thanks