Thanks Vikash for the response. Yes thats very much feasible but we are planning to move to job/application cluster model where in the artifacts are bundled inside the container. When there is a new container image then we might have to do the following.
- Take a savepoint
- Upgrade the JM and TM container images and provide the save point path during start up
I would like to know if this is the standard way or if there are some better options. We currently use terraform for managing the infrastructure and it would be greatly helpful if someone has already done this.
Thanks
Could you not use the JM web address to utilize the rest api? You can start/stop/save point/restore + upload new jars via the rest api. While I did not run on ECS( ran on EMR) I was able to use the rest api to do deployments.
On Sun, Jan 3, 2021 at 19:09 Navneeth Krishnan <
[hidden email]> wrote:
Hi All,
Currently we are using flink in session cluster mode and we manually deploy the jobs i.e. through the web UI. We use AWS ECS for running the docker container with 2 services definitions, one for JM and other for TM. How is everyone managing the CICD process? Is there a better way to run a job in job cluster mode and use jenkins to perform CICD?
Any pointers on how this is being done would really help and greatly appreciated.
Thanks,
Navneeth
Locked
