Flink docker 1.11.3 actually runs 1.11.2

[Flavio Pompermaier]

Hi to all,

I've just build a docker that use the image flink:1.11.3-scala_2.12-java11 but the web UI (and logs too) display Flink 1.11.2 (Commit: fe36135). Was there an error with the release?

Best,

Flavio 

[Chesnay Schepler]

Please provide steps to reproduce the issue.

I can't see anything wrong in the dockerfiles (they reference the
correct release url), and the referenced release correctly identifies
itself as 1.11.3 .
I also started a container with the image, started a jobmanager, and the
logs show 1.11.3 like they are supposed to do.

On 4/13/2021 6:31 PM, Flavio Pompermaier wrote:
> Hi to all,
> I've just build a docker that use the image
> flink:1.11.3-scala_2.12-java11 but the web UI (and logs too) display
> Flink 1.11.2 (Commit: fe36135). Was there an error with the release?
>
> Best,
> Flavio

[Flavio Pompermaier]

Hi Chesnay, 

my tests were done using docker-compose (with the command 'docker-compose up --build -d flink-jobmanager flink-taskmanager').

These are the necessary files (./flink/db-libs/* contains the jdbc libraries I use while /opt/flink/data is used as a volume to share files with other dockers):

PS: before I've used FLINK_VERSION=1.11-scala_2.12-java11 in the .env file...so if you didn't encounter the problem it's probably caused by some docker-compose internal that is not overriding the old image/container (I'm indeed relatively new to docker and docker-compose..)

-----------------------
.env
-----------------------
FLINK_VERSION=1.11.3-scala_2.12-java11

----------------------------
docker-compose.yml
----------------------------

version: '3'
services:
  flink-jobmanager:
    container_name: flink-jobmanager
    build:
      context: .
      dockerfile: Dockerfile.flink
      args:
        - FLINK_VERSION=${FLINK_VERSION}
    image: 'flink-test:${FLINK_VERSION}'
    ports:
      - "8091:8081"
      - "8092:8082"
    command: jobmanager
    environment:
      - |
        FLINK_PROPERTIES=
        jobmanager.rpc.address: flink-jobmanager
        rest.port: 8081
        historyserver.web.port: 8082
        web.upload.dir: /opt/flink
    volumes:
      - '/opt/flink/data:/tmp/flink'
    networks:
      - test-network
  flink-taskmanager:
    container_name: flink-taskmanager
    build:
      context: .
      dockerfile: Dockerfile.flink
      args:
        - FLINK_VERSION=${FLINK_VERSION}
    image: 'flink-test:${FLINK_VERSION}'
    depends_on:
      - flink-jobmanager
    command: taskmanager
    environment:
      - |
        FLINK_PROPERTIES=
        jobmanager.rpc.address: flink-jobmanager
        taskmanager.memory.process.size: 1728m
        taskmanager.numberOfTaskSlots: 2
    volumes:
      - '/opt/flink/data:/tmp/flink'
    networks:
      - test-network

networks:
  test-network:
    driver: bridge

-----------------------
Dockerfile.flink
-----------------------

ARG FLINK_VERSION
FROM flink:$FLINK_VERSION

USER root
RUN set -ex; apt-get update; apt-get -y install openssh-client ssh # python
USER flink
WORKDIR /opt/flink

COPY flink/db-libs/*   lib/

Thanks for the support,

Flavio

On Tue, Apr 13, 2021 at 7:43 PM Chesnay Schepler <[hidden email]> wrote:

Please provide steps to reproduce the issue.

I can't see anything wrong in the dockerfiles (they reference the
correct release url), and the referenced release correctly identifies
itself as 1.11.3 .
I also started a container with the image, started a jobmanager, and the
logs show 1.11.3 like they are supposed to do.

On 4/13/2021 6:31 PM, Flavio Pompermaier wrote:
> Hi to all,
> I've just build a docker that use the image
> flink:1.11.3-scala_2.12-java11 but the web UI (and logs too) display
> Flink 1.11.2 (Commit: fe36135). Was there an error with the release?
>
> Best,
> Flavio

--

[Chesnay Schepler]

Works properly for me. I think your suspicion about the .env is correct; it is probably not considered when checking whether something has changed, so docker juse re-uses the previous image.

On 4/13/2021 9:51 PM, Flavio Pompermaier wrote:

Hi Chesnay, 

my tests were done using docker-compose (with the command 'docker-compose up --build -d flink-jobmanager flink-taskmanager').

These are the necessary files (./flink/db-libs/* contains the jdbc libraries I use while /opt/flink/data is used as a volume to share files with other dockers):

PS: before I've used FLINK_VERSION=1.11-scala_2.12-java11 in the .env file...so if you didn't encounter the problem it's probably caused by some docker-compose internal that is not overriding the old image/container (I'm indeed relatively new to docker and docker-compose..)

-----------------------
.env
-----------------------
FLINK_VERSION=1.11.3-scala_2.12-java11

----------------------------
docker-compose.yml
----------------------------

version: '3'
services:
  flink-jobmanager:
    container_name: flink-jobmanager
    build:
      context: .
      dockerfile: Dockerfile.flink
      args:
        - FLINK_VERSION=${FLINK_VERSION}
    image: 'flink-test:${FLINK_VERSION}'
    ports:
      - "8091:8081"
      - "8092:8082"
    command: jobmanager
    environment:
      - |
        FLINK_PROPERTIES=
        jobmanager.rpc.address: flink-jobmanager
        rest.port: 8081
        historyserver.web.port: 8082
        web.upload.dir: /opt/flink
    volumes:
      - '/opt/flink/<a class="moz-txt-link-freetext" href="data:/tmp/flink">data:/tmp/flink'
    networks:
      - test-network
  flink-taskmanager:
    container_name: flink-taskmanager
    build:
      context: .
      dockerfile: Dockerfile.flink
      args:
        - FLINK_VERSION=${FLINK_VERSION}
    image: 'flink-test:${FLINK_VERSION}'
    depends_on:
      - flink-jobmanager
    command: taskmanager
    environment:
      - |
        FLINK_PROPERTIES=
        jobmanager.rpc.address: flink-jobmanager
        taskmanager.memory.process.size: 1728m
        taskmanager.numberOfTaskSlots: 2
    volumes:
      - '/opt/flink/<a class="moz-txt-link-freetext" href="data:/tmp/flink">data:/tmp/flink'
    networks:
      - test-network

networks:
  test-network:
    driver: bridge

-----------------------
Dockerfile.flink
-----------------------

ARG FLINK_VERSION
FROM flink:$FLINK_VERSION

USER root
RUN set -ex; apt-get update; apt-get -y install openssh-client ssh # python
USER flink
WORKDIR /opt/flink

COPY flink/db-libs/*   lib/

Thanks for the support,

Flavio

On Tue, Apr 13, 2021 at 7:43 PM Chesnay Schepler <[hidden email]> wrote:

Please provide steps to reproduce the issue.

I can't see anything wrong in the dockerfiles (they reference the
correct release url), and the referenced release correctly identifies
itself as 1.11.3 .
I also started a container with the image, started a jobmanager, and the
logs show 1.11.3 like they are supposed to do.

On 4/13/2021 6:31 PM, Flavio Pompermaier wrote:
> Hi to all,
> I've just build a docker that use the image
> flink:1.11.3-scala_2.12-java11 but the web UI (and logs too) display
> Flink 1.11.2 (Commit: fe36135). Was there an error with the release?
>
> Best,
> Flavio

--