docker, error NoResourceAvailableException..

[shyla deshpande]

Hello all,

Trying to use docker as a single node flink cluster.

docker run --name flink_local -p 8081:8081 -t flink local

I submited a job to the cluster using the Web UI. The job failed. I see this error message in the docker logs.

org.apache.flink.runtime.jobmanager.scheduler.NoResourceAvailableException: Could not allocate all requires slots within timeout of 300000 ms. Slots required: 2, slots allocated: 0

The Web UI, shows 0 taskmanagers and 0 task slots on the Flink dashboard.

How do I start the docker with 2 Task slots?

Appreciate any help.

Thanks

[Dominik Wosiński]

Hey,
The problem is that your command does start Job Manager container, but it does not start the Task Manager . That is why you have 0 slots. Currently, the default numberOfTaskSlots is set to the number of CPUs avaialbe on the machine.

So, You generally can to do 2 things:

1) Start Job Manager and 2 Task Managers. If you have Docker Compose available, you can paste this to your docker-compose.yml :

services:
  jobmanager:
    image: ${FLINK_DOCKER_IMAGE_NAME:-flink}
    expose:
      - "6123"
    ports:
      - "8081:8081"
    command: jobmanager
    environment:
      - JOB_MANAGER_RPC_ADDRESS=jobmanager

  taskmanager:
    image: ${FLINK_DOCKER_IMAGE_NAME:-flink}
    expose:
      - "6121"
      - "6122"
    depends_on:
      - jobmanager
    command: taskmanager
    links:
      - "jobmanager:jobmanager"
    environment:
      - JOB_MANAGER_RPC_ADDRESS=jobmanager

  taskmanager1:
    image: ${FLINK_DOCKER_IMAGE_NAME:-flink}
    expose:
      - "6190"
      - "6120"
    depends_on:
      - jobmanager
    command: taskmanager
    links:
      - "jobmanager:jobmanager"
    environment:
      - JOB_MANAGER_RPC_ADDRESS=jobmanager

 

This will give you 1 Job Manager and 2 Task Managers with one task slot each, so 2 Task slots in general.

2) You can deploy 1 Job Manager and 1 Task Manager.Then you need to modify flink-conf.yml and set the following setting :

taskmanager.numberOfTaskSlots: 2

This will give you 2 Task Slots with only 1 Task Manager. But you will need to somehow override config in the container, possibly using : https://docs.docker.com/storage/volumes/

Regards,
Dominik.

Od: [hidden email]
Wysłano: środa, 15 sierpnia 2018 07:23
Do: [hidden email]
Temat: docker, error NoResourceAvailableException..

 

Hello all,

 

Trying to use docker as a single node flink cluster.

 

docker run --name flink_local -p 8081:8081 -t flink local

 

I submited a job to the cluster using the Web UI. The job failed. I see this error message in the docker logs.

 

org.apache.flink.runtime.jobmanager.scheduler.NoResourceAvailableException: Could not allocate all requires slots within timeout of 300000 ms. Slots required: 2, slots allocated: 0

 

The Web UI, shows 0 taskmanagers and 0 task slots on the Flink dashboard.

How do I start the docker with 2 Task slots?

 

Appreciate any help.

 

Thanks

 

[shyla deshpande]

Thanks Dominik, I will try that.

On Wed, Aug 15, 2018 at 3:10 AM, Dominik Wosiński <[hidden email]> wrote:

Hey,
The problem is that your command does start Job Manager container, but it does not start the Task Manager . That is why you have 0 slots. Currently, the default numberOfTaskSlots is set to the number of CPUs avaialbe on the machine.

So, You generally can to do 2 things:

1) Start Job Manager and 2 Task Managers. If you have Docker Compose available, you can paste this to your docker-compose.yml :

services:
  jobmanager:
    image: ${FLINK_DOCKER_IMAGE_NAME:-flink}
    expose:
      - "6123"
    ports:
      - "8081:8081"
    command: jobmanager
    environment:
      - JOB_MANAGER_RPC_ADDRESS=jobmanager

  taskmanager:
    image: ${FLINK_DOCKER_IMAGE_NAME:-flink}
    expose:
      - "6121"
      - "6122"
    depends_on:
      - jobmanager
    command: taskmanager
    links:
      - "jobmanager:jobmanager"
    environment:
      - JOB_MANAGER_RPC_ADDRESS=jobmanager

  taskmanager1:
    image: ${FLINK_DOCKER_IMAGE_NAME:-flink}
    expose:
      - "6190"
      - "6120"
    depends_on:
      - jobmanager
    command: taskmanager
    links:
      - "jobmanager:jobmanager"
    environment:
      - JOB_MANAGER_RPC_ADDRESS=jobmanager

 

This will give you 1 Job Manager and 2 Task Managers with one task slot each, so 2 Task slots in general.

2) You can deploy 1 Job Manager and 1 Task Manager.Then you need to modify flink-conf.yml and set the following setting :

taskmanager.numberOfTaskSlots: 2

This will give you 2 Task Slots with only 1 Task Manager. But you will need to somehow override config in the container, possibly using : https://docs.docker.com/storage/volumes/

Regards,
Dominik.

Od: [hidden email]
Wysłano: środa, 15 sierpnia 2018 07:23
Do: [hidden email]
Temat: docker, error NoResourceAvailableException..

 

Hello all,

 

Trying to use docker as a single node flink cluster.

 

docker run --name flink_local -p 8081:8081 -t flink local

 

I submited a job to the cluster using the Web UI. The job failed. I see this error message in the docker logs.

 

org.apache.flink.runtime.jobmanager.scheduler.NoResourceAvailableException: Could not allocate all requires slots within timeout of 300000 ms. Slots required: 2, slots allocated: 0

 

The Web UI, shows 0 taskmanagers and 0 task slots on the Flink dashboard.

How do I start the docker with 2 Task slots?

 

Appreciate any help.

 

Thanks

 

[Esteban Serrano]

You can also instead of defining 2 services (taskmanager and taskmanager1), set the scale parameter on taskmanager to the number of desired slots.
Something like this:

taskmanager:
  image: "${FLINK_DOCKER_IMAGE:-flink:1.5.2}"
  scale: 2
  expose:
    - "6121"
    - "6122"
    - "8081"

On Wed, Aug 15, 2018 at 1:53 PM shyla deshpande <[hidden email]> wrote:

Thanks Dominik, I will try that.

On Wed, Aug 15, 2018 at 3:10 AM, Dominik Wosiński <[hidden email]> wrote:

Hey,
The problem is that your command does start Job Manager container, but it does not start the Task Manager . That is why you have 0 slots. Currently, the default numberOfTaskSlots is set to the number of CPUs avaialbe on the machine.

So, You generally can to do 2 things:

1) Start Job Manager and 2 Task Managers. If you have Docker Compose available, you can paste this to your docker-compose.yml :

services:
  jobmanager:
    image: ${FLINK_DOCKER_IMAGE_NAME:-flink}
    expose:
      - "6123"
    ports:
      - "8081:8081"
    command: jobmanager
    environment:
      - JOB_MANAGER_RPC_ADDRESS=jobmanager

  taskmanager:
    image: ${FLINK_DOCKER_IMAGE_NAME:-flink}
    expose:
      - "6121"
      - "6122"
    depends_on:
      - jobmanager
    command: taskmanager
    links:
      - "jobmanager:jobmanager"
    environment:
      - JOB_MANAGER_RPC_ADDRESS=jobmanager

  taskmanager1:
    image: ${FLINK_DOCKER_IMAGE_NAME:-flink}
    expose:
      - "6190"
      - "6120"
    depends_on:
      - jobmanager
    command: taskmanager
    links:
      - "jobmanager:jobmanager"
    environment:
      - JOB_MANAGER_RPC_ADDRESS=jobmanager

 

This will give you 1 Job Manager and 2 Task Managers with one task slot each, so 2 Task slots in general.

2) You can deploy 1 Job Manager and 1 Task Manager.Then you need to modify flink-conf.yml and set the following setting :

taskmanager.numberOfTaskSlots: 2

This will give you 2 Task Slots with only 1 Task Manager. But you will need to somehow override config in the container, possibly using : https://docs.docker.com/storage/volumes/

Regards,
Dominik.

Od: [hidden email]
Wysłano: środa, 15 sierpnia 2018 07:23
Do: [hidden email]
Temat: docker, error NoResourceAvailableException..

 

Hello all,

 

Trying to use docker as a single node flink cluster.

 

docker run --name flink_local -p 8081:8081 -t flink local

 

I submited a job to the cluster using the Web UI. The job failed. I see this error message in the docker logs.

 

org.apache.flink.runtime.jobmanager.scheduler.NoResourceAvailableException: Could not allocate all requires slots within timeout of 300000 ms. Slots required: 2, slots allocated: 0

 

The Web UI, shows 0 taskmanagers and 0 task slots on the Flink dashboard.

How do I start the docker with 2 Task slots?

 

Appreciate any help.

 

Thanks