(DEPRECATED) Apache Flink User Mailing List archive.

Flink with Docker: docker-compose and FLINK_JOB_ARGUMENT exception

classic Classic list List threaded Threaded
♦
♦
 4 messages Options
Marke Builder
Reply | Threaded
Open this post in threaded view
♦
♦
|

Flink with Docker: docker-compose and FLINK_JOB_ARGUMENT exception

Marke Builder
Hi,

I'm trying to run flink with docker (docker-compose) and job arguments "config-dev.properties". But it seams that the job arguments are not available:

docker-compose.yml

version: '2'
services:
  job-cluster:
    image: ${FLINK_DOCKER_IMAGE_NAME:-timeseries-v1}
    ports:
      - '8081:8081'
    command: job-cluster --job-classname -Djobmanager.rpc.address=job-cluster -Dparallelism.default=1 --config config-dev.properties

  taskmanager:
    image: ${FLINK_DOCKER_IMAGE_NAME:-timeseries-v1}
    command: task-manager -Djobmanager.rpc.address=job-cluster
    scale: 1


Excpetion:

4:32 AMorg.apache.flink.runtime.entrypoint.FlinkParseException: Failed to parse the command line arguments.

12/7/2018 10:44:32 AM          at org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:52)

12/7/2018 10:44:32 AM          at org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.main(StandaloneJobClusterEntryPoint.java:143)

12/7/2018 10:44:32 AMCaused by: org.apache.commons.cli.MissingArgumentException: Missing argument for option: j

12/7/2018 10:44:32 AM          at org.apache.commons.cli.DefaultParser.checkRequiredArgs(DefaultParser.java:211)

12/7/2018 10:44:32 AM          at org.apache.commons.cli.DefaultParser.handleOption(DefaultParser.java:599)

12/7/2018 10:44:32 AM          at org.apache.commons.cli.DefaultParser.handleShortAndLongOption(DefaultParser.java:548)

12/7/2018 10:44:32 AM          at org.apache.commons.cli.DefaultParser.handleToken(DefaultParser.java:243)

12/7/2018 10:44:32 AM          at org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:120)

12/7/2018 10:44:32 AM          at org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:81)

12/7/2018 10:44:32 AM          at org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:50)

12/7/2018 10:44:32 AM          ... 1 more

12/7/2018 10:44:32 AMException in thread "main" java.lang.NoSuchMethodError: org.apache.flink.runtime.entrypoint.parser.CommandLineParser.printHelp()V

12/7/2018 10:44:32 AM          at org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.main(StandaloneJobClusterEntryPoint.java:146)
Piotr Nowojski
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Flink with Docker: docker-compose and FLINK_JOB_ARGUMENT exception

Piotr Nowojski
Hi,

I have never used flink and docker together, so I’m not sure if I will be able to help, however have you seen this README:
?
Shouldn’t you be passing your arguments via `FLINK_JOB_ARGUMENTS` environment variable? 

Piotrek

On 7 Dec 2018, at 10:55, Marke Builder <[hidden email]> wrote:

Hi,

I'm trying to run flink with docker (docker-compose) and job arguments "config-dev.properties". But it seams that the job arguments are not available:

docker-compose.yml

version: '2'
services:
  job-cluster:
    image: ${FLINK_DOCKER_IMAGE_NAME:-timeseries-v1}
    ports:
      - '8081:8081'
    command: job-cluster --job-classname -Djobmanager.rpc.address=job-cluster -Dparallelism.default=1 --config config-dev.properties

  taskmanager:
    image: ${FLINK_DOCKER_IMAGE_NAME:-timeseries-v1}
    command: task-manager -Djobmanager.rpc.address=job-cluster
    scale: 1


Excpetion:
4:32 AMorg.apache.flink.runtime.entrypoint.FlinkParseException: Failed to parse the command line arguments.
12/7/2018 10:44:32 AM          at org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:52)
12/7/2018 10:44:32 AM          at org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.main(StandaloneJobClusterEntryPoint.java:143)
12/7/2018 10:44:32 AMCaused by: org.apache.commons.cli.MissingArgumentException: Missing argument for option: j
12/7/2018 10:44:32 AM          at org.apache.commons.cli.DefaultParser.checkRequiredArgs(DefaultParser.java:211)
12/7/2018 10:44:32 AM          at org.apache.commons.cli.DefaultParser.handleOption(DefaultParser.java:599)
12/7/2018 10:44:32 AM          at org.apache.commons.cli.DefaultParser.handleShortAndLongOption(DefaultParser.java:548)
12/7/2018 10:44:32 AM          at org.apache.commons.cli.DefaultParser.handleToken(DefaultParser.java:243)
12/7/2018 10:44:32 AM          at org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:120)
12/7/2018 10:44:32 AM          at org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:81)
12/7/2018 10:44:32 AM          at org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:50)
12/7/2018 10:44:32 AM          ... 1 more
12/7/2018 10:44:32 AMException in thread "main" java.lang.NoSuchMethodError: org.apache.flink.runtime.entrypoint.parser.CommandLineParser.printHelp()V
12/7/2018 10:44:32 AM          at org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.main(StandaloneJobClusterEntryPoint.java:146)
Jeff Zhang
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Flink with Docker: docker-compose and FLINK_JOB_ARGUMENT exception

Jeff Zhang
I didn't use the built-in docker of flink, but the following flink docker works for me pretty well.




Piotr Nowojski <[hidden email]> 于2018年12月7日周五 下午6:20写道：
Hi,

I have never used flink and docker together, so I’m not sure if I will be able to help, however have you seen this README:
?
Shouldn’t you be passing your arguments via `FLINK_JOB_ARGUMENTS` environment variable? 

Piotrek

On 7 Dec 2018, at 10:55, Marke Builder <[hidden email]> wrote:

Hi,

I'm trying to run flink with docker (docker-compose) and job arguments "config-dev.properties". But it seams that the job arguments are not available:

docker-compose.yml

version: '2'
services:
  job-cluster:
    image: ${FLINK_DOCKER_IMAGE_NAME:-timeseries-v1}
    ports:
      - '8081:8081'
    command: job-cluster --job-classname -Djobmanager.rpc.address=job-cluster -Dparallelism.default=1 --config config-dev.properties

  taskmanager:
    image: ${FLINK_DOCKER_IMAGE_NAME:-timeseries-v1}
    command: task-manager -Djobmanager.rpc.address=job-cluster
    scale: 1


Excpetion:
4:32 AMorg.apache.flink.runtime.entrypoint.FlinkParseException: Failed to parse the command line arguments.
12/7/2018 10:44:32 AM          at org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:52)
12/7/2018 10:44:32 AM          at org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.main(StandaloneJobClusterEntryPoint.java:143)
12/7/2018 10:44:32 AMCaused by: org.apache.commons.cli.MissingArgumentException: Missing argument for option: j
12/7/2018 10:44:32 AM          at org.apache.commons.cli.DefaultParser.checkRequiredArgs(DefaultParser.java:211)
12/7/2018 10:44:32 AM          at org.apache.commons.cli.DefaultParser.handleOption(DefaultParser.java:599)
12/7/2018 10:44:32 AM          at org.apache.commons.cli.DefaultParser.handleShortAndLongOption(DefaultParser.java:548)
12/7/2018 10:44:32 AM          at org.apache.commons.cli.DefaultParser.handleToken(DefaultParser.java:243)
12/7/2018 10:44:32 AM          at org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:120)
12/7/2018 10:44:32 AM          at org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:81)
12/7/2018 10:44:32 AM          at org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:50)
12/7/2018 10:44:32 AM          ... 1 more
12/7/2018 10:44:32 AMException in thread "main" java.lang.NoSuchMethodError: org.apache.flink.runtime.entrypoint.parser.CommandLineParser.printHelp()V
12/7/2018 10:44:32 AM          at org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.main(StandaloneJobClusterEntryPoint.java:146)



--
Best Regards

Jeff Zhang
Spico Florin
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Flink with Docker: docker-compose and FLINK_JOB_ARGUMENT exception

Spico Florin
In reply to this post by Piotr Nowojski
Hello!

I have used with success Flink with Docker, the approach presented in this article:


It it using a older version of Flink (1.4.2) in order to be compatible with Zeppelin client, but you can check if it works with new versions.

I hope it helps.
Regards,

Florin



On Fri, Dec 7, 2018 at 12:20 PM Piotr Nowojski <[hidden email]> wrote:
Hi,

I have never used flink and docker together, so I’m not sure if I will be able to help, however have you seen this README:
?
Shouldn’t you be passing your arguments via `FLINK_JOB_ARGUMENTS` environment variable? 

Piotrek

On 7 Dec 2018, at 10:55, Marke Builder <[hidden email]> wrote:

Hi,

I'm trying to run flink with docker (docker-compose) and job arguments "config-dev.properties". But it seams that the job arguments are not available:

docker-compose.yml

version: '2'
services:
  job-cluster:
    image: ${FLINK_DOCKER_IMAGE_NAME:-timeseries-v1}
    ports:
      - '8081:8081'
    command: job-cluster --job-classname -Djobmanager.rpc.address=job-cluster -Dparallelism.default=1 --config config-dev.properties

  taskmanager:
    image: ${FLINK_DOCKER_IMAGE_NAME:-timeseries-v1}
    command: task-manager -Djobmanager.rpc.address=job-cluster
    scale: 1


Excpetion:
4:32 AMorg.apache.flink.runtime.entrypoint.FlinkParseException: Failed to parse the command line arguments.
12/7/2018 10:44:32 AM          at org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:52)
12/7/2018 10:44:32 AM          at org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.main(StandaloneJobClusterEntryPoint.java:143)
12/7/2018 10:44:32 AMCaused by: org.apache.commons.cli.MissingArgumentException: Missing argument for option: j
12/7/2018 10:44:32 AM          at org.apache.commons.cli.DefaultParser.checkRequiredArgs(DefaultParser.java:211)
12/7/2018 10:44:32 AM          at org.apache.commons.cli.DefaultParser.handleOption(DefaultParser.java:599)
12/7/2018 10:44:32 AM          at org.apache.commons.cli.DefaultParser.handleShortAndLongOption(DefaultParser.java:548)
12/7/2018 10:44:32 AM          at org.apache.commons.cli.DefaultParser.handleToken(DefaultParser.java:243)
12/7/2018 10:44:32 AM          at org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:120)
12/7/2018 10:44:32 AM          at org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:81)
12/7/2018 10:44:32 AM          at org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:50)
12/7/2018 10:44:32 AM          ... 1 more
12/7/2018 10:44:32 AMException in thread "main" java.lang.NoSuchMethodError: org.apache.flink.runtime.entrypoint.parser.CommandLineParser.printHelp()V
12/7/2018 10:44:32 AM          at org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.main(StandaloneJobClusterEntryPoint.java:146)
Free forum by Nabble Edit this page