Submitting a job in non-blocking mode using curl and the REST API
Locked
Submitting a job in non-blocking mode using curl and the REST API
 
|
Hey everyone, I've got an awesome looking Flink cluster set up with web.submit.enable=true, and plenty of bash for handling jar upload and then submission to a JobManager - all good so far.
Unfortunately, when I try to submit the classic WordCount example, I get a massive error with the jist of it being:
"Job was submitted in detached mode. Results of job execution, such as accumulators, runtime, etc. are not available. Please make sure your program doesn't call an eager execution function [collect, print, printToErr, count]."
So, how do I run it *not* in detached mode using curl please?
I'm intentionally not using the Flink CLI because I am using an nginx with auth proxy set up - so I'm doing everything with curl, in a bash script - (so, two requests - one to upload the jar, then I get the ID from the response, and then submit the job with that ID).
At https://ci.apache.org/projects/flink/flink-docs-release-1.12/ops/rest_api.html if you ctrl-f for /run, there's nothing obvious that indicates how I can run in blocking mode - the biggest clue I've got is `programArg`. So I'm wondering if I can provide that somehow.
For those who prefer code:
curl ${auth_options} ${self_signed_flag} ${ca_cert_flag} -X POST -H "Content-Type: application/json" https://${JOBMANAGER}/jars/${uploaded_jar_string}/run$programArgsToUse
Whereby programArgsToUse is user args, and I'm cool with them being query parameters for now - I think.
I'm passing them on the end with:
if [[ ! -z $program_args ]] ; then programArgsToUse="?programArg=$program_args" fi curl --cacert /etc/ssl/tester/certs/ca.crt -X POST -H 'Content-Type: application/json' 'https://tester-minimal-tls-sample-jobmanager:8081/jars/fdc7684f-323d-49fa-a60a-96683d953be8_WordCount.jar/run?programArg=detached=false'
(obviously, what's at the end looks really wrong, but IDK what to use)
The only mention of "detach" I see documented is at https://ci.apache.org/projects/flink/flink-docs-stable/deployment/config.html, and that says by default it's not in detached mode for execution. If there are any better docs or examples, please send them my way - or if you've spotted me being just plain silly with my bash, that would be fantastic to point out.
Thanks a lot in advance, cheers, happy to share any more code/background if need be IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU |
Re: Submitting a job in non-blocking mode using curl and the REST API
|
|
All jobs going through the
web-submission are run in detached mode for technical reasons
(blocking of threads, and information having to be transported
back to the JobManager for things like collect()).
You unfortunately cannot run
non-detached/attached/blocking jobs via the web submission, which
includes the WordCount example because it uses specific methods
(the ones mentioned in the exception; collect, print, printToErr,
count).
In other words, your setup appears to
be fine correctly, you are just trying to do something that is not
supported.
On 1/5/2021 4:07 PM, Adam Roberts
wrote:
|
RE: Submitting a job in non-blocking mode using curl and the REST API
|
|
Thanks Chesnay for the prompt response - ah, so my cunning plan to use execution.attached=true doesn't sound so reasonable now then (I was going to look at providing that as a programArg next).
I did find, in https://ci.apache.org/projects/flink/flink-docs-release-1.11/release-notes/flink-1.11.html, this which I assume you are referring to
Web Submission behaves the same as detached mode.With FLINK-16657 the web submission logic changes and it exposes the same behavior as submitting a job through the CLI in detached mode. This implies that, for instance, jobs based on the DataSet API that were using sinks like So, here's a question - if we are advised to use a proxy to support alternative auth mechanisms, and those mechanisms don’t work with the CLI (thus forcing the use of curl)...how are we supposed to submit a job with print(), count() or collect() etc?
I know you've said it's not supported, but is that an "at the moment" kinda thing? Is this something planned or something you think I should create a JIRA issue for?
Thanks again, much appreciated
----- Original message ----- IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU |
Re: Submitting a job in non-blocking mode using curl and the REST API
|
|
I'm not aware of any plans to change
this behavior. Overall the community is rather split on what role
the web-submission should play; whether it should be a genuine way
to submit jobs with the same capabilities as the CLI, just for
prototyping or not exist at all.
As it stands we are somewhat hovering
around the prototyping-take, so a more appropriate long-term
solution would be to allow the CLI (which after all also goes
through the REST API) to support proxies/authentication etc., but
that is likely blocked on replacing our HTTP client with some
off-the-shelf library.
One thing to note is that methods are
generally not considered to be used for production-code, and are
more for prototyping.
Ideally you can find a way to simply
not rely on these methods.
How/whether this can be done depends of
course on the use-case; for example let's say a job is executed
and dependent on the result of count/collect a second job should
be executed. An alternative approach might write equivalent data
to some external system, and then read it back on the client-side
to orchestrate the scheduling of other jobs.
Another hacky option might be to use
the CLI, but intercepting the message and enriching it with
additional authentication information?
On 1/5/2021 5:17 PM, Adam Roberts
wrote:
|
«
Return to (DEPRECATED) Apache Flink User Mailing List archive.
|
1 view|%1 views
| Free forum by Nabble | Edit this page |