Hi Till
Thank you for your reply. Just like your suggestion, in our current implements, we periodically check the free slots through REST API, and then submitting jobs once slots are enough.
However, since there is a concept of ‘Flink cluster’, why can't we think about ‘cluster scheduling strategy’? The strategy may be different between long-running jobs and batch jobs. Could you please think about it?
Xinyu Zhang
From: Till Rohrmann [mailto:[hidden email]]
Sent: Wednesday, January 02, 2019 10:58 PM
To: 张馨予 <[hidden email]>
Cc: user <[hidden email]>
Subject: Re: Are Jobs allowed to be pending when slots are not enough
Hi Xinyu,
at the moment there is no such functionality in Flink. Whenever you submit a job, Flink will try to execute the job right away. If the job cannot get enough slots, then it will wait until the slot.request.timeout occurs and either fail or retry if you have a RestartStrategy configured.
If you want to wait until you have enough slots before submitting a job, I would suggest that you write yourself a small service which uses Flink's REST API [1] to query the status and finally submit the job if there are enough free slots.
Hi all
We submit some batch jobs to a Flink cluster which with 500 slots for example. The parallelism of these jobs may be different, between 1 and 500.
Is there any configuration that can make jobs running in submitting order once the cluster has enough slots? If not, could we meet this requirement?
Thanks.
Xinyu Zhang
Locked
