Submitting jobs via REST

[Timothy Victor]

I have a flink docker image with my job's JAR already contained within.  I would like to run a job with this jar via the REST api.  Is that possible?

I know I can run a job via REST using JarID (ID assigned by flink when a jar is uploaded).   However I don't have such an ID since this jar is already part of the image.   

Via CLI I can start a job using classpath.  But can I do the same via the REST api.   Any other ways to achieve this?

Thanks 

Tim

[Pritam Sadhukhan]

Hi Tim,

I have the similar scenario where I have embedded my jar within the image.

I used the following command to submit the job :

curl -X POST  http://localhost:8081/jars/<jar-name>.jar/run

with the request parameters if any.

Please let me know if this helps.

Regards,

Pritam.

On Sat, 19 Oct 2019 at 20:06, Timothy Victor <[hidden email]> wrote:

I have a flink docker image with my job's JAR already contained within.  I would like to run a job with this jar via the REST api.  Is that possible?

I know I can run a job via REST using JarID (ID assigned by flink when a jar is uploaded).   However I don't have such an ID since this jar is already part of the image.   

Via CLI I can start a job using classpath.  But can I do the same via the REST api.   Any other ways to achieve this?

Thanks 

Tim

[Timothy Victor]

Thanks Pritam.

Unfortunately this does not work for me.  I get a response that says  "jar file /tmp/flink-web-<guid>/flink-web-upload/<jarname> does not exist".

It is looking for the jar in the tmp folder.  Wonder of there is a way to change that so that it looks in the right folder.

Thanks 

Tim

On Sun, Oct 20, 2019, 7:55 AM Pritam Sadhukhan <[hidden email]> wrote:

Hi Tim,

I have the similar scenario where I have embedded my jar within the image.

I used the following command to submit the job :

curl -X POST  http://localhost:8081/jars/<jar-name>.jar/run

with the request parameters if any.

Please let me know if this helps.

Regards,

Pritam.

On Sat, 19 Oct 2019 at 20:06, Timothy Victor <[hidden email]> wrote:

I have a flink docker image with my job's JAR already contained within.  I would like to run a job with this jar via the REST api.  Is that possible?

I know I can run a job via REST using JarID (ID assigned by flink when a jar is uploaded).   However I don't have such an ID since this jar is already part of the image.   

Via CLI I can start a job using classpath.  But can I do the same via the REST api.   Any other ways to achieve this?

Thanks 

Tim

[Pritam Sadhukhan]

Can you please share your dockerfile?

Please upload your jar at /opt/flink/product-libs/flink-web-upload/.

Regards,

Pritam.

On Mon, 21 Oct 2019 at 19:58, Timothy Victor <[hidden email]> wrote:

Thanks Pritam.

Unfortunately this does not work for me.  I get a response that says  "jar file /tmp/flink-web-<guid>/flink-web-upload/<jarname> does not exist".

It is looking for the jar in the tmp folder.  Wonder of there is a way to change that so that it looks in the right folder.

Thanks 

Tim

On Sun, Oct 20, 2019, 7:55 AM Pritam Sadhukhan <[hidden email]> wrote:

Hi Tim,

I have the similar scenario where I have embedded my jar within the image.

I used the following command to submit the job :

curl -X POST  http://localhost:8081/jars/<jar-name>.jar/run

with the request parameters if any.

Please let me know if this helps.

Regards,

Pritam.

On Sat, 19 Oct 2019 at 20:06, Timothy Victor <[hidden email]> wrote:

I have a flink docker image with my job's JAR already contained within.  I would like to run a job with this jar via the REST api.  Is that possible?

I know I can run a job via REST using JarID (ID assigned by flink when a jar is uploaded).   However I don't have such an ID since this jar is already part of the image.   

Via CLI I can start a job using classpath.  But can I do the same via the REST api.   Any other ways to achieve this?

Thanks 

Tim

[Timothy Victor]

Thank you for your help Pritam.  I got this working now.   The missing pool piece for me was that I had to set the "web.upload.dir" which my default is some auto generated directory under tmp.

One problem I had was that the jar actually needs to be placed under $web.upload.dir/flink-web-upload, i.e.a hardcoded subdir under the user specified dir.   It would be nice if that didn't happen and flink just read directly from the root ($web.upload.dir).   I can create a ticket for it to see if there is any general interest in such a change.   

Thanks again for your help

Tim

On Mon, Oct 21, 2019, 10:37 AM Pritam Sadhukhan <[hidden email]> wrote:

Can you please share your dockerfile?

Please upload your jar at /opt/flink/product-libs/flink-web-upload/.

Regards,

Pritam.

On Mon, 21 Oct 2019 at 19:58, Timothy Victor <[hidden email]> wrote:

Thanks Pritam.

Unfortunately this does not work for me.  I get a response that says  "jar file /tmp/flink-web-<guid>/flink-web-upload/<jarname> does not exist".

It is looking for the jar in the tmp folder.  Wonder of there is a way to change that so that it looks in the right folder.

Thanks 

Tim

On Sun, Oct 20, 2019, 7:55 AM Pritam Sadhukhan <[hidden email]> wrote:

Hi Tim,

I have the similar scenario where I have embedded my jar within the image.

I used the following command to submit the job :

curl -X POST  http://localhost:8081/jars/<jar-name>.jar/run

with the request parameters if any.

Please let me know if this helps.

Regards,

Pritam.

On Sat, 19 Oct 2019 at 20:06, Timothy Victor <[hidden email]> wrote:

I have a flink docker image with my job's JAR already contained within.  I would like to run a job with this jar via the REST api.  Is that possible?

I know I can run a job via REST using JarID (ID assigned by flink when a jar is uploaded).   However I don't have such an ID since this jar is already part of the image.   

Via CLI I can start a job using classpath.  But can I do the same via the REST api.   Any other ways to achieve this?

Thanks 

Tim