Customised RESTful trigger

[Jacek Grzebyta]

Hello,

I am a newbie in the Apache Flink environment. I found it is possible to trigger a job using the MONITORING REST API. Is it possible to customise a request to start a job with some parameters? From the bigger perspective I would like to provide a large file URL into a Flink application to do a TFL job. 

For example after request: 

/job?inputFile=s3://my-bucket/input/input-600m.json

Flink will start the FTL on an instance. Independently if the service will receive another query:

/job?inputFile=s3://my-bucket/input/input-other2G.json

Flink would start the other processing on the other job instance.

I thought I could deploy the jar file with parameters but that would be quite weird.

I have no idea how can I solve that without converting a REST request into a stream event first which would be the simplest.

Regards,

Jacek

[Chesnay Schepler]

You can specify arguments to your job via query parameters or a json body (recommended) as documented here.

On 10/07/2020 18:48, Jacek Grzebyta wrote:

Hello,

I am a newbie in the Apache Flink environment. I found it is possible to trigger a job using the MONITORING REST API. Is it possible to customise a request to start a job with some parameters? From the bigger perspective I would like to provide a large file URL into a Flink application to do a TFL job. 

For example after request: 

/job?inputFile=s3://my-bucket/input/input-600m.json

Flink will start the FTL on an instance. Independently if the service will receive another query:

/job?inputFile=s3://my-bucket/input/input-other2G.json

Flink would start the other processing on the other job instance.

I thought I could deploy the jar file with parameters but that would be quite weird.

I have no idea how can I solve that without converting a REST request into a stream event first which would be the simplest.

Regards,

Jacek