How long it took a Flink Job to start up ?

[Vijay Balakrishnan]

Hi,

I am trying to figure how long it took a Flink Job to start up ?

I used /jobs/overview and it gave me just the start-time as a long value. The Flink DashBoard UI shows the Start-Time as auser friendly date. I am trying to find or compute how long it took for the Flink Job to start up ?

Right now, I am manually looking at when the JobManager is up and running in the logs and then subtracting the UI start time to get an approximate idea. Trying to automate the process.

Attaching 2 screenshots of the /jobs/overview response and the Flink DashBoard UI Start Time.

TIA,

[Till Rohrmann]

Hi Vijay,

one way to solve this problem is to use Flink's REST API. By querying REST_ENDPOINT:PORT/jobs/JOB_ID [1] you obtain the job information. The job information contains the start-time for the whole job and the start-time for every vertex of the job. By subtracting the job's start time from the maximum start time of all vertices you should obtain the time it takes to bring up the whole topology (this only applies to streaming jobs since in batch jobs not all vertices/operators have to run at the same time).

[1] https://ci.apache.org/projects/flink/flink-docs-stable/monitoring/rest_api.html#jobs-jobid

Cheers,

Till

On Fri, Jul 31, 2020 at 7:56 PM Vijay Balakrishnan <[hidden email]> wrote:

Hi,

I am trying to figure how long it took a Flink Job to start up ?

I used /jobs/overview and it gave me just the start-time as a long value. The Flink DashBoard UI shows the Start-Time as auser friendly date. I am trying to find or compute how long it took for the Flink Job to start up ?

Right now, I am manually looking at when the JobManager is up and running in the logs and then subtracting the UI start time to get an approximate idea. Trying to automate the process.

Attaching 2 screenshots of the /jobs/overview response and the Flink DashBoard UI Start Time.

TIA,