Watermark through Rest Api

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Watermark through Rest Api

Gregory Fee
Hello! I am interested in getting the current low watermark for tasks
in my Flink jobs. I know how to see them in the UI. I'm interested in
getting them programmatically, hopefully via rest api. The
documentation says that they are exposed as metrics but I don't see
watermark info in the 'metrics' section in the job detail. Does anyone
know how I might get the watermark information?

Thanks,
--
Gregory Fee
Engineer
Reply | Threaded
Open this post in threaded view
|

Re: Watermark through Rest Api

Piotr Nowojski
Hi,

Watermarks are tracked per Task/Operator level:


Tracking watermarks on the job level would be problematic, since it would require some kind of aggregation (min? Avg?) of all of the operators.

Piotrek 

On 9 Oct 2018, at 02:13, Gregory Fee <[hidden email]> wrote:

Hello! I am interested in getting the current low watermark for tasks
in my Flink jobs. I know how to see them in the UI. I'm interested in
getting them programmatically, hopefully via rest api. The
documentation says that they are exposed as metrics but I don't see
watermark info in the 'metrics' section in the job detail. Does anyone
know how I might get the watermark information?

Thanks,
--
Gregory Fee
Engineer