is there a way to get the watermark per operator in tabular form?

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

is there a way to get the watermark per operator in tabular form?

Jin Yi
in the flink ui, is there a way to update the columns being shown to include the watermarks?

in lieu of this, is it possible to query the watermarks throughout a flink job somehow?  the rest api?

thanks.
Reply | Threaded
Open this post in threaded view
|

Re: is there a way to get the watermark per operator in tabular form?

JING ZHANG
Hi Jin,
In Flink ui, there is already 'currentInputWatermark' [1] which represents the lowest watermark received by this task.

Besides, Flink provided the following metrics about watermark.
* currentInputWatermark [2], represents the lowest watermark received by this task
currentInputNWatermark [2], represents last watermark this operator has received in its N'th input (in milliseconds), with index N starting from 1. For example currentInput1Watermark, currentInput2Watermark, ...
currentOutputWatermark [2], represents the last watermark this operator has emitted

You could add those metrics in the Metric tab of Flink WEB UI Dashboard [3], or query these metrics through the Monitoring REST API [4].

BTW, there is a feature in progressed [5] to introduce a CURRENT_WATERMARK function in FLINK SQL. 


Best,
JING ZHANG

Jin Yi <[hidden email]> 于2021年6月16日周三 上午4:18写道:
in the flink ui, is there a way to update the columns being shown to include the watermarks?

in lieu of this, is it possible to query the watermarks throughout a flink job somehow?  the rest api?

thanks.