is there a way to get the watermark per operator in tabular form?

[Jin Yi]

in the flink ui, is there a way to update the columns being shown to include the watermarks?

in lieu of this, is it possible to query the watermarks throughout a flink job somehow?  the rest api?

thanks.

[JING ZHANG]

Hi Jin,

In Flink ui, there is already 'currentInputWatermark' [1] which represents the lowest watermark received by this task.

Besides, Flink provided the following metrics about watermark.

* currentInputWatermark [2], represents the lowest watermark received by this task

* currentInputNWatermark [2], represents last watermark this operator has received in its N'th input (in milliseconds), with index N starting from 1. For example currentInput1Watermark, currentInput2Watermark, ...

* currentOutputWatermark [2], represents the last watermark this operator has emitted

You could add those metrics in the Metric tab of Flink WEB UI Dashboard [3], or query these metrics through the Monitoring REST API [4].

BTW, there is a feature in progressed [5] to introduce a CURRENT_WATERMARK function in FLINK SQL. 

[1] https://ci.apache.org/projects/flink/flink-docs-master/docs/ops/debugging/debugging_event_time/

[2] https://ci.apache.org/projects/flink/flink-docs-master/docs/ops/metrics/

[3] https://ci.apache.org/projects/flink/flink-docs-master/docs/ops/metrics/#dashboard-integration

[4] https://ci.apache.org/projects/flink/flink-docs-master/docs/ops/metrics/#rest-api-integration

[5] https://issues.apache.org/jira/browse/FLINK-22737

Best,

JING ZHANG

Jin Yi <[hidden email]> 于2021年6月16日周三 上午4:18写道：

in the flink ui, is there a way to update the columns being shown to include the watermarks?

in lieu of this, is it possible to query the watermarks throughout a flink job somehow?  the rest api?

thanks.