Flink 1.13.0: check flamegraph on WebUI shows 404 not found

[Yik San Chan]

Hi community,

Flink 1.13.0 releases flamegraph. However, when I run Flink 1.13.0 locally, and try to check the flamegraph of an operator that is running, I got this "404 not found" error. This is the request and response.

Request:

curl 'http://localhost:8081/jobs/9f4ba3d39de6b2d9de0ea77ba8f8431a/vertices/398b7d36c52033324fc448876a5dfea6/flamegraph?type=on_cpu' \
  -H 'Connection: keep-alive' \
  -H 'sec-ch-ua: " Not A;Brand";v="99", "Chromium";v="90", "Google Chrome";v="90"' \
  -H 'Accept: application/json, text/plain, */*' \
  -H 'sec-ch-ua-mobile: ?0' \
  -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.93 Safari/537.36' \
  -H 'Sec-Fetch-Site: same-origin' \
  -H 'Sec-Fetch-Mode: cors' \
  -H 'Sec-Fetch-Dest: empty' \
  -H 'Referer: http://localhost:8081/' \
  -H 'Accept-Language: en-US,en;q=0.9' \
  -H 'Cookie: _ga=GA1.1.564121319.1616060869; _ga_QW5M5B4MME=GS1.1.1616334849.11.1.1616336196.0; username-localhost-8888="2|1:0|10:1618320509|23:username-localhost-8888|44:OGZmODRjYjcwZTYzNGYxY2E5NDNmM2VlZTk5MWNkM2E=|88feaedd691046c7c6bb4d7f474c3d9b165f2c63016d449a1a795943c536813d"; _xsrf=2|0afa28c9|61e67479ca4bc7fc9e4ef1f12621dbc7|1618320509' \
  --compressed

Response:

{"errors":["Unable to load requested file /jobs/9f4ba3d39de6b2d9de0ea77ba8f8431a/vertices/398b7d36c52033324fc448876a5dfea6/flamegraph."]}

This can be consistently reproduced regardless of what job I run.

Any idea why? Thanks!

Best,

Yik San

[Chesnay Schepler]

Did you enable the flamegraph feature in the configuration?

On 5/12/2021 10:51 AM, Yik San Chan wrote:

> Hi community,
>
> Flink 1.13.0 releases flamegraph. However, when I run Flink 1.13.0
> locally, and try to check the flamegraph of an operator that is
> running, I got this "404 not found" error. This is the request and
> response.
>
> Request:
> curl
> 'http://localhost:8081/jobs/9f4ba3d39de6b2d9de0ea77ba8f8431a/vertices/398b7d36c52033324fc448876a5dfea6/flamegraph?type=on_cpu 
> <http://localhost:8081/jobs/9f4ba3d39de6b2d9de0ea77ba8f8431a/vertices/398b7d36c52033324fc448876a5dfea6/flamegraph?type=on_cpu>'
> \
>   -H 'Connection: keep-alive' \
>   -H 'sec-ch-ua: " Not A;Brand";v="99", "Chromium";v="90", "Google
> Chrome";v="90"' \
>   -H 'Accept: application/json, text/plain, */*' \
>   -H 'sec-ch-ua-mobile: ?0' \
>   -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.93
> Safari/537.36' \
>   -H 'Sec-Fetch-Site: same-origin' \
>   -H 'Sec-Fetch-Mode: cors' \
>   -H 'Sec-Fetch-Dest: empty' \
>   -H 'Referer: http://localhost:8081/ <http://localhost:8081/>' \
>   -H 'Accept-Language: en-US,en;q=0.9' \
>   -H 'Cookie: _ga=GA1.1.564121319.1616060869;
> _ga_QW5M5B4MME=GS1.1.1616334849.11.1.1616336196.0;
> username-localhost-8888="2|1:0|10:1618320509|23:username-localhost-8888|44:OGZmODRjYjcwZTYzNGYxY2E5NDNmM2VlZTk5MWNkM2E=|88feaedd691046c7c6bb4d7f474c3d9b165f2c63016d449a1a795943c536813d";
> _xsrf=2|0afa28c9|61e67479ca4bc7fc9e4ef1f12621dbc7|1618320509' \
>   --compressed
>
> Response:
> {"errors":["Unable to load requested file
> /jobs/9f4ba3d39de6b2d9de0ea77ba8f8431a/vertices/398b7d36c52033324fc448876a5dfea6/flamegraph."]}
>
> This can be consistently reproduced regardless of what job I run.
>
> Any idea why? Thanks!
>
> Best,
> Yik San

[Yik San Chan]

Add rest.flamegraph.enabled: true to conf/flink-conf.yaml solves my problem. Thanks!

Best,

Yik San

On Wed, May 12, 2021 at 5:00 PM Chesnay Schepler <[hidden email]> wrote:

Did you enable the flamegraph feature in the configuration?

On 5/12/2021 10:51 AM, Yik San Chan wrote:
> Hi community,
>
> Flink 1.13.0 releases flamegraph. However, when I run Flink 1.13.0
> locally, and try to check the flamegraph of an operator that is
> running, I got this "404 not found" error. This is the request and
> response.
>
> Request:
> curl
> 'http://localhost:8081/jobs/9f4ba3d39de6b2d9de0ea77ba8f8431a/vertices/398b7d36c52033324fc448876a5dfea6/flamegraph?type=on_cpu
> <http://localhost:8081/jobs/9f4ba3d39de6b2d9de0ea77ba8f8431a/vertices/398b7d36c52033324fc448876a5dfea6/flamegraph?type=on_cpu>'
> \
>   -H 'Connection: keep-alive' \
>   -H 'sec-ch-ua: " Not A;Brand";v="99", "Chromium";v="90", "Google
> Chrome";v="90"' \
>   -H 'Accept: application/json, text/plain, */*' \
>   -H 'sec-ch-ua-mobile: ?0' \
>   -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.93
> Safari/537.36' \
>   -H 'Sec-Fetch-Site: same-origin' \
>   -H 'Sec-Fetch-Mode: cors' \
>   -H 'Sec-Fetch-Dest: empty' \
>   -H 'Referer: http://localhost:8081/ <http://localhost:8081/>' \
>   -H 'Accept-Language: en-US,en;q=0.9' \
>   -H 'Cookie: _ga=GA1.1.564121319.1616060869;
> _ga_QW5M5B4MME=GS1.1.1616334849.11.1.1616336196.0;
> username-localhost-8888="2|1:0|10:1618320509|23:username-localhost-8888|44:OGZmODRjYjcwZTYzNGYxY2E5NDNmM2VlZTk5MWNkM2E=|88feaedd691046c7c6bb4d7f474c3d9b165f2c63016d449a1a795943c536813d";
> _xsrf=2|0afa28c9|61e67479ca4bc7fc9e4ef1f12621dbc7|1618320509' \
>   --compressed
>
> Response:
> {"errors":["Unable to load requested file
> /jobs/9f4ba3d39de6b2d9de0ea77ba8f8431a/vertices/398b7d36c52033324fc448876a5dfea6/flamegraph."]}
>
> This can be consistently reproduced regardless of what job I run.
>
> Any idea why? Thanks!
>
> Best,
> Yik San