"405 HTTP method POST is not supported by this URL" is returned when POST a REST url on Flink on yarn

[徐涛]

Hi All,

I am trying to POST a RESTful url and want to generate a savepoint, the Flink version is 1.6.0.

When I executed the POST in local, everything is OK, but when I POST the url on a Flink on YARN application. The following error is returned:

“405 HTTP method POST is not supported by this URL”, I guess it is caused by YARN limitation. (http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/REST-API-quot-broken-quot-on-YARN-because-POST-is-not-allowed-via-YARN-proxy-td19329.html)

But does it have a workaround now?

Best

Henry

[Till Rohrmann]

Hi Henry,

I think when running Flink on Yarn, then you must not go through the Yarn proxy. Instead you should directly send the post request to the node on which the application master runs. When starting a Flink Yarn session via yarn-session.sh, then the web interface URL is printed to stdout, for example.

Cheers,

Till

On Tue, Sep 25, 2018 at 9:43 AM 徐涛 <[hidden email]> wrote:

Hi All,

I am trying to POST a RESTful url and want to generate a savepoint, the Flink version is 1.6.0.

When I executed the POST in local, everything is OK, but when I POST the url on a Flink on YARN application. The following error is returned:

“405 HTTP method POST is not supported by this URL”, I guess it is caused by YARN limitation. (http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/REST-API-quot-broken-quot-on-YARN-because-POST-is-not-allowed-via-YARN-proxy-td19329.html)

But does it have a workaround now?

Best

Henry

[徐涛]

Hi Till,

Actually I do send to request to the application master:

"http://storage4.test.lan:8089/proxy/application_1532081306755_0124/jobs/269608367e0f548f30d98aa4efa2211e/savepoints”

with post body

{ "target-directory" : "<a href="hdfs://flinkDsl/xxx" class="">hdfs://flinkDsl/xxx", "cancel-job" : "true” }

If I use the following GET url, everything is OK

"http://storage4.test.lan:8089/proxy/application_1532081306755_0124/jobs/269608367e0f548f30d98aa4efa2211e/checkpoints"

Best

Henry

在 2018年9月26日，上午5:32，Till Rohrmann <[hidden email]> 写道：

Hi Henry,

I think when running Flink on Yarn, then you must not go through the Yarn proxy. Instead you should directly send the post request to the node on which the application master runs. When starting a Flink Yarn session via yarn-session.sh, then the web interface URL is printed to stdout, for example.

Cheers,

Till

On Tue, Sep 25, 2018 at 9:43 AM 徐涛 <[hidden email]> wrote:

Hi All,

I am trying to POST a RESTful url and want to generate a savepoint, the Flink version is 1.6.0.

When I executed the POST in local, everything is OK, but when I POST the url on a Flink on YARN application. The following error is returned:

“405 HTTP method POST is not supported by this URL”, I guess it is caused by YARN limitation. (http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/REST-API-quot-broken-quot-on-YARN-because-POST-is-not-allowed-via-YARN-proxy-td19329.html)

But does it have a workaround now?

Best

Henry

[Gary Yao-2]

Hi Henry,

The URL below looks like the one from the YARN proxy (note that "proxy"
appears in the URL):

    http://storage4.test.lan:8089/proxy/application_1532081306755_0124/jobs/269608367e0f548f30d98aa4efa2211e/savepoints

You can use

    yarn application -status <APPLICATION_ID>

to find the host and port of the application master (AM host & RPC Port).

Best,
Gary

On Wed, Sep 26, 2018 at 3:23 AM 徐涛 <[hidden email]> wrote:

Hi Till,

Actually I do send to request to the application master:

"http://storage4.test.lan:8089/proxy/application_1532081306755_0124/jobs/269608367e0f548f30d98aa4efa2211e/savepoints”

with post body

{ "target-directory" : "hdfs://flinkDsl/xxx", "cancel-job" : "true” }

If I use the following GET url, everything is OK

"http://storage4.test.lan:8089/proxy/application_1532081306755_0124/jobs/269608367e0f548f30d98aa4efa2211e/checkpoints"

Best

Henry

在 2018年9月26日，上午5:32，Till Rohrmann <[hidden email]> 写道：

Hi Henry,

I think when running Flink on Yarn, then you must not go through the Yarn proxy. Instead you should directly send the post request to the node on which the application master runs. When starting a Flink Yarn session via yarn-session.sh, then the web interface URL is printed to stdout, for example.

Cheers,

Till

On Tue, Sep 25, 2018 at 9:43 AM 徐涛 <[hidden email]> wrote:

Hi All,

I am trying to POST a RESTful url and want to generate a savepoint, the Flink version is 1.6.0.

When I executed the POST in local, everything is OK, but when I POST the url on a Flink on YARN application. The following error is returned:

“405 HTTP method POST is not supported by this URL”, I guess it is caused by YARN limitation. (http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/REST-API-quot-broken-quot-on-YARN-because-POST-is-not-allowed-via-YARN-proxy-td19329.html)

But does it have a workaround now?

Best

Henry

[徐涛]

Hi Gary,

Thank you for your help.

Finally I get the host and address of application master using yarn client.

Best

Henry

在 2018年9月26日，下午2:26，Gary Yao <[hidden email]> 写道：

Hi Henry,

The URL below looks like the one from the YARN proxy (note that "proxy"
appears in the URL):

    http://storage4.test.lan:8089/proxy/application_1532081306755_0124/jobs/269608367e0f548f30d98aa4efa2211e/savepoints

You can use

    yarn application -status <APPLICATION_ID>

to find the host and port of the application master (AM host & RPC Port).

Best,
Gary

On Wed, Sep 26, 2018 at 3:23 AM 徐涛 <[hidden email]> wrote:

Hi Till,

Actually I do send to request to the application master:

"http://storage4.test.lan:8089/proxy/application_1532081306755_0124/jobs/269608367e0f548f30d98aa4efa2211e/savepoints”

with post body

{ "target-directory" : "hdfs://flinkDsl/xxx", "cancel-job" : "true” }

If I use the following GET url, everything is OK

"http://storage4.test.lan:8089/proxy/application_1532081306755_0124/jobs/269608367e0f548f30d98aa4efa2211e/checkpoints"

Best

Henry

在 2018年9月26日，上午5:32，Till Rohrmann <[hidden email]> 写道：

Hi Henry,

I think when running Flink on Yarn, then you must not go through the Yarn proxy. Instead you should directly send the post request to the node on which the application master runs. When starting a Flink Yarn session via yarn-session.sh, then the web interface URL is printed to stdout, for example.

Cheers,

Till

On Tue, Sep 25, 2018 at 9:43 AM 徐涛 <[hidden email]> wrote:

Hi All,

I am trying to POST a RESTful url and want to generate a savepoint, the Flink version is 1.6.0.

When I executed the POST in local, everything is OK, but when I POST the url on a Flink on YARN application. The following error is returned:

“405 HTTP method POST is not supported by this URL”, I guess it is caused by YARN limitation. (http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/REST-API-quot-broken-quot-on-YARN-because-POST-is-not-allowed-via-YARN-proxy-td19329.html)

But does it have a workaround now?

Best

Henry

[徐涛]

Hi Gary,

Thank you for your help.

Finally I get the host and address of application master using yarn client.

Best

Henry

在 2018年9月26日，下午2:26，Gary Yao <[hidden email]> 写道：

Hi Henry,

The URL below looks like the one from the YARN proxy (note that "proxy"
appears in the URL):

    http://storage4.test.lan:8089/proxy/application_1532081306755_0124/jobs/269608367e0f548f30d98aa4efa2211e/savepoints

You can use

    yarn application -status <APPLICATION_ID>

to find the host and port of the application master (AM host & RPC Port).

Best,
Gary

On Wed, Sep 26, 2018 at 3:23 AM 徐涛 <[hidden email]> wrote:

Hi Till,

Actually I do send to request to the application master:

"http://storage4.test.lan:8089/proxy/application_1532081306755_0124/jobs/269608367e0f548f30d98aa4efa2211e/savepoints”

with post body

{ "target-directory" : "hdfs://flinkDsl/xxx", "cancel-job" : "true” }

If I use the following GET url, everything is OK

"http://storage4.test.lan:8089/proxy/application_1532081306755_0124/jobs/269608367e0f548f30d98aa4efa2211e/checkpoints"

Best

Henry

在 2018年9月26日，上午5:32，Till Rohrmann <[hidden email]> 写道：

Hi Henry,

I think when running Flink on Yarn, then you must not go through the Yarn proxy. Instead you should directly send the post request to the node on which the application master runs. When starting a Flink Yarn session via yarn-session.sh, then the web interface URL is printed to stdout, for example.

Cheers,

Till

On Tue, Sep 25, 2018 at 9:43 AM 徐涛 <[hidden email]> wrote:

Hi All,

I am trying to POST a RESTful url and want to generate a savepoint, the Flink version is 1.6.0.

When I executed the POST in local, everything is OK, but when I POST the url on a Flink on YARN application. The following error is returned:

“405 HTTP method POST is not supported by this URL”, I guess it is caused by YARN limitation. (http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/REST-API-quot-broken-quot-on-YARN-because-POST-is-not-allowed-via-YARN-proxy-td19329.html)

But does it have a workaround now?

Best

Henry