Flink 'Job Cluster' mode Ui Access

[Jatin Banger]

Hi,

It seems there is Web Ui for Flink Session cluster, But for Flink Job Cluster it is Showing 

{"errors":["Not found."]}

Is it the expected behavior for Flink Job Cluster Mode ?

Best Regards,

Jatin

[vino yang]

Hi Jatin,

Flink web UI does not depend on any deployment mode.

You should check if there are error logs in the log file and the job status is running state.

Best,

Vino

Jatin Banger <[hidden email]> 于2019年11月28日周四 下午3:43写道：

Hi,

It seems there is Web Ui for Flink Session cluster, But for Flink Job Cluster it is Showing 

{"errors":["Not found."]}

Is it the expected behavior for Flink Job Cluster Mode ?

Best Regards,

Jatin

[Jatin Banger]

Hi,

I checked the log file there is no error.

And I checked the pods internal ports by using rest api.

# curl <pod-Ip>: 4081

{"errors":["Not found."]}

4081 is the Ui port

# curl <pod-Ip>:4081/config

{"refresh-interval":3000,"timezone-name":"Coordinated Universal Time","timezone-offset":0,"flink-version":"<unknown>","flink-revision":"ceba8af @ 11.02.2019 @ 22:17:09 CST"}

# curl <pod-Ip>:4081/jobs

{"jobs":[{"id":"___job_Id_____","status":"RUNNING"}]}

Which shows the state of the job as running.

What else can we do ?

Best regards,

Jatin

On Thu, Nov 28, 2019 at 1:28 PM vino yang <[hidden email]> wrote:

Hi Jatin,

Flink web UI does not depend on any deployment mode.

You should check if there are error logs in the log file and the job status is running state.

Best,

Vino

Jatin Banger <[hidden email]> 于2019年11月28日周四 下午3:43写道：

Hi,

It seems there is Web Ui for Flink Session cluster, But for Flink Job Cluster it is Showing 

{"errors":["Not found."]}

Is it the expected behavior for Flink Job Cluster Mode ?

Best Regards,

Jatin

[vino yang]

Hi Jatin,

Which version do you use?

Best,

Vino

Jatin Banger <[hidden email]> 于2019年11月28日周四 下午5:03写道：

Hi,

I checked the log file there is no error.

And I checked the pods internal ports by using rest api.

# curl <pod-Ip>: 4081

{"errors":["Not found."]}

4081 is the Ui port

# curl <pod-Ip>:4081/config

{"refresh-interval":3000,"timezone-name":"Coordinated Universal Time","timezone-offset":0,"flink-version":"<unknown>","flink-revision":"ceba8af @ 11.02.2019 @ 22:17:09 CST"}

# curl <pod-Ip>:4081/jobs

{"jobs":[{"id":"___job_Id_____","status":"RUNNING"}]}

Which shows the state of the job as running.

What else can we do ?

Best regards,

Jatin

On Thu, Nov 28, 2019 at 1:28 PM vino yang <[hidden email]> wrote:

Hi Jatin,

Flink web UI does not depend on any deployment mode.

You should check if there are error logs in the log file and the job status is running state.

Best,

Vino

Jatin Banger <[hidden email]> 于2019年11月28日周四 下午3:43写道：

Hi,

It seems there is Web Ui for Flink Session cluster, But for Flink Job Cluster it is Showing 

{"errors":["Not found."]}

Is it the expected behavior for Flink Job Cluster Mode ?

Best Regards,

Jatin

[Chesnay Schepler]

Could you try accessing <pid-Ip>:/#/overview ?

The REST API is obviously accessible, and hence the WebUI should be too.

How did you setup the session cluster? Are you using some custom Flink build or something, which potentially excluded flink-runtime-web from the classpath?

On 28/11/2019 10:02, Jatin Banger wrote:

Hi,

I checked the log file there is no error.

And I checked the pods internal ports by using rest api.

# curl <pod-Ip>: 4081

{"errors":["Not found."]}

4081 is the Ui port

# curl <pod-Ip>:4081/config

{"refresh-interval":3000,"timezone-name":"Coordinated Universal Time","timezone-offset":0,"flink-version":"<unknown>","flink-revision":"ceba8af @ 11.02.2019 @ 22:17:09 CST"}

# curl <pod-Ip>:4081/jobs

{"jobs":[{"id":"___job_Id_____","status":"RUNNING"}]}

Which shows the state of the job as running.

What else can we do ?

Best regards,

Jatin

On Thu, Nov 28, 2019 at 1:28 PM vino yang <[hidden email]> wrote:

Hi Jatin,

Flink web UI does not depend on any deployment mode.

You should check if there are error logs in the log file and the job status is running state.

Best,

Vino

Jatin Banger <[hidden email]> 于2019年11月28日周四 下午3:43写道：

Hi,

It seems there is Web Ui for Flink Session cluster, But for Flink Job Cluster it is Showing 

{"errors":["Not found."]}

Is it the expected behavior for Flink Job Cluster Mode ?

Best Regards,

Jatin

[Jatin Banger]

Hi,

[hidden email]  I am using flink 1.8.1

I am using the following procedure for the deployment:

https://github.com/apache/flink/blob/master/flink-container/docker/README.md

And i tried accessing the path you mentioned:

# curl <pod-Ip>:4081/#/overview

{"errors":["Not found."]}

Best Regards,

Jatin 

On Thu, Nov 28, 2019 at 10:21 PM Chesnay Schepler <[hidden email]> wrote:

Could you try accessing <pid-Ip>:/#/overview ?

The REST API is obviously accessible, and hence the WebUI should be too.

How did you setup the session cluster? Are you using some custom Flink build or something, which potentially excluded flink-runtime-web from the classpath?

On 28/11/2019 10:02, Jatin Banger wrote:

Hi,

I checked the log file there is no error.

And I checked the pods internal ports by using rest api.

# curl <pod-Ip>: 4081

{"errors":["Not found."]}

4081 is the Ui port

# curl <pod-Ip>:4081/config

{"refresh-interval":3000,"timezone-name":"Coordinated Universal Time","timezone-offset":0,"flink-version":"<unknown>","flink-revision":"ceba8af @ 11.02.2019 @ 22:17:09 CST"}

# curl <pod-Ip>:4081/jobs

{"jobs":[{"id":"___job_Id_____","status":"RUNNING"}]}

Which shows the state of the job as running.

What else can we do ?

Best regards,

Jatin

On Thu, Nov 28, 2019 at 1:28 PM vino yang <[hidden email]> wrote:

Hi Jatin,

Flink web UI does not depend on any deployment mode.

You should check if there are error logs in the log file and the job status is running state.

Best,

Vino

Jatin Banger <[hidden email]> 于2019年11月28日周四 下午3:43写道：

Hi,

It seems there is Web Ui for Flink Session cluster, But for Flink Job Cluster it is Showing 

{"errors":["Not found."]}

Is it the expected behavior for Flink Job Cluster Mode ?

Best Regards,

Jatin

[Chesnay Schepler]

To clarify, you ran "mvn package -pl flink-dist -am" to build Fink?

If so, could you run that again and provide us with the maven output?




On 29/11/2019 11:23, Jatin Banger wrote:

Hi,

[hidden email]  I am using flink 1.8.1

I am using the following procedure for the deployment:

https://github.com/apache/flink/blob/master/flink-container/docker/README.md

And i tried accessing the path you mentioned:

# curl <pod-Ip>:4081/#/overview

{"errors":["Not found."]}

Best Regards,

Jatin 

On Thu, Nov 28, 2019 at 10:21 PM Chesnay Schepler <[hidden email]> wrote:

Could you try accessing <pid-Ip>:/#/overview ?

The REST API is obviously accessible, and hence the WebUI should be too.

How did you setup the session cluster? Are you using some custom Flink build or something, which potentially excluded flink-runtime-web from the classpath?

On 28/11/2019 10:02, Jatin Banger wrote:

Hi,

I checked the log file there is no error.

And I checked the pods internal ports by using rest api.

# curl <pod-Ip>: 4081

{"errors":["Not found."]}

4081 is the Ui port

# curl <pod-Ip>:4081/config

{"refresh-interval":3000,"timezone-name":"Coordinated Universal Time","timezone-offset":0,"flink-version":"<unknown>","flink-revision":"ceba8af @ 11.02.2019 @ 22:17:09 CST"}

# curl <pod-Ip>:4081/jobs

{"jobs":[{"id":"___job_Id_____","status":"RUNNING"}]}

Which shows the state of the job as running.

What else can we do ?

Best regards,

Jatin

On Thu, Nov 28, 2019 at 1:28 PM vino yang <[hidden email]> wrote:

Hi Jatin,

Flink web UI does not depend on any deployment mode.

You should check if there are error logs in the log file and the job status is running state.

Best,

Vino

Jatin Banger <[hidden email]> 于2019年11月28日周四 下午3:43写道：

Hi,

It seems there is Web Ui for Flink Session cluster, But for Flink Job Cluster it is Showing 

{"errors":["Not found."]}

Is it the expected behavior for Flink Job Cluster Mode ?

Best Regards,

Jatin

[Jatin Banger]

Hi,

I am using flink binary directly.

I am using this command to deploy the script.

                        "$FLINK_HOME/bin/standalone-job.sh" start-foreground --job-classname ${ARGS_FOR_JOB}

where ARGS_FOR_JOB contain job class name and all other necessary details needed by the job.

Best regards,

Jatin

On Fri, Nov 29, 2019 at 4:18 PM Chesnay Schepler <[hidden email]> wrote:

To clarify, you ran "mvn package -pl flink-dist -am" to build Fink?

If so, could you run that again and provide us with the maven output?




On 29/11/2019 11:23, Jatin Banger wrote:

Hi,

[hidden email]  I am using flink 1.8.1

I am using the following procedure for the deployment:

https://github.com/apache/flink/blob/master/flink-container/docker/README.md

And i tried accessing the path you mentioned:

# curl <pod-Ip>:4081/#/overview

{"errors":["Not found."]}

Best Regards,

Jatin 

On Thu, Nov 28, 2019 at 10:21 PM Chesnay Schepler <[hidden email]> wrote:

Could you try accessing <pid-Ip>:/#/overview ?

The REST API is obviously accessible, and hence the WebUI should be too.

How did you setup the session cluster? Are you using some custom Flink build or something, which potentially excluded flink-runtime-web from the classpath?

On 28/11/2019 10:02, Jatin Banger wrote:

Hi,

I checked the log file there is no error.

And I checked the pods internal ports by using rest api.

# curl <pod-Ip>: 4081

{"errors":["Not found."]}

4081 is the Ui port

# curl <pod-Ip>:4081/config

{"refresh-interval":3000,"timezone-name":"Coordinated Universal Time","timezone-offset":0,"flink-version":"<unknown>","flink-revision":"ceba8af @ 11.02.2019 @ 22:17:09 CST"}

# curl <pod-Ip>:4081/jobs

{"jobs":[{"id":"___job_Id_____","status":"RUNNING"}]}

Which shows the state of the job as running.

What else can we do ?

Best regards,

Jatin

On Thu, Nov 28, 2019 at 1:28 PM vino yang <[hidden email]> wrote:

Hi Jatin,

Flink web UI does not depend on any deployment mode.

You should check if there are error logs in the log file and the job status is running state.

Best,

Vino

Jatin Banger <[hidden email]> 于2019年11月28日周四 下午3:43写道：

Hi,

It seems there is Web Ui for Flink Session cluster, But for Flink Job Cluster it is Showing 

{"errors":["Not found."]}

Is it the expected behavior for Flink Job Cluster Mode ?

Best Regards,

Jatin

[Chesnay Schepler]

hmm...this is quite odd.

Let's try to narrow things down a bit.

Could you try starting a local cluster (using the same distribution) and checking whether the UI is accessible?

Could you also check whether the flink-dist.jar in /lib contains web/index.html?

On 04/12/2019 06:02, Jatin Banger wrote:

Hi,

I am using flink binary directly.

I am using this command to deploy the script.

                        "$FLINK_HOME/bin/standalone-job.sh" start-foreground --job-classname ${ARGS_FOR_JOB}

where ARGS_FOR_JOB contain job class name and all other necessary details needed by the job.

Best regards,

Jatin

On Fri, Nov 29, 2019 at 4:18 PM Chesnay Schepler <[hidden email]> wrote:

To clarify, you ran "mvn package -pl flink-dist -am" to build Fink?

If so, could you run that again and provide us with the maven output?




On 29/11/2019 11:23, Jatin Banger wrote:

Hi,

[hidden email]  I am using flink 1.8.1

I am using the following procedure for the deployment:

https://github.com/apache/flink/blob/master/flink-container/docker/README.md

And i tried accessing the path you mentioned:

# curl <pod-Ip>:4081/#/overview

{"errors":["Not found."]}

Best Regards,

Jatin 

On Thu, Nov 28, 2019 at 10:21 PM Chesnay Schepler <[hidden email]> wrote:

Could you try accessing <pid-Ip>:/#/overview ?

The REST API is obviously accessible, and hence the WebUI should be too.

How did you setup the session cluster? Are you using some custom Flink build or something, which potentially excluded flink-runtime-web from the classpath?

On 28/11/2019 10:02, Jatin Banger wrote:

Hi,

I checked the log file there is no error.

And I checked the pods internal ports by using rest api.

# curl <pod-Ip>: 4081

{"errors":["Not found."]}

4081 is the Ui port

# curl <pod-Ip>:4081/config

{"refresh-interval":3000,"timezone-name":"Coordinated Universal Time","timezone-offset":0,"flink-version":"<unknown>","flink-revision":"ceba8af @ 11.02.2019 @ 22:17:09 CST"}

# curl <pod-Ip>:4081/jobs

{"jobs":[{"id":"___job_Id_____","status":"RUNNING"}]}

Which shows the state of the job as running.

What else can we do ?

Best regards,

Jatin

On Thu, Nov 28, 2019 at 1:28 PM vino yang <[hidden email]> wrote:

Hi Jatin,

Flink web UI does not depend on any deployment mode.

You should check if there are error logs in the log file and the job status is running state.

Best,

Vino

Jatin Banger <[hidden email]> 于2019年11月28日周四 下午3:43写道：

Hi,

It seems there is Web Ui for Flink Session cluster, But for Flink Job Cluster it is Showing 

{"errors":["Not found."]}

Is it the expected behavior for Flink Job Cluster Mode ?

Best Regards,

Jatin

[Jatin Banger]

I have tried that already using 

'$FLINK_HOME/bin/jobmanager.sh" start-foreground

Ui comes fine with this one.

Which means web/index.html is present.

On Wed, Dec 4, 2019 at 9:01 PM Chesnay Schepler <[hidden email]> wrote:

hmm...this is quite odd.

Let's try to narrow things down a bit.

Could you try starting a local cluster (using the same distribution) and checking whether the UI is accessible?

Could you also check whether the flink-dist.jar in /lib contains web/index.html?

On 04/12/2019 06:02, Jatin Banger wrote:

Hi,

I am using flink binary directly.

I am using this command to deploy the script.

                        "$FLINK_HOME/bin/standalone-job.sh" start-foreground --job-classname ${ARGS_FOR_JOB}

where ARGS_FOR_JOB contain job class name and all other necessary details needed by the job.

Best regards,

Jatin

On Fri, Nov 29, 2019 at 4:18 PM Chesnay Schepler <[hidden email]> wrote:

To clarify, you ran "mvn package -pl flink-dist -am" to build Fink?

If so, could you run that again and provide us with the maven output?




On 29/11/2019 11:23, Jatin Banger wrote:

Hi,

[hidden email]  I am using flink 1.8.1

I am using the following procedure for the deployment:

https://github.com/apache/flink/blob/master/flink-container/docker/README.md

And i tried accessing the path you mentioned:

# curl <pod-Ip>:4081/#/overview

{"errors":["Not found."]}

Best Regards,

Jatin 

On Thu, Nov 28, 2019 at 10:21 PM Chesnay Schepler <[hidden email]> wrote:

Could you try accessing <pid-Ip>:/#/overview ?

The REST API is obviously accessible, and hence the WebUI should be too.

How did you setup the session cluster? Are you using some custom Flink build or something, which potentially excluded flink-runtime-web from the classpath?

On 28/11/2019 10:02, Jatin Banger wrote:

Hi,

I checked the log file there is no error.

And I checked the pods internal ports by using rest api.

# curl <pod-Ip>: 4081

{"errors":["Not found."]}

4081 is the Ui port

# curl <pod-Ip>:4081/config

{"refresh-interval":3000,"timezone-name":"Coordinated Universal Time","timezone-offset":0,"flink-version":"<unknown>","flink-revision":"ceba8af @ 11.02.2019 @ 22:17:09 CST"}

# curl <pod-Ip>:4081/jobs

{"jobs":[{"id":"___job_Id_____","status":"RUNNING"}]}

Which shows the state of the job as running.

What else can we do ?

Best regards,

Jatin

On Thu, Nov 28, 2019 at 1:28 PM vino yang <[hidden email]> wrote:

Hi Jatin,

Flink web UI does not depend on any deployment mode.

You should check if there are error logs in the log file and the job status is running state.

Best,

Vino

Jatin Banger <[hidden email]> 于2019年11月28日周四 下午3:43写道：

Hi,

It seems there is Web Ui for Flink Session cluster, But for Flink Job Cluster it is Showing 

{"errors":["Not found."]}

Is it the expected behavior for Flink Job Cluster Mode ?

Best Regards,

Jatin

[Chesnay Schepler]

Ok, it's good to know that the WebUI files are there.

Please enable DEBUG logging and try again, searching for messages from the StaticFileServerHandler.

This handler logs every file that is requested (which effectively happens when the WebUI is being served); let's see what is actually being requested.

On 05/12/2019 05:57, Jatin Banger wrote:

I have tried that already using 

'$FLINK_HOME/bin/jobmanager.sh" start-foreground

Ui comes fine with this one.

Which means web/index.html is present.

On Wed, Dec 4, 2019 at 9:01 PM Chesnay Schepler <[hidden email]> wrote:

hmm...this is quite odd.

Let's try to narrow things down a bit.

Could you try starting a local cluster (using the same distribution) and checking whether the UI is accessible?

Could you also check whether the flink-dist.jar in /lib contains web/index.html?

On 04/12/2019 06:02, Jatin Banger wrote:

Hi,

I am using flink binary directly.

I am using this command to deploy the script.

                        "$FLINK_HOME/bin/standalone-job.sh" start-foreground --job-classname ${ARGS_FOR_JOB}

where ARGS_FOR_JOB contain job class name and all other necessary details needed by the job.

Best regards,

Jatin

On Fri, Nov 29, 2019 at 4:18 PM Chesnay Schepler <[hidden email]> wrote:

To clarify, you ran "mvn package -pl flink-dist -am" to build Fink?

If so, could you run that again and provide us with the maven output?




On 29/11/2019 11:23, Jatin Banger wrote:

Hi,

[hidden email]  I am using flink 1.8.1

I am using the following procedure for the deployment:

https://github.com/apache/flink/blob/master/flink-container/docker/README.md

And i tried accessing the path you mentioned:

# curl <pod-Ip>:4081/#/overview

{"errors":["Not found."]}

Best Regards,

Jatin 

On Thu, Nov 28, 2019 at 10:21 PM Chesnay Schepler <[hidden email]> wrote:

Could you try accessing <pid-Ip>:/#/overview ?

The REST API is obviously accessible, and hence the WebUI should be too.

How did you setup the session cluster? Are you using some custom Flink build or something, which potentially excluded flink-runtime-web from the classpath?

On 28/11/2019 10:02, Jatin Banger wrote:

Hi,

I checked the log file there is no error.

And I checked the pods internal ports by using rest api.

# curl <pod-Ip>: 4081

{"errors":["Not found."]}

4081 is the Ui port

# curl <pod-Ip>:4081/config

{"refresh-interval":3000,"timezone-name":"Coordinated Universal Time","timezone-offset":0,"flink-version":"<unknown>","flink-revision":"ceba8af @ 11.02.2019 @ 22:17:09 CST"}

# curl <pod-Ip>:4081/jobs

{"jobs":[{"id":"___job_Id_____","status":"RUNNING"}]}

Which shows the state of the job as running.

What else can we do ?

Best regards,

Jatin

On Thu, Nov 28, 2019 at 1:28 PM vino yang <[hidden email]> wrote:

Hi Jatin,

Flink web UI does not depend on any deployment mode.

You should check if there are error logs in the log file and the job status is running state.

Best,

Vino

Jatin Banger <[hidden email]> 于2019年11月28日周四 下午3:43写道：

Hi,

It seems there is Web Ui for Flink Session cluster, But for Flink Job Cluster it is Showing 

{"errors":["Not found."]}

Is it the expected behavior for Flink Job Cluster Mode ?

Best Regards,

Jatin

[Jatin Banger]

Hi,

I have checked the logs with this keyword  StaticFileServerHandler   in it, But there were no logs coming for "Flink Job Cluster".

Then i checked for Flink Session Cluster, i was able to find the logs for the StaticFileServerHandler keyword.

Can i raise this as bug ?

Best Regards,

Jatin

On Thu, Dec 5, 2019 at 8:59 PM Chesnay Schepler <[hidden email]> wrote:

Ok, it's good to know that the WebUI files are there.

Please enable DEBUG logging and try again, searching for messages from the StaticFileServerHandler.

This handler logs every file that is requested (which effectively happens when the WebUI is being served); let's see what is actually being requested.

On 05/12/2019 05:57, Jatin Banger wrote:

I have tried that already using 

'$FLINK_HOME/bin/jobmanager.sh" start-foreground

Ui comes fine with this one.

Which means web/index.html is present.

On Wed, Dec 4, 2019 at 9:01 PM Chesnay Schepler <[hidden email]> wrote:

hmm...this is quite odd.

Let's try to narrow things down a bit.

Could you try starting a local cluster (using the same distribution) and checking whether the UI is accessible?

Could you also check whether the flink-dist.jar in /lib contains web/index.html?

On 04/12/2019 06:02, Jatin Banger wrote:

Hi,

I am using flink binary directly.

I am using this command to deploy the script.

                        "$FLINK_HOME/bin/standalone-job.sh" start-foreground --job-classname ${ARGS_FOR_JOB}

where ARGS_FOR_JOB contain job class name and all other necessary details needed by the job.

Best regards,

Jatin

On Fri, Nov 29, 2019 at 4:18 PM Chesnay Schepler <[hidden email]> wrote:

To clarify, you ran "mvn package -pl flink-dist -am" to build Fink?

If so, could you run that again and provide us with the maven output?




On 29/11/2019 11:23, Jatin Banger wrote:

Hi,

[hidden email]  I am using flink 1.8.1

I am using the following procedure for the deployment:

https://github.com/apache/flink/blob/master/flink-container/docker/README.md

And i tried accessing the path you mentioned:

# curl <pod-Ip>:4081/#/overview

{"errors":["Not found."]}

Best Regards,

Jatin 

On Thu, Nov 28, 2019 at 10:21 PM Chesnay Schepler <[hidden email]> wrote:

Could you try accessing <pid-Ip>:/#/overview ?

The REST API is obviously accessible, and hence the WebUI should be too.

How did you setup the session cluster? Are you using some custom Flink build or something, which potentially excluded flink-runtime-web from the classpath?

On 28/11/2019 10:02, Jatin Banger wrote:

Hi,

I checked the log file there is no error.

And I checked the pods internal ports by using rest api.

# curl <pod-Ip>: 4081

{"errors":["Not found."]}

4081 is the Ui port

# curl <pod-Ip>:4081/config

{"refresh-interval":3000,"timezone-name":"Coordinated Universal Time","timezone-offset":0,"flink-version":"<unknown>","flink-revision":"ceba8af @ 11.02.2019 @ 22:17:09 CST"}

# curl <pod-Ip>:4081/jobs

{"jobs":[{"id":"___job_Id_____","status":"RUNNING"}]}

Which shows the state of the job as running.

What else can we do ?

Best regards,

Jatin

On Thu, Nov 28, 2019 at 1:28 PM vino yang <[hidden email]> wrote:

Hi Jatin,

Flink web UI does not depend on any deployment mode.

You should check if there are error logs in the log file and the job status is running state.

Best,

Vino

Jatin Banger <[hidden email]> 于2019年11月28日周四 下午3:43写道：

Hi,

It seems there is Web Ui for Flink Session cluster, But for Flink Job Cluster it is Showing 

{"errors":["Not found."]}

Is it the expected behavior for Flink Job Cluster Mode ?

Best Regards,

Jatin

[Arvid Heise-3]

Hi Jatin,

just to be sure. Did you increase the log level to debug [1] before checking for StaticFileServerHandler?

Best,

Arvid

[1] https://ci.apache.org/projects/flink/flink-docs-stable/monitoring/logging.html#configuring-log4j

On Mon, Dec 9, 2019 at 7:54 AM Jatin Banger <[hidden email]> wrote:

Hi,

I have checked the logs with this keyword  StaticFileServerHandler   in it, But there were no logs coming for "Flink Job Cluster".

Then i checked for Flink Session Cluster, i was able to find the logs for the StaticFileServerHandler keyword.

Can i raise this as bug ?

Best Regards,

Jatin

On Thu, Dec 5, 2019 at 8:59 PM Chesnay Schepler <[hidden email]> wrote:

Ok, it's good to know that the WebUI files are there.

Please enable DEBUG logging and try again, searching for messages from the StaticFileServerHandler.

This handler logs every file that is requested (which effectively happens when the WebUI is being served); let's see what is actually being requested.

On 05/12/2019 05:57, Jatin Banger wrote:

I have tried that already using 

'$FLINK_HOME/bin/jobmanager.sh" start-foreground

Ui comes fine with this one.

Which means web/index.html is present.

On Wed, Dec 4, 2019 at 9:01 PM Chesnay Schepler <[hidden email]> wrote:

hmm...this is quite odd.

Let's try to narrow things down a bit.

Could you try starting a local cluster (using the same distribution) and checking whether the UI is accessible?

Could you also check whether the flink-dist.jar in /lib contains web/index.html?

On 04/12/2019 06:02, Jatin Banger wrote:

Hi,

I am using flink binary directly.

I am using this command to deploy the script.

                        "$FLINK_HOME/bin/standalone-job.sh" start-foreground --job-classname ${ARGS_FOR_JOB}

where ARGS_FOR_JOB contain job class name and all other necessary details needed by the job.

Best regards,

Jatin

On Fri, Nov 29, 2019 at 4:18 PM Chesnay Schepler <[hidden email]> wrote:

To clarify, you ran "mvn package -pl flink-dist -am" to build Fink?

If so, could you run that again and provide us with the maven output?




On 29/11/2019 11:23, Jatin Banger wrote:

Hi,

[hidden email]  I am using flink 1.8.1

I am using the following procedure for the deployment:

https://github.com/apache/flink/blob/master/flink-container/docker/README.md

And i tried accessing the path you mentioned:

# curl <pod-Ip>:4081/#/overview

{"errors":["Not found."]}

Best Regards,

Jatin 

On Thu, Nov 28, 2019 at 10:21 PM Chesnay Schepler <[hidden email]> wrote:

Could you try accessing <pid-Ip>:/#/overview ?

The REST API is obviously accessible, and hence the WebUI should be too.

How did you setup the session cluster? Are you using some custom Flink build or something, which potentially excluded flink-runtime-web from the classpath?

On 28/11/2019 10:02, Jatin Banger wrote:

Hi,

I checked the log file there is no error.

And I checked the pods internal ports by using rest api.

# curl <pod-Ip>: 4081

{"errors":["Not found."]}

4081 is the Ui port

# curl <pod-Ip>:4081/config

{"refresh-interval":3000,"timezone-name":"Coordinated Universal Time","timezone-offset":0,"flink-version":"<unknown>","flink-revision":"ceba8af @ 11.02.2019 @ 22:17:09 CST"}

# curl <pod-Ip>:4081/jobs

{"jobs":[{"id":"___job_Id_____","status":"RUNNING"}]}

Which shows the state of the job as running.

What else can we do ?

Best regards,

Jatin

On Thu, Nov 28, 2019 at 1:28 PM vino yang <[hidden email]> wrote:

Hi Jatin,

Flink web UI does not depend on any deployment mode.

You should check if there are error logs in the log file and the job status is running state.

Best,

Vino

Jatin Banger <[hidden email]> 于2019年11月28日周四 下午3:43写道：

Hi,

It seems there is Web Ui for Flink Session cluster, But for Flink Job Cluster it is Showing 

{"errors":["Not found."]}

Is it the expected behavior for Flink Job Cluster Mode ?

Best Regards,

Jatin

[Jatin Banger]

Yes, I did.

On Tue, Dec 10, 2019 at 3:47 PM Arvid Heise <[hidden email]> wrote:

Hi Jatin,

just to be sure. Did you increase the log level to debug [1] before checking for StaticFileServerHandler?

Best,

Arvid

[1] https://ci.apache.org/projects/flink/flink-docs-stable/monitoring/logging.html#configuring-log4j

On Mon, Dec 9, 2019 at 7:54 AM Jatin Banger <[hidden email]> wrote:

Hi,

I have checked the logs with this keyword  StaticFileServerHandler   in it, But there were no logs coming for "Flink Job Cluster".

Then i checked for Flink Session Cluster, i was able to find the logs for the StaticFileServerHandler keyword.

Can i raise this as bug ?

Best Regards,

Jatin

On Thu, Dec 5, 2019 at 8:59 PM Chesnay Schepler <[hidden email]> wrote:

Ok, it's good to know that the WebUI files are there.

Please enable DEBUG logging and try again, searching for messages from the StaticFileServerHandler.

This handler logs every file that is requested (which effectively happens when the WebUI is being served); let's see what is actually being requested.

On 05/12/2019 05:57, Jatin Banger wrote:

I have tried that already using 

'$FLINK_HOME/bin/jobmanager.sh" start-foreground

Ui comes fine with this one.

Which means web/index.html is present.

On Wed, Dec 4, 2019 at 9:01 PM Chesnay Schepler <[hidden email]> wrote:

hmm...this is quite odd.

Let's try to narrow things down a bit.

Could you try starting a local cluster (using the same distribution) and checking whether the UI is accessible?

Could you also check whether the flink-dist.jar in /lib contains web/index.html?

On 04/12/2019 06:02, Jatin Banger wrote:

Hi,

I am using flink binary directly.

I am using this command to deploy the script.

                        "$FLINK_HOME/bin/standalone-job.sh" start-foreground --job-classname ${ARGS_FOR_JOB}

where ARGS_FOR_JOB contain job class name and all other necessary details needed by the job.

Best regards,

Jatin

On Fri, Nov 29, 2019 at 4:18 PM Chesnay Schepler <[hidden email]> wrote:

To clarify, you ran "mvn package -pl flink-dist -am" to build Fink?

If so, could you run that again and provide us with the maven output?




On 29/11/2019 11:23, Jatin Banger wrote:

Hi,

[hidden email]  I am using flink 1.8.1

I am using the following procedure for the deployment:

https://github.com/apache/flink/blob/master/flink-container/docker/README.md

And i tried accessing the path you mentioned:

# curl <pod-Ip>:4081/#/overview

{"errors":["Not found."]}

Best Regards,

Jatin 

On Thu, Nov 28, 2019 at 10:21 PM Chesnay Schepler <[hidden email]> wrote:

Could you try accessing <pid-Ip>:/#/overview ?

The REST API is obviously accessible, and hence the WebUI should be too.

How did you setup the session cluster? Are you using some custom Flink build or something, which potentially excluded flink-runtime-web from the classpath?

On 28/11/2019 10:02, Jatin Banger wrote:

Hi,

I checked the log file there is no error.

And I checked the pods internal ports by using rest api.

# curl <pod-Ip>: 4081

{"errors":["Not found."]}

4081 is the Ui port

# curl <pod-Ip>:4081/config

{"refresh-interval":3000,"timezone-name":"Coordinated Universal Time","timezone-offset":0,"flink-version":"<unknown>","flink-revision":"ceba8af @ 11.02.2019 @ 22:17:09 CST"}

# curl <pod-Ip>:4081/jobs

{"jobs":[{"id":"___job_Id_____","status":"RUNNING"}]}

Which shows the state of the job as running.

What else can we do ?

Best regards,

Jatin

On Thu, Nov 28, 2019 at 1:28 PM vino yang <[hidden email]> wrote:

Hi Jatin,

Flink web UI does not depend on any deployment mode.

You should check if there are error logs in the log file and the job status is running state.

Best,

Vino

Jatin Banger <[hidden email]> 于2019年11月28日周四 下午3:43写道：

Hi,

It seems there is Web Ui for Flink Session cluster, But for Flink Job Cluster it is Showing 

{"errors":["Not found."]}

Is it the expected behavior for Flink Job Cluster Mode ?

Best Regards,

Jatin

[Chesnay Schepler]

Would it be possible for you to provide us with full debug log file?

On 10/12/2019 18:07, Jatin Banger wrote:

Yes, I did.

On Tue, Dec 10, 2019 at 3:47 PM Arvid Heise <[hidden email]> wrote:

Hi Jatin,

just to be sure. Did you increase the log level to debug [1] before checking for StaticFileServerHandler?

Best,

Arvid

[1] https://ci.apache.org/projects/flink/flink-docs-stable/monitoring/logging.html#configuring-log4j

On Mon, Dec 9, 2019 at 7:54 AM Jatin Banger <[hidden email]> wrote:

Hi,

I have checked the logs with this keyword  StaticFileServerHandler   in it, But there were no logs coming for "Flink Job Cluster".

Then i checked for Flink Session Cluster, i was able to find the logs for the StaticFileServerHandler keyword.

Can i raise this as bug ?

Best Regards,

Jatin

On Thu, Dec 5, 2019 at 8:59 PM Chesnay Schepler <[hidden email]> wrote:

Ok, it's good to know that the WebUI files are there.

Please enable DEBUG logging and try again, searching for messages from the StaticFileServerHandler.

This handler logs every file that is requested (which effectively happens when the WebUI is being served); let's see what is actually being requested.

On 05/12/2019 05:57, Jatin Banger wrote:

I have tried that already using 

'$FLINK_HOME/bin/jobmanager.sh" start-foreground

Ui comes fine with this one.

Which means web/index.html is present.

On Wed, Dec 4, 2019 at 9:01 PM Chesnay Schepler <[hidden email]> wrote:

hmm...this is quite odd.

Let's try to narrow things down a bit.

Could you try starting a local cluster (using the same distribution) and checking whether the UI is accessible?

Could you also check whether the flink-dist.jar in /lib contains web/index.html?

On 04/12/2019 06:02, Jatin Banger wrote:

Hi,

I am using flink binary directly.

I am using this command to deploy the script.

                        "$FLINK_HOME/bin/standalone-job.sh" start-foreground --job-classname ${ARGS_FOR_JOB}

where ARGS_FOR_JOB contain job class name and all other necessary details needed by the job.

Best regards,

Jatin

On Fri, Nov 29, 2019 at 4:18 PM Chesnay Schepler <[hidden email]> wrote:

To clarify, you ran "mvn package -pl flink-dist -am" to build Fink?

If so, could you run that again and provide us with the maven output?




On 29/11/2019 11:23, Jatin Banger wrote:

Hi,

[hidden email]  I am using flink 1.8.1

I am using the following procedure for the deployment:

https://github.com/apache/flink/blob/master/flink-container/docker/README.md

And i tried accessing the path you mentioned:

# curl <pod-Ip>:4081/#/overview

{"errors":["Not found."]}

Best Regards,

Jatin 

On Thu, Nov 28, 2019 at 10:21 PM Chesnay Schepler <[hidden email]> wrote:

Could you try accessing <pid-Ip>:/#/overview ?

The REST API is obviously accessible, and hence the WebUI should be too.

How did you setup the session cluster? Are you using some custom Flink build or something, which potentially excluded flink-runtime-web from the classpath?

On 28/11/2019 10:02, Jatin Banger wrote:

Hi,

I checked the log file there is no error.

And I checked the pods internal ports by using rest api.

# curl <pod-Ip>: 4081

{"errors":["Not found."]}

4081 is the Ui port

# curl <pod-Ip>:4081/config

{"refresh-interval":3000,"timezone-name":"Coordinated Universal Time","timezone-offset":0,"flink-version":"<unknown>","flink-revision":"ceba8af @ 11.02.2019 @ 22:17:09 CST"}

# curl <pod-Ip>:4081/jobs

{"jobs":[{"id":"___job_Id_____","status":"RUNNING"}]}

Which shows the state of the job as running.

What else can we do ?

Best regards,

Jatin

On Thu, Nov 28, 2019 at 1:28 PM vino yang <[hidden email]> wrote:

Hi Jatin,

Flink web UI does not depend on any deployment mode.

You should check if there are error logs in the log file and the job status is running state.

Best,

Vino

Jatin Banger <[hidden email]> 于2019年11月28日周四 下午3:43写道：

Hi,

It seems there is Web Ui for Flink Session cluster, But for Flink Job Cluster it is Showing 

{"errors":["Not found."]}

Is it the expected behavior for Flink Job Cluster Mode ?

Best Regards,

Jatin

[Jatin Banger]

Sure, here it is. 

Job Manager Logs with logging level as DEBUG

On Wed, Dec 11, 2019 at 3:14 PM Chesnay Schepler <[hidden email]> wrote:

Would it be possible for you to provide us with full debug log file?

On 10/12/2019 18:07, Jatin Banger wrote:

Yes, I did.

On Tue, Dec 10, 2019 at 3:47 PM Arvid Heise <[hidden email]> wrote:

Hi Jatin,

just to be sure. Did you increase the log level to debug [1] before checking for StaticFileServerHandler?

Best,

Arvid

[1] https://ci.apache.org/projects/flink/flink-docs-stable/monitoring/logging.html#configuring-log4j

On Mon, Dec 9, 2019 at 7:54 AM Jatin Banger <[hidden email]> wrote:

Hi,

I have checked the logs with this keyword  StaticFileServerHandler   in it, But there were no logs coming for "Flink Job Cluster".

Then i checked for Flink Session Cluster, i was able to find the logs for the StaticFileServerHandler keyword.

Can i raise this as bug ?

Best Regards,

Jatin

On Thu, Dec 5, 2019 at 8:59 PM Chesnay Schepler <[hidden email]> wrote:

Ok, it's good to know that the WebUI files are there.

Please enable DEBUG logging and try again, searching for messages from the StaticFileServerHandler.

This handler logs every file that is requested (which effectively happens when the WebUI is being served); let's see what is actually being requested.

On 05/12/2019 05:57, Jatin Banger wrote:

I have tried that already using 

'$FLINK_HOME/bin/jobmanager.sh" start-foreground

Ui comes fine with this one.

Which means web/index.html is present.

On Wed, Dec 4, 2019 at 9:01 PM Chesnay Schepler <[hidden email]> wrote:

hmm...this is quite odd.

Let's try to narrow things down a bit.

Could you try starting a local cluster (using the same distribution) and checking whether the UI is accessible?

Could you also check whether the flink-dist.jar in /lib contains web/index.html?

On 04/12/2019 06:02, Jatin Banger wrote:

Hi,

I am using flink binary directly.

I am using this command to deploy the script.

                        "$FLINK_HOME/bin/standalone-job.sh" start-foreground --job-classname ${ARGS_FOR_JOB}

where ARGS_FOR_JOB contain job class name and all other necessary details needed by the job.

Best regards,

Jatin

On Fri, Nov 29, 2019 at 4:18 PM Chesnay Schepler <[hidden email]> wrote:

To clarify, you ran "mvn package -pl flink-dist -am" to build Fink?

If so, could you run that again and provide us with the maven output?




On 29/11/2019 11:23, Jatin Banger wrote:

Hi,

[hidden email]  I am using flink 1.8.1

I am using the following procedure for the deployment:

https://github.com/apache/flink/blob/master/flink-container/docker/README.md

And i tried accessing the path you mentioned:

# curl <pod-Ip>:4081/#/overview

{"errors":["Not found."]}

Best Regards,

Jatin 

On Thu, Nov 28, 2019 at 10:21 PM Chesnay Schepler <[hidden email]> wrote:

Could you try accessing <pid-Ip>:/#/overview ?

The REST API is obviously accessible, and hence the WebUI should be too.

How did you setup the session cluster? Are you using some custom Flink build or something, which potentially excluded flink-runtime-web from the classpath?

On 28/11/2019 10:02, Jatin Banger wrote:

Hi,

I checked the log file there is no error.

And I checked the pods internal ports by using rest api.

# curl <pod-Ip>: 4081

{"errors":["Not found."]}

4081 is the Ui port

# curl <pod-Ip>:4081/config

{"refresh-interval":3000,"timezone-name":"Coordinated Universal Time","timezone-offset":0,"flink-version":"<unknown>","flink-revision":"ceba8af @ 11.02.2019 @ 22:17:09 CST"}

# curl <pod-Ip>:4081/jobs

{"jobs":[{"id":"___job_Id_____","status":"RUNNING"}]}

Which shows the state of the job as running.

What else can we do ?

Best regards,

Jatin

On Thu, Nov 28, 2019 at 1:28 PM vino yang <[hidden email]> wrote:

Hi Jatin,

Flink web UI does not depend on any deployment mode.

You should check if there are error logs in the log file and the job status is running state.

Best,

Vino

Jatin Banger <[hidden email]> 于2019年11月28日周四 下午3:43写道：

Hi,

It seems there is Web Ui for Flink Session cluster, But for Flink Job Cluster it is Showing 

{"errors":["Not found."]}

Is it the expected behavior for Flink Job Cluster Mode ?

Best Regards,

Jatin

[Chesnay Schepler]

Thank you for the logs.

Flink can indeed find the WebUI files in the distribution, which is a bit odd.

Since there are no static files serve in this case, the StaticFileServerHandler is never set up in the first place (hence why we didn't find any log statements).

What I also found in the logs (and, looking back, in one of your earlier replies) was this: Version: <unknown>, Rev:ceba8af, Date:11.02.2019 @ 22:17:09 CST

This tells us a few things.

a) You are not using 1.8.1, but 1.7.2 (based on the revision)

b) You are not using an official release, since the build-date differs from the official releases

I tried one of the official 1.7.2 releases, and the WebUI is shown both when using:

(after copying the wordcount example into /lib)

./bin/standalone-job.sh start-foreground -j org.apache.flink.examples.java.wordcount.WordCount

./bin/standalone-job.sh start -j org.apache.flink.examples.java.wordcount.WordCount

Right now I don't know what else to look for; there are some discrepancies as to what your environment is vs what you described, and as such I can only recommend to carefully evaluate what you have actually running and possibly try again with an official release.

Regards,

Chesnay

On 13/12/2019 09:58, Jatin Banger wrote:

Sure, here it is. 

Job Manager Logs with logging level as DEBUG

On Wed, Dec 11, 2019 at 3:14 PM Chesnay Schepler <[hidden email]> wrote:

Would it be possible for you to provide us with full debug log file?

On 10/12/2019 18:07, Jatin Banger wrote:

Yes, I did.

On Tue, Dec 10, 2019 at 3:47 PM Arvid Heise <[hidden email]> wrote:

Hi Jatin,

just to be sure. Did you increase the log level to debug [1] before checking for StaticFileServerHandler?

Best,

Arvid

[1] https://ci.apache.org/projects/flink/flink-docs-stable/monitoring/logging.html#configuring-log4j

On Mon, Dec 9, 2019 at 7:54 AM Jatin Banger <[hidden email]> wrote:

Hi,

I have checked the logs with this keyword  StaticFileServerHandler   in it, But there were no logs coming for "Flink Job Cluster".

Then i checked for Flink Session Cluster, i was able to find the logs for the StaticFileServerHandler keyword.

Can i raise this as bug ?

Best Regards,

Jatin

On Thu, Dec 5, 2019 at 8:59 PM Chesnay Schepler <[hidden email]> wrote:

Ok, it's good to know that the WebUI files are there.

Please enable DEBUG logging and try again, searching for messages from the StaticFileServerHandler.

This handler logs every file that is requested (which effectively happens when the WebUI is being served); let's see what is actually being requested.

On 05/12/2019 05:57, Jatin Banger wrote:

I have tried that already using 

'$FLINK_HOME/bin/jobmanager.sh" start-foreground

Ui comes fine with this one.

Which means web/index.html is present.

On Wed, Dec 4, 2019 at 9:01 PM Chesnay Schepler <[hidden email]> wrote:

hmm...this is quite odd.

Let's try to narrow things down a bit.

Could you try starting a local cluster (using the same distribution) and checking whether the UI is accessible?

Could you also check whether the flink-dist.jar in /lib contains web/index.html?

On 04/12/2019 06:02, Jatin Banger wrote:

Hi,

I am using flink binary directly.

I am using this command to deploy the script.

                        "$FLINK_HOME/bin/standalone-job.sh" start-foreground --job-classname ${ARGS_FOR_JOB}

where ARGS_FOR_JOB contain job class name and all other necessary details needed by the job.

Best regards,

Jatin

On Fri, Nov 29, 2019 at 4:18 PM Chesnay Schepler <[hidden email]> wrote:

To clarify, you ran "mvn package -pl flink-dist -am" to build Fink?

If so, could you run that again and provide us with the maven output?




On 29/11/2019 11:23, Jatin Banger wrote:

Hi,

[hidden email]  I am using flink 1.8.1

I am using the following procedure for the deployment:

https://github.com/apache/flink/blob/master/flink-container/docker/README.md

And i tried accessing the path you mentioned:

# curl <pod-Ip>:4081/#/overview

{"errors":["Not found."]}

Best Regards,

Jatin 

On Thu, Nov 28, 2019 at 10:21 PM Chesnay Schepler <[hidden email]> wrote:

Could you try accessing <pid-Ip>:/#/overview ?

The REST API is obviously accessible, and hence the WebUI should be too.

How did you setup the session cluster? Are you using some custom Flink build or something, which potentially excluded flink-runtime-web from the classpath?

On 28/11/2019 10:02, Jatin Banger wrote:

Hi,

I checked the log file there is no error.

And I checked the pods internal ports by using rest api.

# curl <pod-Ip>: 4081

{"errors":["Not found."]}

4081 is the Ui port

# curl <pod-Ip>:4081/config

{"refresh-interval":3000,"timezone-name":"Coordinated Universal Time","timezone-offset":0,"flink-version":"<unknown>","flink-revision":"ceba8af @ 11.02.2019 @ 22:17:09 CST"}

# curl <pod-Ip>:4081/jobs

{"jobs":[{"id":"___job_Id_____","status":"RUNNING"}]}

Which shows the state of the job as running.

What else can we do ?

Best regards,

Jatin

On Thu, Nov 28, 2019 at 1:28 PM vino yang <[hidden email]> wrote:

Hi Jatin,

Flink web UI does not depend on any deployment mode.

You should check if there are error logs in the log file and the job status is running state.

Best,

Vino

Jatin Banger <[hidden email]> 于2019年11月28日周四 下午3:43写道：

Hi,

It seems there is Web Ui for Flink Session cluster, But for Flink Job Cluster it is Showing 

{"errors":["Not found."]}

Is it the expected behavior for Flink Job Cluster Mode ?

Best Regards,

Jatin

[Jatin Banger]

Hi,

Recently i upgraded flink version to 1.8.3

For Session cluster it shows the version correctly.

But for job cluster. I get this in the logs 

Starting StandaloneJobClusterEntryPoint (Version: <unknown>, Rev:6322618, Date:04.09.2019 @ 22:07:41 CST)

And my Classpath has these jars:

Classpath: /opt/flink/lib/flink-metrics-prometheus-1.8.3.jar:/opt/flink/lib/flink-shaded-hadoop-2-uber-2.6.5-7.0.jar:/opt/flink/lib/job.jar:/opt/flink/lib/log4j-over-slf4j-1.7.28.jar:/opt/flink/lib/logback-classic-1.2.3.jar:/opt/flink/lib/logback-core-1.2.3.jar:/opt/flink/lib/flink-dist_2.11-1.8.3.jar::/opt/hdfs:

Do you have any idea what could have caused this?

Best Regards,

Jatin

On Fri, Dec 13, 2019 at 6:18 PM Chesnay Schepler <[hidden email]> wrote:

Thank you for the logs.

Flink can indeed find the WebUI files in the distribution, which is a bit odd.

Since there are no static files serve in this case, the StaticFileServerHandler is never set up in the first place (hence why we didn't find any log statements).

What I also found in the logs (and, looking back, in one of your earlier replies) was this: Version: <unknown>, Rev:ceba8af, Date:11.02.2019 @ 22:17:09 CST

This tells us a few things.

a) You are not using 1.8.1, but 1.7.2 (based on the revision)

b) You are not using an official release, since the build-date differs from the official releases

I tried one of the official 1.7.2 releases, and the WebUI is shown both when using:

(after copying the wordcount example into /lib)

./bin/standalone-job.sh start-foreground -j org.apache.flink.examples.java.wordcount.WordCount

./bin/standalone-job.sh start -j org.apache.flink.examples.java.wordcount.WordCount

Right now I don't know what else to look for; there are some discrepancies as to what your environment is vs what you described, and as such I can only recommend to carefully evaluate what you have actually running and possibly try again with an official release.

Regards,

Chesnay

On 13/12/2019 09:58, Jatin Banger wrote:

Sure, here it is. 

Job Manager Logs with logging level as DEBUG

On Wed, Dec 11, 2019 at 3:14 PM Chesnay Schepler <[hidden email]> wrote:

Would it be possible for you to provide us with full debug log file?

On 10/12/2019 18:07, Jatin Banger wrote:

Yes, I did.

On Tue, Dec 10, 2019 at 3:47 PM Arvid Heise <[hidden email]> wrote:

Hi Jatin,

just to be sure. Did you increase the log level to debug [1] before checking for StaticFileServerHandler?

Best,

Arvid

[1] https://ci.apache.org/projects/flink/flink-docs-stable/monitoring/logging.html#configuring-log4j

On Mon, Dec 9, 2019 at 7:54 AM Jatin Banger <[hidden email]> wrote:

Hi,

I have checked the logs with this keyword  StaticFileServerHandler   in it, But there were no logs coming for "Flink Job Cluster".

Then i checked for Flink Session Cluster, i was able to find the logs for the StaticFileServerHandler keyword.

Can i raise this as bug ?

Best Regards,

Jatin

On Thu, Dec 5, 2019 at 8:59 PM Chesnay Schepler <[hidden email]> wrote:

Ok, it's good to know that the WebUI files are there.

Please enable DEBUG logging and try again, searching for messages from the StaticFileServerHandler.

This handler logs every file that is requested (which effectively happens when the WebUI is being served); let's see what is actually being requested.

On 05/12/2019 05:57, Jatin Banger wrote:

I have tried that already using 

'$FLINK_HOME/bin/jobmanager.sh" start-foreground

Ui comes fine with this one.

Which means web/index.html is present.

On Wed, Dec 4, 2019 at 9:01 PM Chesnay Schepler <[hidden email]> wrote:

hmm...this is quite odd.

Let's try to narrow things down a bit.

Could you try starting a local cluster (using the same distribution) and checking whether the UI is accessible?

Could you also check whether the flink-dist.jar in /lib contains web/index.html?

On 04/12/2019 06:02, Jatin Banger wrote:

Hi,

I am using flink binary directly.

I am using this command to deploy the script.

                        "$FLINK_HOME/bin/standalone-job.sh" start-foreground --job-classname ${ARGS_FOR_JOB}

where ARGS_FOR_JOB contain job class name and all other necessary details needed by the job.

Best regards,

Jatin

On Fri, Nov 29, 2019 at 4:18 PM Chesnay Schepler <[hidden email]> wrote:

To clarify, you ran "mvn package -pl flink-dist -am" to build Fink?

If so, could you run that again and provide us with the maven output?




On 29/11/2019 11:23, Jatin Banger wrote:

Hi,

[hidden email]  I am using flink 1.8.1

I am using the following procedure for the deployment:

https://github.com/apache/flink/blob/master/flink-container/docker/README.md

And i tried accessing the path you mentioned:

# curl <pod-Ip>:4081/#/overview

{"errors":["Not found."]}

Best Regards,

Jatin 

On Thu, Nov 28, 2019 at 10:21 PM Chesnay Schepler <[hidden email]> wrote:

Could you try accessing <pid-Ip>:/#/overview ?

The REST API is obviously accessible, and hence the WebUI should be too.

How did you setup the session cluster? Are you using some custom Flink build or something, which potentially excluded flink-runtime-web from the classpath?

On 28/11/2019 10:02, Jatin Banger wrote:

Hi,

I checked the log file there is no error.

And I checked the pods internal ports by using rest api.

# curl <pod-Ip>: 4081

{"errors":["Not found."]}

4081 is the Ui port

# curl <pod-Ip>:4081/config

{"refresh-interval":3000,"timezone-name":"Coordinated Universal Time","timezone-offset":0,"flink-version":"<unknown>","flink-revision":"ceba8af @ 11.02.2019 @ 22:17:09 CST"}

# curl <pod-Ip>:4081/jobs

{"jobs":[{"id":"___job_Id_____","status":"RUNNING"}]}

Which shows the state of the job as running.

What else can we do ?

Best regards,

Jatin

On Thu, Nov 28, 2019 at 1:28 PM vino yang <[hidden email]> wrote:

Hi Jatin,

Flink web UI does not depend on any deployment mode.

You should check if there are error logs in the log file and the job status is running state.

Best,

Vino

Jatin Banger <[hidden email]> 于2019年11月28日周四 下午3:43写道：

Hi,

It seems there is Web Ui for Flink Session cluster, But for Flink Job Cluster it is Showing 

{"errors":["Not found."]}

Is it the expected behavior for Flink Job Cluster Mode ?

Best Regards,

Jatin