flink JPS result changes

[Will Du]

Hi folks
Do we have any information about the process changes after v1.5.0? I used to see jobManager and TaskManager process once the start-cluster.sh is being called. But, it shows below in v1.5.0 once started. Everything works, but no idea where is the jobManager.

$jps
2523 TaskManagerRunner
2190 StandaloneSessionClusterEntrypoint

thanks,
Will

[miki haiat]

Flink 6 changed  the execution model compactly 
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=65147077 

https://docs.google.com/document/d/1zwBP3LKnJ0LI1R4-9hLXBVjOXAkQS5jKrGYYoPz-xRk/edit#heading=h.giuwq6q8d23j

 

On Wed, Jul 11, 2018 at 5:09 PM Will Du <[hidden email]> wrote:

Hi folks
Do we have any information about the process changes after v1.5.0? I used to see jobManager and TaskManager process once the start-cluster.sh is being called. But, it shows below in v1.5.0 once started. Everything works, but no idea where is the jobManager.

$jps
2523 TaskManagerRunner
2190 StandaloneSessionClusterEntrypoint

thanks,
Will

[Will Du]

In this case, do i need to add a jobManager

On Jul 11, 2018, at 10:14 AM, miki haiat <[hidden email]> wrote:

Flink 6 changed  the execution model compactly 
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=65147077 

https://docs.google.com/document/d/1zwBP3LKnJ0LI1R4-9hLXBVjOXAkQS5jKrGYYoPz-xRk/edit#heading=h.giuwq6q8d23j

 

On Wed, Jul 11, 2018 at 5:09 PM Will Du <[hidden email]> wrote:

Hi folks
Do we have any information about the process changes after v1.5.0? I used to see jobManager and TaskManager process once the start-cluster.sh is being called. But, it shows below in v1.5.0 once started. Everything works, but no idea where is the jobManager.

$jps
2523 TaskManagerRunner
2190 StandaloneSessionClusterEntrypoint

thanks,
Will

[Chesnay Schepler]

Generally speaking no, the DIspatcher (here called StandaloneSessionClusterEntrypoint) will spawn a jobmanager internally when a job is submitted

On 11.07.2018 16:42, Will Du wrote:

In this case, do i need to add a jobManager

On Jul 11, 2018, at 10:14 AM, miki haiat <[hidden email]> wrote:

Flink 6 changed  the execution model compactly 
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=65147077 

https://docs.google.com/document/d/1zwBP3LKnJ0LI1R4-9hLXBVjOXAkQS5jKrGYYoPz-xRk/edit#heading=h.giuwq6q8d23j

 

On Wed, Jul 11, 2018 at 5:09 PM Will Du <[hidden email]> wrote:

Hi folks
Do we have any information about the process changes after v1.5.0? I used to see jobManager and TaskManager process once the start-cluster.sh is being called. But, it shows below in v1.5.0 once started. Everything works, but no idea where is the jobManager.

$jps
2523 TaskManagerRunner
2190 StandaloneSessionClusterEntrypoint

thanks,
Will

[Will Du]

Hi folks,

I have program working in FLink 1.3 and tested working in v1.5.0 command line too. However, it has no response in web UI or rest API.

The rest call to submit job has no response. All the other rest part working, such as list of jars. What are the possible reason here?

The only thing I set in flink-conf.yaml is to make rest.port: 8001, then start-cluster.sh

See it works in command line in Flink 1.5.0

But it is hanging there in web UI.

thanks

Will

[Gary Yao-2]

Hi Will,

Is the HTTP request hanging, or is the Web UI not able to handle the error? You
can inspect the HTTP requests and responses using your browser's developer tools
[1]. When submitting a job, the Web UI issues a POST request to
/jars/<jarid>/run. Also jobmanager.log might give some hints.

Best,
Gary

[1] https://developer.mozilla.org/en-US/docs/Tools/Network_Monitor

On Wed, Jul 11, 2018 at 5:39 PM, Will Du <[hidden email]> wrote:

Hi folks,

I have program working in FLink 1.3 and tested working in v1.5.0 command line too. However, it has no response in web UI or rest API.

The rest call to submit job has no response. All the other rest part working, such as list of jars. What are the possible reason here?

The only thing I set in flink-conf.yaml is to make rest.port: 8001, then start-cluster.sh

See it works in command line in Flink 1.5.0

But it is hanging there in web UI.

thanks

Will

[Will Du]

Thank you for the response. The issue is caused by rest API use comma , as separator which breaks the code. Also, single quote is somehow removed by rest call process.

Sent from my iPhone

will

On Jul 12, 2018, at 01:05, Gary Yao <[hidden email]> wrote:

Hi Will,

Is the HTTP request hanging, or is the Web UI not able to handle the error? You
can inspect the HTTP requests and responses using your browser's developer tools
[1]. When submitting a job, the Web UI issues a POST request to
/jars/<jarid>/run. Also jobmanager.log might give some hints.

Best,
Gary

[1] https://developer.mozilla.org/en-US/docs/Tools/Network_Monitor

On Wed, Jul 11, 2018 at 5:39 PM, Will Du <[hidden email]> wrote:

Hi folks,

I have program working in FLink 1.3 and tested working in v1.5.0 command line too. However, it has no response in web UI or rest API.

The rest call to submit job has no response. All the other rest part working, such as list of jars. What are the possible reason here?

The only thing I set in flink-conf.yaml is to make rest.port: 8001, then start-cluster.sh

See it works in command line in Flink 1.5.0

<screenshot.png>

But it is hanging there in web UI.

<p1928.png>

thanks

Will