Wondering how to check if taskmanager is registered with the jobmanager, without asking job manager

[Bariša]

As part of deploying task managers and job managers, I'd like to expose healthcheck on both task managers and job managers.

For the task managers, one of the requirements that they are healthy, is that they have successfully registered themselves with the job manager.

Is there a way to achieve this, without making a call to job manager ( to do that, I first need to make a call to the zookeeper to find the job manager, so I'm trying to simplify the health check ).

Ideally, taskmanager would have a metric that says, ( am registered ), but afaik, that doesn't exist https://ci.apache.org/projects/flink/flink-docs-stable/monitoring/metrics.html#cluster

P.S.

This is my first post in the email list, happy to update/change my question, if I messed up, or misunderstood something.

Cheers,

Barisa

[Barisa Obradovic]

Version of flink I'm using is 1.6.1, if that helps.



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[Piotr Nowojski]

Hi,

I don’t think that’s exposed on the TaskManager.

Maybe it would simplify things a bit if you implement this as a single “JobManager” health check, not multiple TaskManagers health check - for example verify that there are expected number of registered TaskManagers. It might cover your case.

Piotrek

On 9 Oct 2018, at 12:21, Bariša <[hidden email]> wrote:

As part of deploying task managers and job managers, I'd like to expose healthcheck on both task managers and job managers.

For the task managers, one of the requirements that they are healthy, is that they have successfully registered themselves with the job manager.

Is there a way to achieve this, without making a call to job manager ( to do that, I first need to make a call to the zookeeper to find the job manager, so I'm trying to simplify the health check ).

Ideally, taskmanager would have a metric that says, ( am registered ), but afaik, that doesn't exist https://ci.apache.org/projects/flink/flink-docs-stable/monitoring/metrics.html#cluster

P.S.

This is my first post in the email list, happy to update/change my question, if I messed up, or misunderstood something.

Cheers,

Barisa

[Bariša]

Thnx Piotr. I agree, that would work. It's a bit chicken and the egg problem, since at that point we can't just spin up a task manager, and have it register itself, we need to have flinkmanager know how many task managers should be there. Bit more logic, but doable. Thnx for the tip.

Cheers,
Barisa

On Wed, 10 Oct 2018 at 09:05, Piotr Nowojski <[hidden email]> wrote:

Hi,

I don’t think that’s exposed on the TaskManager.

Maybe it would simplify things a bit if you implement this as a single “JobManager” health check, not multiple TaskManagers health check - for example verify that there are expected number of registered TaskManagers. It might cover your case.

Piotrek

On 9 Oct 2018, at 12:21, Bariša <[hidden email]> wrote:

As part of deploying task managers and job managers, I'd like to expose healthcheck on both task managers and job managers.

For the task managers, one of the requirements that they are healthy, is that they have successfully registered themselves with the job manager.

Is there a way to achieve this, without making a call to job manager ( to do that, I first need to make a call to the zookeeper to find the job manager, so I'm trying to simplify the health check ).

Ideally, taskmanager would have a metric that says, ( am registered ), but afaik, that doesn't exist https://ci.apache.org/projects/flink/flink-docs-stable/monitoring/metrics.html#cluster

P.S.

This is my first post in the email list, happy to update/change my question, if I messed up, or misunderstood something.

Cheers,

Barisa

[Piotr Nowojski]

You’re welcome :)

On 10 Oct 2018, at 10:28, Bariša <[hidden email]> wrote:

Thnx Piotr. I agree, that would work. It's a bit chicken and the egg problem, since at that point we can't just spin up a task manager, and have it register itself, we need to have flinkmanager know how many task managers should be there. Bit more logic, but doable. Thnx for the tip.

Cheers,
Barisa

On Wed, 10 Oct 2018 at 09:05, Piotr Nowojski <[hidden email]> wrote:

Hi,

I don’t think that’s exposed on the TaskManager.

Maybe it would simplify things a bit if you implement this as a single “JobManager” health check, not multiple TaskManagers health check - for example verify that there are expected number of registered TaskManagers. It might cover your case.

Piotrek

On 9 Oct 2018, at 12:21, Bariša <[hidden email]> wrote:

As part of deploying task managers and job managers, I'd like to expose healthcheck on both task managers and job managers.

For the task managers, one of the requirements that they are healthy, is that they have successfully registered themselves with the job manager.

Is there a way to achieve this, without making a call to job manager ( to do that, I first need to make a call to the zookeeper to find the job manager, so I'm trying to simplify the health check ).

Ideally, taskmanager would have a metric that says, ( am registered ), but afaik, that doesn't exist https://ci.apache.org/projects/flink/flink-docs-stable/monitoring/metrics.html#cluster

P.S.

This is my first post in the email list, happy to update/change my question, if I messed up, or misunderstood something.

Cheers,

Barisa