Can not stop cluster gracefully

[Andrew Ge Wu]

Hi all,


You may hit this problem before, from time to time when i ran the stop-cluster script, I get this

> No taskmanager daemon to stop on host app25
> No taskmanager daemon to stop on host app26
> No taskmanager daemon to stop on host app27
> No taskmanager daemon to stop on host app83
> No taskmanager daemon to stop on host app84
> No taskmanager daemon to stop on host app85
> No taskmanager daemon to stop on host app86
> No taskmanager daemon to stop on host app87
> No jobmanager daemon to stop on host app23
> No jobmanager daemon to stop on host app24

But my cluster is actually running…
Without tool the only choice I have is to terminate those processes on each server manually.

Anyone know if this is a setup issue? or a way to solve this?


Thanks!



Andrew


--
Confidentiality Notice: This e-mail transmission may contain confidential
or legally privileged information that is intended only for the individual
or entity named in the e-mail address. If you are not the intended
recipient, you are hereby notified that any disclosure, copying,
distribution, or reliance upon the contents of this e-mail is strictly
prohibited and may be unlawful. If you have received this e-mail in error,
please notify the sender immediately by return e-mail and delete all copies
of this message.

[Maximilian Michels]

The stop script relies on a file in the /tmp directory (location can
be changed by setting env.pid.dir in the Flink config). If that file
somehow gets cleanup up occasionally, the stop script can't find the
process identifiers inside that file to kill the processes.

Another explanation could be that you're running the stop script from
a different user. The file contains the user name of the user who
started the cluster and the stop script looks for the same name again.

-Max


On Tue, Nov 22, 2016 at 11:27 AM, Andrew Ge Wu <[hidden email]> wrote:

> Hi all,
>
>
> You may hit this problem before, from time to time when i ran the stop-cluster script, I get this
>
>> No taskmanager daemon to stop on host app25
>> No taskmanager daemon to stop on host app26
>> No taskmanager daemon to stop on host app27
>> No taskmanager daemon to stop on host app83
>> No taskmanager daemon to stop on host app84
>> No taskmanager daemon to stop on host app85
>> No taskmanager daemon to stop on host app86
>> No taskmanager daemon to stop on host app87
>> No jobmanager daemon to stop on host app23
>> No jobmanager daemon to stop on host app24
>
>
> But my cluster is actually running…
> Without tool the only choice I have is to terminate those processes on each server manually.
>
> Anyone know if this is a setup issue? or a way to solve this?
>
>
> Thanks!
>
>
>
> Andrew
>
>
> --
> Confidentiality Notice: This e-mail transmission may contain confidential
> or legally privileged information that is intended only for the individual
> or entity named in the e-mail address. If you are not the intended
> recipient, you are hereby notified that any disclosure, copying,
> distribution, or reliance upon the contents of this e-mail is strictly
> prohibited and may be unlawful. If you have received this e-mail in error,
> please notify the sender immediately by return e-mail and delete all copies
> of this message.

[Andrew Ge Wu]

Thanks, I’ll give that a try.

> On 22 Nov 2016, at 12:18, Maximilian Michels <[hidden email]> wrote:
>
> The stop script relies on a file in the /tmp directory (location can
> be changed by setting env.pid.dir in the Flink config). If that file
> somehow gets cleanup up occasionally, the stop script can't find the
> process identifiers inside that file to kill the processes.
>
> Another explanation could be that you're running the stop script from
> a different user. The file contains the user name of the user who
> started the cluster and the stop script looks for the same name again.
>
> -Max
>
>
> On Tue, Nov 22, 2016 at 11:27 AM, Andrew Ge Wu <[hidden email]> wrote:
>> Hi all,
>>
>>
>> You may hit this problem before, from time to time when i ran the stop-cluster script, I get this
>>
>>> No taskmanager daemon to stop on host app25
>>> No taskmanager daemon to stop on host app26
>>> No taskmanager daemon to stop on host app27
>>> No taskmanager daemon to stop on host app83
>>> No taskmanager daemon to stop on host app84
>>> No taskmanager daemon to stop on host app85
>>> No taskmanager daemon to stop on host app86
>>> No taskmanager daemon to stop on host app87
>>> No jobmanager daemon to stop on host app23
>>> No jobmanager daemon to stop on host app24
>>
>>
>> But my cluster is actually running…
>> Without tool the only choice I have is to terminate those processes on each server manually.
>>
>> Anyone know if this is a setup issue? or a way to solve this?
>>
>>
>> Thanks!
>>
>>
>>
>> Andrew
>>
>>
>> --
>> Confidentiality Notice: This e-mail transmission may contain confidential
>> or legally privileged information that is intended only for the individual
>> or entity named in the e-mail address. If you are not the intended
>> recipient, you are hereby notified that any disclosure, copying,
>> distribution, or reliance upon the contents of this e-mail is strictly
>> prohibited and may be unlawful. If you have received this e-mail in error,
>> please notify the sender immediately by return e-mail and delete all copies
>> of this message.

--
Confidentiality Notice: This e-mail transmission may contain confidential
or legally privileged information that is intended only for the individual
or entity named in the e-mail address. If you are not the intended
recipient, you are hereby notified that any disclosure, copying,
distribution, or reliance upon the contents of this e-mail is strictly
prohibited and may be unlawful. If you have received this e-mail in error,
please notify the sender immediately by return e-mail and delete all copies
of this message.