automatically submit a job to a HA cluster

[Andrew Ge Wu-2]

Hi,

We  have a streaming job wants to submit to a HA cluster via jenkins.
Recently we had a downtime on one of the master node and we have it restarted, it seems the backup master became master and submitting to the original master does not do anything.

Currently we are using command line to cancel and submit new job on one of the master when deploying new jobs.
Anyone know if it is possible to submit to a job to the current acting master in the cluster? and is it possible to detach from execution gracefully since it is a streaming job?


Thanks very much!


Andrew

[Ufuk Celebi]

On 7 November 2016 at 13:00:30, Andrew Ge Wu ([hidden email]) wrote:

> Hi,
>  
> We have a streaming job wants to submit to a HA cluster via jenkins.
> Recently we had a downtime on one of the master node and we have it restarted, it seems the  
> backup master became master and submitting to the original master does not do anything.  
>  
> Currently we are using command line to cancel and submit new job on one of the master when  
> deploying new jobs.
> Anyone know if it is possible to submit to a job to the current acting master in the cluster?  
> and is it possible to detach from execution gracefully since it is a streaming job?

The CLI already uses ZooKeeper to retrieve the current master. You have to make sure that the client side config is also setup for HA (similar to the cluster).

You can submit a job in detached mode via the -d flag.

Does this help with your problem?

– Ufuk