How to stop job through java API

[Will Du]

Hi folks,
How to stop link job given job_id through java API?
Thanks,
Will

[Aljoscha Krettek]

Hi,

right now this is not possible, I'm afraid.

I'm looping in Max who has done some work in that direction. Maybe he's got something to say.

Cheers,

Aljoscha

On Wed, 14 Sep 2016 at 03:54 Will Du <[hidden email]> wrote:

Hi folks,
How to stop link job given job_id through java API?
Thanks,
Will

[LINZ, Arnaud]

Hi,

 

I have a similar issue. Here is how I deal with programmatically stopping permanent streaming jobs, and I’m interested in knowing if there is a better way now.

 

Currently, I use hand-made streaming sources that periodically check for some flag and end if a stop request was made. Stopping the source allow the on-going streaming jobs to finish smoothly.

That way, by using a “live file” presence check in my sources, and creating a “stopped ok file” when the streaming execution ends ok, I’m able to make  “a stopping shell script” that notify the streaming chain of a user cancellation by deleting the “live file” and checking for the “stopped ok file” in a waiting loop before ending.

 

This stopping shell script allow me to automate “stop/start” scripts when I need to perform some task (like a reference table refresh) that requires the streaming job to be stopped (without cancelling it from the rest interface, so without relying on the snapshot system to prevent the loss of intermediate data, and allowing me to implement clean-up code if needed).

 

The drawback of this method is not being able to use “off-the-shelf” sources directly, I always have to wrap or patch them to implement the live file check.

 

Best regards,

Arnaud

 

 

De : Aljoscha Krettek [mailto:[hidden email]]
Envoyé : mercredi 21 septembre 2016 14:54
À : [hidden email]; Max Michels <[hidden email]>
Objet : Re: How to stop job through java API

 

Hi,

right now this is not possible, I'm afraid.

 

I'm looping in Max who has done some work in that direction. Maybe he's got something to say.

 

Cheers,

Aljoscha

 

On Wed, 14 Sep 2016 at 03:54 Will Du <[hidden email]> wrote:

Hi folks,
How to stop link job given job_id through java API?
Thanks,
Will

---

L'intégrité de ce message n'étant pas assurée sur internet, la société expéditrice ne peut être tenue responsable de son contenu ni de ses pièces jointes. Toute utilisation ou diffusion non autorisée est interdite. Si vous n'êtes pas destinataire de ce message, merci de le détruire et d'avertir l'expéditeur.

The integrity of this message cannot be guaranteed on the Internet. The company that sent this message cannot therefore be held liable for its content nor attachments. Any unauthorized use or dissemination is prohibited. If you are not the intended recipient of this message, then please delete it and notify the sender.

[Will Du]

Now, I am able to create an class of flink she'll front end UI class. Once I have this instance, I can launch cmd in Java. Cmd has features to connect remote resource manager to do most if actions.

Thanks,

Will

On Oct 3, 2016, at 4:31 AM, "LINZ, Arnaud" <[hidden email]> wrote:

Hi,

 

I have a similar issue. Here is how I deal with programmatically stopping permanent streaming jobs, and I’m interested in knowing if there is a better way now.

 

Currently, I use hand-made streaming sources that periodically check for some flag and end if a stop request was made. Stopping the source allow the on-going streaming jobs to finish smoothly.

That way, by using a “live file” presence check in my sources, and creating a “stopped ok file” when the streaming execution ends ok, I’m able to make  “a stopping shell script” that notify the streaming chain of a user cancellation by deleting the “live file” and checking for the “stopped ok file” in a waiting loop before ending.

 

This stopping shell script allow me to automate “stop/start” scripts when I need to perform some task (like a reference table refresh) that requires the streaming job to be stopped (without cancelling it from the rest interface, so without relying on the snapshot system to prevent the loss of intermediate data, and allowing me to implement clean-up code if needed).

 

The drawback of this method is not being able to use “off-the-shelf” sources directly, I always have to wrap or patch them to implement the live file check.

 

Best regards,

Arnaud

 

 

De : Aljoscha Krettek [[hidden email]]
Envoyé : mercredi 21 septembre 2016 14:54
À : [hidden email]; Max Michels <[hidden email]>
Objet : Re: How to stop job through java API

 

Hi,

right now this is not possible, I'm afraid.

 

I'm looping in Max who has done some work in that direction. Maybe he's got something to say.

 

Cheers,

Aljoscha

 

On Wed, 14 Sep 2016 at 03:54 Will Du <[hidden email]> wrote:

Hi folks,
How to stop link job given job_id through java API?
Thanks,
Will

---

L'intégrité de ce message n'étant pas assurée sur internet, la société expéditrice ne peut être tenue responsable de son contenu ni de ses pièces jointes. Toute utilisation ou diffusion non autorisée est interdite. Si vous n'êtes pas destinataire de ce message, merci de le détruire et d'avertir l'expéditeur.

The integrity of this message cannot be guaranteed on the Internet. The company that sent this message cannot therefore be held liable for its content nor attachments. Any unauthorized use or dissemination is prohibited. If you are not the intended recipient of this message, then please delete it and notify the sender.

[Maximilian Michels]

Just to add to your solution, you can actually use the ClusterClient
(StandaloneClusterClient or YarnClusterClient) to achieve that. For
example,

ClusterClient client = new StandaloneClusterClient(config);
JobID jobID = JobID.fromHexString(jobID);
client.cancelJob(jobID);

Cheers,
Max

On Mon, Oct 3, 2016 at 1:25 PM, Dayong <[hidden email]> wrote:

> Now, I am able to create an class of flink she'll front end UI class. Once I
> have this instance, I can launch cmd in Java. Cmd has features to connect
> remote resource manager to do most if actions.
>
> Thanks,
> Will
>
> On Oct 3, 2016, at 4:31 AM, "LINZ, Arnaud" <[hidden email]> wrote:
>
> Hi,
>
>
>
> I have a similar issue. Here is how I deal with programmatically stopping
> permanent streaming jobs, and I’m interested in knowing if there is a better
> way now.
>
>
>
> Currently, I use hand-made streaming sources that periodically check for
> some flag and end if a stop request was made. Stopping the source allow the
> on-going streaming jobs to finish smoothly.
>
> That way, by using a “live file” presence check in my sources, and creating
> a “stopped ok file” when the streaming execution ends ok, I’m able to make
> “a stopping shell script” that notify the streaming chain of a user
> cancellation by deleting the “live file” and checking for the “stopped ok
> file” in a waiting loop before ending.
>
>
>
> This stopping shell script allow me to automate “stop/start” scripts when I
> need to perform some task (like a reference table refresh) that requires the
> streaming job to be stopped (without cancelling it from the rest interface,
> so without relying on the snapshot system to prevent the loss of
> intermediate data, and allowing me to implement clean-up code if needed).
>
>
>
> The drawback of this method is not being able to use “off-the-shelf” sources
> directly, I always have to wrap or patch them to implement the live file
> check.
>
>
>
> Best regards,
>
> Arnaud
>
>
>
>
>
> De : Aljoscha Krettek [mailto:[hidden email]]
> Envoyé : mercredi 21 septembre 2016 14:54
> À : [hidden email]; Max Michels <[hidden email]>
> Objet : Re: How to stop job through java API
>
>
>
> Hi,
>
> right now this is not possible, I'm afraid.
>
>
>
> I'm looping in Max who has done some work in that direction. Maybe he's got
> something to say.
>
>
>
> Cheers,
>
> Aljoscha
>
>
>
> On Wed, 14 Sep 2016 at 03:54 Will Du <[hidden email]> wrote:
>
> Hi folks,
> How to stop link job given job_id through java API?
> Thanks,
> Will
>
>
> ________________________________
>
> L'intégrité de ce message n'étant pas assurée sur internet, la société
> expéditrice ne peut être tenue responsable de son contenu ni de ses pièces
> jointes. Toute utilisation ou diffusion non autorisée est interdite. Si vous
> n'êtes pas destinataire de ce message, merci de le détruire et d'avertir
> l'expéditeur.
>
> The integrity of this message cannot be guaranteed on the Internet. The
> company that sent this message cannot therefore be held liable for its
> content nor attachments. Any unauthorized use or dissemination is
> prohibited. If you are not the intended recipient of this message, then
> please delete it and notify the sender.