Job Manager not able to fetch job info when restarted

[Sushil Ks]

Hi,
      We are launching Flink on Yarn job, here whenever the job manager gets restarted, the previous task that was running doesn't get restarted instead it shows no jobs running. Is there a way to resume previous running task, when job manager restarts.

Regards,
Sushil Ks 

[Stefan Richter]

Hi,

did you configure high availability with Zookeeper, as described here: https://ci.apache.org/projects/flink/flink-docs-release-1.3/setup/jobmanager_high_availability.html ? This should bring the desired behaviour.

Best,

Stefan

Am 03.01.2018 um 14:04 schrieb Sushil Ks <[hidden email]>:

Hi,
      We are launching Flink on Yarn job, here whenever the job manager gets restarted, the previous task that was running doesn't get restarted instead it shows no jobs running. Is there a way to resume previous running task, when job manager restarts.

Regards,
Sushil Ks 

[Sushil Ks]

Hi Stefan,
             It was just ran without HA, via yarn. Will try running a yarn session with HA and test, thanks for you time. Also is there a way we can get alerts if a pipeline restarts or gets cancelled in Flink?

Regards,
Sushil Ks

On Jan 3, 2018 7:07 PM, "Stefan Richter" <[hidden email]> wrote:

Hi,

did you configure high availability with Zookeeper, as described here: https://ci.apache.org/projects/flink/flink-docs-release-1.3/setup/jobmanager_high_availability.html ? This should bring the desired behaviour.

Best,

Stefan

Am 03.01.2018 um 14:04 schrieb Sushil Ks <[hidden email]>:

Hi,
      We are launching Flink on Yarn job, here whenever the job manager gets restarted, the previous task that was running doesn't get restarted instead it shows no jobs running. Is there a way to resume previous running task, when job manager restarts.

Regards,
Sushil Ks 

[Stefan Richter]

Yes, HA is required for what you want to do. I am not aware of an alert mechanism in Flink, but would assume that this is something that should better be solved on the YARN level?

Best,

Stefan

Am 03.01.2018 um 19:27 schrieb Sushil Ks <[hidden email]>:

Hi Stefan,
             It was just ran without HA, via yarn. Will try running a yarn session with HA and test, thanks for you time. Also is there a way we can get alerts if a pipeline restarts or gets cancelled in Flink?

Regards,
Sushil Ks

On Jan 3, 2018 7:07 PM, "Stefan Richter" <[hidden email]> wrote:

Hi,

did you configure high availability with Zookeeper, as described here: https://ci.apache.org/projects/flink/flink-docs-release-1.3/setup/jobmanager_high_availability.html ? This should bring the desired behaviour.

Best,

Stefan

Am 03.01.2018 um 14:04 schrieb Sushil Ks <[hidden email]>:

Hi,
      We are launching Flink on Yarn job, here whenever the job manager gets restarted, the previous task that was running doesn't get restarted instead it shows no jobs running. Is there a way to resume previous running task, when job manager restarts.

Regards,
Sushil Ks 

[Sushil Ks]

Oh okay! Thanks!

On Jan 4, 2018 2:47 PM, "Stefan Richter" <[hidden email]> wrote:

Yes, HA is required for what you want to do. I am not aware of an alert mechanism in Flink, but would assume that this is something that should better be solved on the YARN level?

Best,

Stefan

Am 03.01.2018 um 19:27 schrieb Sushil Ks <[hidden email]>:

Hi Stefan,
             It was just ran without HA, via yarn. Will try running a yarn session with HA and test, thanks for you time. Also is there a way we can get alerts if a pipeline restarts or gets cancelled in Flink?

Regards,
Sushil Ks

On Jan 3, 2018 7:07 PM, "Stefan Richter" <[hidden email]> wrote:

Hi,

did you configure high availability with Zookeeper, as described here: https://ci.apache.org/projects/flink/flink-docs-release-1.3/setup/jobmanager_high_availability.html ? This should bring the desired behaviour.

Best,

Stefan

Am 03.01.2018 um 14:04 schrieb Sushil Ks <[hidden email]>:

Hi,
      We are launching Flink on Yarn job, here whenever the job manager gets restarted, the previous task that was running doesn't get restarted instead it shows no jobs running. Is there a way to resume previous running task, when job manager restarts.

Regards,
Sushil Ks 

[sen]

If I want the job be restarted after jobmanager restart ,is it must be
zookeeper when on HA mode?

high-availability: zookeeper



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/