Flink not giving full reason as to why job submission failed

[HarshithBolar]

Hi all,

 

After upgrading Flink to 1.7.2, when I try to submit a job from the dashboard and there's some issue with the job, the job submission fails with the following error.

 

    Exception occurred in REST handler: org.apache.flink.client.program.ProgramInvocationException: The main method caused an error.

There's no other reason given as to why the job failed to submit. This was not the case in 1.4.2. Is there a way to see the full reason why the job failed to deploy? I see the same error in the logs too with no additional information.

Thanks,

Harshith

[Wouter Zorgdrager-2]

Hi Harshith,

This was indeed an issue in 1.7.2, but fixed in 1.8.0. See the corresponding Jira issue [1].

Cheers,

Wouter

[1]: https://issues.apache.org/jira/browse/FLINK-11902 

Op do 16 mei 2019 om 16:05 schreef Kumar Bolar, Harshith <[hidden email]>:

Hi all,

 

After upgrading Flink to 1.7.2, when I try to submit a job from the dashboard and there's some issue with the job, the job submission fails with the following error.

 

    Exception occurred in REST handler: org.apache.flink.client.program.ProgramInvocationException: The main method caused an error.

There's no other reason given as to why the job failed to submit. This was not the case in 1.4.2. Is there a way to see the full reason why the job failed to deploy? I see the same error in the logs too with no additional information.

Thanks,

Harshith

[HarshithBolar]

Thank you ☺

 

From: Wouter Zorgdrager <[hidden email]>
Date: Thursday, 16 May 2019 at 7:56 PM
To: Harshith Kumar Bolar <[hidden email]>
Cc: user <[hidden email]>
Subject: [External] Re: Flink not giving full reason as to why job submission failed

 

Hi Harshith,

 

This was indeed an issue in 1.7.2, but fixed in 1.8.0. See the corresponding Jira issue [1].

 

Cheers,

Wouter

 

[1]: https://issues.apache.org/jira/browse/FLINK-11902 

 

Op do 16 mei 2019 om 16:05 schreef Kumar Bolar, Harshith <[hidden email]>:

Hi all,

 

After upgrading Flink to 1.7.2, when I try to submit a job from the dashboard and there's some issue with the job, the job submission fails with the following error.

 

    Exception occurred in REST handler: org.apache.flink.client.program.ProgramInvocationException: The main method caused an error.

There's no other reason given as to why the job failed to submit. This was not the case in 1.4.2. Is there a way to see the full reason why the job failed to deploy? I see the same error in the logs too with no additional information.

Thanks,

Harshith

[HarshithBolar]

Hi Wouter,

 

I’ve upgraded Flink to 1.8, but now I only see Internal server error on the dashboard when a job deployment fails.

 

 

But in the logs I see the correct exception -
                Caused by: java.lang.ArrayIndexOutOfBoundsException: 1

      at functions.PersistPIDMessagesToCassandra.main(PersistPIDMessagesToCassandra.java:59)

      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

      at java.lang.reflect.Method.invoke(Method.java:498)

      at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:529)

      ... 9 more

 

Is there any way to show these errors on the dashboard? Because many application teams deploy jobs through the dashboard and don’t have ready access to the logs.

 

Thanks,

Harshith

 

From: Wouter Zorgdrager <[hidden email]>
Date: Thursday, 16 May 2019 at 7:56 PM
To: Harshith Kumar Bolar <[hidden email]>
Cc: user <[hidden email]>
Subject: [External] Re: Flink not giving full reason as to why job submission failed

 

Hi Harshith,

 

This was indeed an issue in 1.7.2, but fixed in 1.8.0. See the corresponding Jira issue [1].

 

Cheers,

Wouter

 

[1]: https://issues.apache.org/jira/browse/FLINK-11902 

 

Op do 16 mei 2019 om 16:05 schreef Kumar Bolar, Harshith <[hidden email]>:

Hi all,

 

After upgrading Flink to 1.7.2, when I try to submit a job from the dashboard and there's some issue with the job, the job submission fails with the following error.

 

    Exception occurred in REST handler: org.apache.flink.client.program.ProgramInvocationException: The main method caused an error.

There's no other reason given as to why the job failed to submit. This was not the case in 1.4.2. Is there a way to see the full reason why the job failed to deploy? I see the same error in the logs too with no additional information.

Thanks,

Harshith

[Wouter Zorgdrager-2]

Hi Harshith,

This is indeed an issue not resolved in 1.8. I added a comment to the (closed) Jira issue, so this might be fixed in further releases.

Cheers,

Wouter

Op ma 20 mei 2019 om 16:18 schreef Kumar Bolar, Harshith <[hidden email]>:

Hi Wouter,

 

I’ve upgraded Flink to 1.8, but now I only see Internal server error on the dashboard when a job deployment fails.

 

 

But in the logs I see the correct exception -
                Caused by: java.lang.ArrayIndexOutOfBoundsException: 1

      at functions.PersistPIDMessagesToCassandra.main(PersistPIDMessagesToCassandra.java:59)

      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

      at java.lang.reflect.Method.invoke(Method.java:498)

      at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:529)

      ... 9 more

 

Is there any way to show these errors on the dashboard? Because many application teams deploy jobs through the dashboard and don’t have ready access to the logs.

 

Thanks,

Harshith

 

From: Wouter Zorgdrager <[hidden email]>
Date: Thursday, 16 May 2019 at 7:56 PM
To: Harshith Kumar Bolar <[hidden email]>
Cc: user <[hidden email]>
Subject: [External] Re: Flink not giving full reason as to why job submission failed

 

Hi Harshith,

 

This was indeed an issue in 1.7.2, but fixed in 1.8.0. See the corresponding Jira issue [1].

 

Cheers,

Wouter

 

[1]: https://issues.apache.org/jira/browse/FLINK-11902 

 

Op do 16 mei 2019 om 16:05 schreef Kumar Bolar, Harshith <[hidden email]>:

Hi all,

 

After upgrading Flink to 1.7.2, when I try to submit a job from the dashboard and there's some issue with the job, the job submission fails with the following error.

 

    Exception occurred in REST handler: org.apache.flink.client.program.ProgramInvocationException: The main method caused an error.

There's no other reason given as to why the job failed to submit. This was not the case in 1.4.2. Is there a way to see the full reason why the job failed to deploy? I see the same error in the logs too with no additional information.

Thanks,

Harshith

[Chesnay Schepler]

Please open a new JIRA. FLINK-11902 modified the REST API to no longer hide the exception, but the WebUI isn't handling the error response properly (it only reads and displays part of it).

On 20/05/2019 16:24, Wouter Zorgdrager wrote:

Hi Harshith,

This is indeed an issue not resolved in 1.8. I added a comment to the (closed) Jira issue, so this might be fixed in further releases.

Cheers,

Wouter

Op ma 20 mei 2019 om 16:18 schreef Kumar Bolar, Harshith <[hidden email]>:

Hi Wouter,

 

I’ve upgraded Flink to 1.8, but now I only see Internal server error on the dashboard when a job deployment fails.

 

 

But in the logs I see the correct exception -
                Caused by: java.lang.ArrayIndexOutOfBoundsException: 1

      at functions.PersistPIDMessagesToCassandra.main(PersistPIDMessagesToCassandra.java:59)

      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

      at java.lang.reflect.Method.invoke(Method.java:498)

      at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:529)

      ... 9 more

 

Is there any way to show these errors on the dashboard? Because many application teams deploy jobs through the dashboard and don’t have ready access to the logs.

 

Thanks,

Harshith

 

From: Wouter Zorgdrager <[hidden email]>
Date: Thursday, 16 May 2019 at 7:56 PM
To: Harshith Kumar Bolar <[hidden email]>
Cc: user <[hidden email]>
Subject: [External] Re: Flink not giving full reason as to why job submission failed

 

Hi Harshith,

 

This was indeed an issue in 1.7.2, but fixed in 1.8.0. See the corresponding Jira issue [1].

 

Cheers,

Wouter

 

[1]: https://issues.apache.org/jira/browse/FLINK-11902 

 

Op do 16 mei 2019 om 16:05 schreef Kumar Bolar, Harshith <[hidden email]>:

Hi all,

 

After upgrading Flink to 1.7.2, when I try to submit a job from the dashboard and there's some issue with the job, the job submission fails with the following error.

 

    Exception occurred in REST handler: org.apache.flink.client.program.ProgramInvocationException: The main method caused an error.

There's no other reason given as to why the job failed to submit. This was not the case in 1.4.2. Is there a way to see the full reason why the job failed to deploy? I see the same error in the logs too with no additional information.

Thanks,

Harshith

[HarshithBolar]

Hi Chesnay,

 

I’ve created the ticket here - https://issues.apache.org/jira/browse/FLINK-12632

 

From: Chesnay Schepler <[hidden email]>
Date: Thursday, 23 May 2019 at 5:48 PM
To: Wouter Zorgdrager <[hidden email]>, Harshith Kumar Bolar <[hidden email]>
Cc: user <[hidden email]>
Subject: [External] Re: Flink not giving full reason as to why job submission failed

 

Please open a new JIRA. FLINK-11902 modified the REST API to no longer hide the exception, but the WebUI isn't handling the error response properly (it only reads and displays part of it).

On 20/05/2019 16:24, Wouter Zorgdrager wrote:

Hi Harshith,

 

This is indeed an issue not resolved in 1.8. I added a comment to the (closed) Jira issue, so this might be fixed in further releases.

 

Cheers,

Wouter

 

Op ma 20 mei 2019 om 16:18 schreef Kumar Bolar, Harshith <[hidden email]>:

Hi Wouter,

 

I’ve upgraded Flink to 1.8, but now I only see Internal server error on the dashboard when a job deployment fails.

 

 

But in the logs I see the correct exception -
                Caused by: java.lang.ArrayIndexOutOfBoundsException: 1

      at functions.PersistPIDMessagesToCassandra.main(PersistPIDMessagesToCassandra.java:59)

      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

      at java.lang.reflect.Method.invoke(Method.java:498)

      at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:529)

      ... 9 more

 

Is there any way to show these errors on the dashboard? Because many application teams deploy jobs through the dashboard and don’t have ready access to the logs.

 

Thanks,

Harshith

 

From: Wouter Zorgdrager <[hidden email]>
Date: Thursday, 16 May 2019 at 7:56 PM
To: Harshith Kumar Bolar <[hidden email]>
Cc: user <[hidden email]>
Subject: [External] Re: Flink not giving full reason as to why job submission failed

 

Hi Harshith,

 

This was indeed an issue in 1.7.2, but fixed in 1.8.0. See the corresponding Jira issue [1].

 

Cheers,

Wouter

 

[1]: https://issues.apache.org/jira/browse/FLINK-11902 

 

Op do 16 mei 2019 om 16:05 schreef Kumar Bolar, Harshith <[hidden email]>:

Hi all,

 

After upgrading Flink to 1.7.2, when I try to submit a job from the dashboard and there's some issue with the job, the job submission fails with the following error.

 

    Exception occurred in REST handler: org.apache.flink.client.program.ProgramInvocationException: The main method caused an error.

There's no other reason given as to why the job failed to submit. This was not the case in 1.4.2. Is there a way to see the full reason why the job failed to deploy? I see the same error in the logs too with no additional information.

Thanks,

Harshith