Logging Exceptions

[David Kim]

Hello!

Using flink 1.0.2, I noticed that exceptions thrown during a flink program would show up on the flink dashboard in the 'Exceptions' tab. That's great!

However, I don't think flink currently logs this same exception. I was hoping there would be an equivalent `log.error` call so that third party logging frameworks can also act upon such errors.

If this currently the known behavior, would it be troublesome to also make a `log.error` call around the code that is responsible for sending the exception to the dashboard?

If there's a misconfiguration on my end, let me know!

Thanks!

David

[David Kim]

Hello!

Just wanted to check up on this. :)

I grepped around for `log.error` and it seems that currently the only events for logging out exceptions are for non-application related errors.

Thanks!

David

On Fri, May 20, 2016 at 12:35 PM David Kim <[hidden email]> wrote:

Hello!

Using flink 1.0.2, I noticed that exceptions thrown during a flink program would show up on the flink dashboard in the 'Exceptions' tab. That's great!

However, I don't think flink currently logs this same exception. I was hoping there would be an equivalent `log.error` call so that third party logging frameworks can also act upon such errors.

If this currently the known behavior, would it be troublesome to also make a `log.error` call around the code that is responsible for sending the exception to the dashboard?

If there's a misconfiguration on my end, let me know!

Thanks!

David

[Maximilian Michels]

Hi David,

I'm afraid Flink logs all exceptions. You'll find the exceptions in the /log directory.

Cheers,

Max

On Mon, May 23, 2016 at 6:18 PM, David Kim <[hidden email]> wrote:

Hello!

Just wanted to check up on this. :)

I grepped around for `log.error` and it seems that currently the only events for logging out exceptions are for non-application related errors.

Thanks!

David

On Fri, May 20, 2016 at 12:35 PM David Kim <[hidden email]> wrote:

Hello!

Using flink 1.0.2, I noticed that exceptions thrown during a flink program would show up on the flink dashboard in the 'Exceptions' tab. That's great!

However, I don't think flink currently logs this same exception. I was hoping there would be an equivalent `log.error` call so that third party logging frameworks can also act upon such errors.

If this currently the known behavior, would it be troublesome to also make a `log.error` call around the code that is responsible for sending the exception to the dashboard?

If there's a misconfiguration on my end, let me know!

Thanks!

David

[David Kim]

Hi Max!

Unfortunately, that's not the behavior I'm seeing.

I verified my log4.properties is configured properly because I do see messages in the /log directory.

However, for this stack trace (grabbed from the web dashboard), I do not see it in my log file:

java.lang.RuntimeException: Could not forward element to next operator
	at org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:354)
	at org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:337)
	at org.apache.flink.streaming.api.operators.StreamSource$NonTimestampContext.collect(StreamSource.java:158)
	at org.apache.flink.streaming.api.functions.source.FromElementsFunction.run(FromElementsFunction.java:127)
	at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:78)
	at org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run(SourceStreamTask.java:56)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:225)
	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:559)
	at java.lang.Thread.run(Thread.java:745)

Here's my sample program:

val env = StreamExecutionEnvironment.getExecutionEnvironment
env
  .fromCollection(List(1))
  .filter(element => {
    throw new RuntimeException("Throwing a runtime to test exception logging :)")
  })
env.execute("MyTestJobName")

The error is thrown in Task.java[1] and it was hard to track where flink would actually log to the log file.

[1] https://github.com/apache/flink/blob/release-1.0.2-rc3/flink-runtime/src/main/java/org/apache/flink/runtime/taskmanager/Task.java#L559

Thanks!
David

On Mon, May 23, 2016 at 12:01 PM Maximilian Michels <[hidden email]> wrote:

Hi David,

I'm afraid Flink logs all exceptions. You'll find the exceptions in the /log directory.

Cheers,

Max

On Mon, May 23, 2016 at 6:18 PM, David Kim <[hidden email]> wrote:

Hello!

Just wanted to check up on this. :)

I grepped around for `log.error` and it seems that currently the only events for logging out exceptions are for non-application related errors.

Thanks!

David

On Fri, May 20, 2016 at 12:35 PM David Kim <[hidden email]> wrote:

Hello!

Using flink 1.0.2, I noticed that exceptions thrown during a flink program would show up on the flink dashboard in the 'Exceptions' tab. That's great!

However, I don't think flink currently logs this same exception. I was hoping there would be an equivalent `log.error` call so that third party logging frameworks can also act upon such errors.

If this currently the known behavior, would it be troublesome to also make a `log.error` call around the code that is responsible for sending the exception to the dashboard?

If there's a misconfiguration on my end, let me know!

Thanks!

David

[Aljoscha Krettek]

Hi David,

you are right, for some exceptions Flink only forwards to the web-dashboard/application client but does not print to the log file. I opened a Jira issue to track this:  FLINK-3969.

Thanks for reporting!

Aljoscha

On Mon, 23 May 2016 at 22:08 David Kim <[hidden email]> wrote:

Hi Max!

Unfortunately, that's not the behavior I'm seeing.

I verified my log4.properties is configured properly because I do see messages in the /log directory.

However, for this stack trace (grabbed from the web dashboard), I do not see it in my log file:

java.lang.RuntimeException: Could not forward element to next operator
	at org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:354)
	at org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:337)
	at org.apache.flink.streaming.api.operators.StreamSource$NonTimestampContext.collect(StreamSource.java:158)
	at org.apache.flink.streaming.api.functions.source.FromElementsFunction.run(FromElementsFunction.java:127)
	at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:78)
	at org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run(SourceStreamTask.java:56)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:225)
	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:559)
	at java.lang.Thread.run(Thread.java:745)

Here's my sample program:

val env = StreamExecutionEnvironment.getExecutionEnvironment
env
  .fromCollection(List(1))
  .filter(element => {
    throw new RuntimeException("Throwing a runtime to test exception logging :)")
  })
env.execute("MyTestJobName")

The error is thrown in Task.java[1] and it was hard to track where flink would actually log to the log file.

[1] https://github.com/apache/flink/blob/release-1.0.2-rc3/flink-runtime/src/main/java/org/apache/flink/runtime/taskmanager/Task.java#L559

Thanks!

David

On Mon, May 23, 2016 at 12:01 PM Maximilian Michels <[hidden email]> wrote:

Hi David,

I'm afraid Flink logs all exceptions. You'll find the exceptions in the /log directory.

Cheers,

Max

On Mon, May 23, 2016 at 6:18 PM, David Kim <[hidden email]> wrote:

Hello!

Just wanted to check up on this. :)

I grepped around for `log.error` and it seems that currently the only events for logging out exceptions are for non-application related errors.

Thanks!

David

On Fri, May 20, 2016 at 12:35 PM David Kim <[hidden email]> wrote:

Hello!

Using flink 1.0.2, I noticed that exceptions thrown during a flink program would show up on the flink dashboard in the 'Exceptions' tab. That's great!

However, I don't think flink currently logs this same exception. I was hoping there would be an equivalent `log.error` call so that third party logging frameworks can also act upon such errors.

If this currently the known behavior, would it be troublesome to also make a `log.error` call around the code that is responsible for sending the exception to the dashboard?

If there's a misconfiguration on my end, let me know!

Thanks!

David

[David Kim]

Awesome, thank you!

David

On Wed, May 25, 2016 at 4:54 AM Aljoscha Krettek <[hidden email]> wrote:

Hi David,

you are right, for some exceptions Flink only forwards to the web-dashboard/application client but does not print to the log file. I opened a Jira issue to track this:  FLINK-3969.

Thanks for reporting!

Aljoscha

On Mon, 23 May 2016 at 22:08 David Kim <[hidden email]> wrote:

Hi Max!

Unfortunately, that's not the behavior I'm seeing.

I verified my log4.properties is configured properly because I do see messages in the /log directory.

However, for this stack trace (grabbed from the web dashboard), I do not see it in my log file:

java.lang.RuntimeException: Could not forward element to next operator
	at org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:354)
	at org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:337)
	at org.apache.flink.streaming.api.operators.StreamSource$NonTimestampContext.collect(StreamSource.java:158)
	at org.apache.flink.streaming.api.functions.source.FromElementsFunction.run(FromElementsFunction.java:127)
	at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:78)
	at org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run(SourceStreamTask.java:56)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:225)
	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:559)
	at java.lang.Thread.run(Thread.java:745)

Here's my sample program:

val env = StreamExecutionEnvironment.getExecutionEnvironment
env
  .fromCollection(List(1))
  .filter(element => {
    throw new RuntimeException("Throwing a runtime to test exception logging :)")
  })
env.execute("MyTestJobName")

The error is thrown in Task.java[1] and it was hard to track where flink would actually log to the log file.

[1] https://github.com/apache/flink/blob/release-1.0.2-rc3/flink-runtime/src/main/java/org/apache/flink/runtime/taskmanager/Task.java#L559

Thanks!

David

On Mon, May 23, 2016 at 12:01 PM Maximilian Michels <[hidden email]> wrote:

Hi David,

I'm afraid Flink logs all exceptions. You'll find the exceptions in the /log directory.

Cheers,

Max

On Mon, May 23, 2016 at 6:18 PM, David Kim <[hidden email]> wrote:

Hello!

Just wanted to check up on this. :)

I grepped around for `log.error` and it seems that currently the only events for logging out exceptions are for non-application related errors.

Thanks!

David

On Fri, May 20, 2016 at 12:35 PM David Kim <[hidden email]> wrote:

Hello!

Using flink 1.0.2, I noticed that exceptions thrown during a flink program would show up on the flink dashboard in the 'Exceptions' tab. That's great!

However, I don't think flink currently logs this same exception. I was hoping there would be an equivalent `log.error` call so that third party logging frameworks can also act upon such errors.

If this currently the known behavior, would it be troublesome to also make a `log.error` call around the code that is responsible for sending the exception to the dashboard?

If there's a misconfiguration on my end, let me know!

Thanks!

David