Exceptions in Web UI do not appear in logs

[orips]

In the "Exceptions" tab in the Web UI, we see exceptions that for some reason do not appear in the logs.

Specifically the exception I'm talking about is a Runtime Exception which is not logged by the code, so it's essentially uncaught exception that is propagated to the Task Manager.

We use SLF4J with Logback, and all our appenders and log levels are configured correctly. Some exceptions do appear in the logs though.

This is the exception:

java.lang.RuntimeException: Rate Exceeded for getRecords operation - all 3 retry attempts returned ProvisionedThroughputExceededException.
	at org.apache.flink.streaming.connectors.kinesis.proxy.KinesisProxy.getRecords(KinesisProxy.java:234)
	at org.apache.flink.streaming.connectors.kinesis.internals.ShardConsumer.getRecords(ShardConsumer.java:311)
	at org.apache.flink.streaming.connectors.kinesis.internals.ShardConsumer.run(ShardConsumer.java:219)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

Does someone know how to make all the exceptions in the Web UI also appear in the logs?

Thanks

[rmetzger0]

Thanks a lot for reporting this issue.

Which version of Flink are you using?

I checked the code of the Kinesis ShardConsumer (the current version though), and I found that exceptions from the ShardConsumer are properly forwarded to the lower level runtime.

Did you check the *.out files of the TaskManager where the exception occurred as well?

If you find the exception somewhere else, I'd be curious to see a longer stack trace (if there's one)

On Thu, Feb 6, 2020 at 5:59 PM Ori Popowski <[hidden email]> wrote:

In the "Exceptions" tab in the Web UI, we see exceptions that for some reason do not appear in the logs.

Specifically the exception I'm talking about is a Runtime Exception which is not logged by the code, so it's essentially uncaught exception that is propagated to the Task Manager.

We use SLF4J with Logback, and all our appenders and log levels are configured correctly. Some exceptions do appear in the logs though.

This is the exception:

java.lang.RuntimeException: Rate Exceeded for getRecords operation - all 3 retry attempts returned ProvisionedThroughputExceededException.
	at org.apache.flink.streaming.connectors.kinesis.proxy.KinesisProxy.getRecords(KinesisProxy.java:234)
	at org.apache.flink.streaming.connectors.kinesis.internals.ShardConsumer.getRecords(ShardConsumer.java:311)
	at org.apache.flink.streaming.connectors.kinesis.internals.ShardConsumer.run(ShardConsumer.java:219)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

Does someone know how to make all the exceptions in the Web UI also appear in the logs?

Thanks

[orips]

Hi,

It's version 1.5.2.

I actually found the place in the code responsible for it.
In the "catch" block, it doesn't log the error and it lets it propagate.

https://github.com/apache/flink/blob/62839e88e15b338a8af9afcef698c38a194c592f/flink-connectors/flink-connector-kinesis/src/main/java/org/apache/flink/streaming/connectors/kinesis/internals/ShardConsumer.java





--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[Arvid Heise-3]

If an exception is unhandled in connectors, it will eventually be handled by the runtime, where it is logged and the task fails. Doing both logging and throwing an exception is an anti-pattern as the consumer of an exception should have the sole responsibility of handling it correctly.

In your case, the question is why the error is not properly logged on task level. Since you are using a very old version of Flink, chances are high that this issue is already resolved in a more recent version.

Do you have the option to upgrade?

On Sun, Mar 1, 2020 at 4:56 PM orips <[hidden email]> wrote:

Hi,

It's version 1.5.2.

I actually found the place in the code responsible for it.
In the "catch" block, it doesn't log the error and it lets it propagate.

https://github.com/apache/flink/blob/62839e88e15b338a8af9afcef698c38a194c592f/flink-connectors/flink-connector-kinesis/src/main/java/org/apache/flink/streaming/connectors/kinesis/internals/ShardConsumer.java





--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/