How to log uncaught exceptions during job execution

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

How to log uncaught exceptions during job execution

Tzanko Matev
Hello,

I am trying to attach Sentry to our Flink cluster to track job execution. Sentry acts as a logger which captures messages and sends them to a central server. By default it captures all messages with level WARN or higher.

To get Sentry to catch all problems, I need to write a WARN or ERROR log message whenever an operator raises an uncaught exception. If the restart strategy fails, the execute() method in the Execution Environment will throw the final exception, which I can log appropriately. But I have not yet found a way to log exceptions which cause the job to restart. Flink logs them as INFO messages, but that makes them difficult to filter from the rest.

What is the appropriate way to handle uncaught exceptions in Flink jobs? 

Best wishes,
Tzanko