Application logging on YARN

[stefanobaghino]

There's another open thread on a similar situation but I'm not quite sure this is completely related, so I'm opening a new one.

Is there a particular setting that has to be used when logging from an application that is running in YARN? I quickly put together a simple application that reads from and writes to Kafka, starting from the Maven Java archetype. I didn't edit the log4j.properties coming with the archetype, so it's currently using a ConsoleAppender, but I'm not sure on how I should edit it in order to make it appear in the YARN aggregate logs; as of now, yarn logs just show the TaskManager and JobManager logs, with no signs of my logs appearing on them. Should I look elsewhere.

In the application I use SLF4J, like this:

private static final Logger logger = LoggerFactory.getLogger(Job.class);

To make sure I'm not just skipping the points where I log, I put error level logs at the very first line of my application. Still nothing.

--

[stefanobaghino]

Ok, my bad, I was simply looking in the wrong place. I though the logs were sent to YARN but they were actually stored in the Flink logs folder. Problem solved, sorry for the mix up.

On Sun, Mar 13, 2016 at 8:48 PM, Stefano Baghino <[hidden email]> wrote:

There's another open thread on a similar situation but I'm not quite sure this is completely related, so I'm opening a new one.

Is there a particular setting that has to be used when logging from an application that is running in YARN? I quickly put together a simple application that reads from and writes to Kafka, starting from the Maven Java archetype. I didn't edit the log4j.properties coming with the archetype, so it's currently using a ConsoleAppender, but I'm not sure on how I should edit it in order to make it appear in the YARN aggregate logs; as of now, yarn logs just show the TaskManager and JobManager logs, with no signs of my logs appearing on them. Should I look elsewhere.

In the application I use SLF4J, like this:

private static final Logger logger = LoggerFactory.getLogger(Job.class);

To make sure I'm not just skipping the points where I log, I put error level logs at the very first line of my application. Still nothing.

--

BR,

Stefano Baghino

Software Engineer @ Radicalbit

--