Log messages - redirect

[Juan Fumero]

Hi,
  is there any option (from API level) to redirect the log messages to a
file instead of stdout?

Thanks


log4j:WARN No appenders could be found for logger
(org.apache.flink.api.java.ExecutionEnvironment).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for
more info.
06/19/2015 12:14:12     Job execution switched to status RUNNING.
....
06/19/2015 12:14:12     DataSink (collect() sink)(1/1) switched to
DEPLOYING
06/19/2015 12:14:12     DataSink (collect() sink)(1/1) switched to
RUNNING
06/19/2015 12:14:12     DataSink (collect() sink)(1/1) switched to
FINISHED
06/19/2015 12:14:12     Job execution switched to status FINISHED.

[Chiwan Park]

Hi, Flink uses slf4j and log4j for logging.

You can override default log4j configuration programmatically. [1]
Or you can use logback as a logging backend and override default logback configuration also. [2][3]

[1] http://stackoverflow.com/a/9003191
[2] http://ci.apache.org/projects/flink/flink-docs-master/internals/logging.html
[3] http://stackoverflow.com/a/3810936

Regards,
Chiwan Park

> On Jun 19, 2015, at 8:05 PM, Juan Fumero <[hidden email]> wrote:
>
> Hi,
>  is there any option (from API level) to redirect the log messages to a
> file instead of stdout?
>
> Thanks
>
>
> log4j:WARN No appenders could be found for logger
> (org.apache.flink.api.java.ExecutionEnvironment).
> log4j:WARN Please initialize the log4j system properly.
> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for
> more info.
> 06/19/2015 12:14:12     Job execution switched to status RUNNING.
> ....
> 06/19/2015 12:14:12     DataSink (collect() sink)(1/1) switched to
> DEPLOYING
> 06/19/2015 12:14:12     DataSink (collect() sink)(1/1) switched to
> RUNNING
> 06/19/2015 12:14:12     DataSink (collect() sink)(1/1) switched to
> FINISHED
> 06/19/2015 12:14:12     Job execution switched to status FINISHED.
>
>

[Till Rohrmann]

If I’m not mistaken from the shown output, you’re talking about the stdout output of the client, right? This output is not controlled by the log4j.properties or logback.xml file.

However, you can use any command line tool available on your platform to redirect the stdout. For example on a Linux system, you could redirect the log output with ./flink run Flink.jar > myFile or you could use ./flink run Flink.jar | tee myFile to redirect the output and still print it.

Cheers,
Till

On Fri, Jun 19, 2015 at 1:19 PM Chiwan Park chiwanpark@... wrote:

Hi, Flink uses slf4j and log4j for logging.

You can override default log4j configuration programmatically. [1]
Or you can use logback as a logging backend and override default logback configuration also. [2][3]

[1] http://stackoverflow.com/a/9003191
[2] http://ci.apache.org/projects/flink/flink-docs-master/internals/logging.html
[3] http://stackoverflow.com/a/3810936

Regards,
Chiwan Park


> On Jun 19, 2015, at 8:05 PM, Juan Fumero <[hidden email]> wrote:
>
> Hi,
>  is there any option (from API level) to redirect the log messages to a
> file instead of stdout?
>
> Thanks
>
>
> log4j:WARN No appenders could be found for logger
> (org.apache.flink.api.java.ExecutionEnvironment).
> log4j:WARN Please initialize the log4j system properly.
> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for
> more info.
> 06/19/2015 12:14:12     Job execution switched to status RUNNING.
> ....
> 06/19/2015 12:14:12     DataSink (collect() sink)(1/1) switched to
> DEPLOYING
> 06/19/2015 12:14:12     DataSink (collect() sink)(1/1) switched to
> RUNNING
> 06/19/2015 12:14:12     DataSink (collect() sink)(1/1) switched to
> FINISHED
> 06/19/2015 12:14:12     Job execution switched to status FINISHED.
>
>

​

[Juan Fumero]

Hi Till,
   thanks for your answer, yes I am referring to the client output. What
I want is either redirect *only* these logs or disable them.

I can not redirect the full output because I need my application output
(I am inside another compiler).

Thanks
Juan

On Fre, 2015-06-19 at 12:39 +0000, Till Rohrmann wrote:

> If I’m not mistaken from the shown output, you’re talking about the
> stdout output of the client, right? This output is not controlled by
> the log4j.properties or logback.xml file.
>
> However, you can use any command line tool available on your platform
> to redirect the stdout. For example on a Linux system, you could
> redirect the log output with ./flink run Flink.jar > myFile or you
> could use ./flink run Flink.jar | tee myFile to redirect the output
> and still print it.
>
> Cheers,
> Till
>
> On Fri, Jun 19, 2015 at 1:19 PM Chiwan Park [hidden email]
> wrote:
>
>         Hi, Flink uses slf4j and log4j for logging.
>        
>         You can override default log4j configuration programmatically.
>         [1]
>         Or you can use logback as a logging backend and override
>         default logback configuration also. [2][3]
>        
>         [1] http://stackoverflow.com/a/9003191
>         [2]
>         http://ci.apache.org/projects/flink/flink-docs-master/internals/logging.html
>         [3] http://stackoverflow.com/a/3810936
>        
>         Regards,
>         Chiwan Park
>        
>        
>         > On Jun 19, 2015, at 8:05 PM, Juan Fumero
>         <[hidden email]> wrote:
>         >
>         > Hi,
>         >  is there any option (from API level) to redirect the log
>         messages to a
>         > file instead of stdout?
>         >
>         > Thanks
>         >
>         >
>         > log4j:WARN No appenders could be found for logger
>         > (org.apache.flink.api.java.ExecutionEnvironment).
>         > log4j:WARN Please initialize the log4j system properly.
>         > log4j:WARN See
>         http://logging.apache.org/log4j/1.2/faq.html#noconfig for
>         > more info.
>         > 06/19/2015 12:14:12     Job execution switched to status
>         RUNNING.
>         > ....
>         > 06/19/2015 12:14:12     DataSink (collect() sink)(1/1)
>         switched to
>         > DEPLOYING
>         > 06/19/2015 12:14:12     DataSink (collect() sink)(1/1)
>         switched to
>         > RUNNING
>         > 06/19/2015 12:14:12     DataSink (collect() sink)(1/1)
>         switched to
>         > FINISHED
>         > 06/19/2015 12:14:12     Job execution switched to status
>         FINISHED.
>         >
>         >
>        
>        
>        
>
> ​

[Stephan Ewen]

The client produces logs as "sysout" messages and log messages.

You can suppress the "sysout" messages by setting "env.getConfig().disableSysoutLogging()" in your program.

On Mon, Jun 22, 2015 at 10:55 AM, Juan Fumero <[hidden email]> wrote:

Hi Till,
   thanks for your answer, yes I am referring to the client output. What
I want is either redirect *only* these logs or disable them.

I can not redirect the full output because I need my application output
(I am inside another compiler).

Thanks
Juan

On Fre, 2015-06-19 at 12:39 +0000, Till Rohrmann wrote:
> If I’m not mistaken from the shown output, you’re talking about the
> stdout output of the client, right? This output is not controlled by
> the log4j.properties or logback.xml file.
>
> However, you can use any command line tool available on your platform
> to redirect the stdout. For example on a Linux system, you could
> redirect the log output with ./flink run Flink.jar > myFile or you
> could use ./flink run Flink.jar | tee myFile to redirect the output
> and still print it.
>
> Cheers,
> Till
>
> On Fri, Jun 19, 2015 at 1:19 PM Chiwan Park [hidden email]
> wrote:
>
>         Hi, Flink uses slf4j and log4j for logging.
>
>         You can override default log4j configuration programmatically.
>         [1]
>         Or you can use logback as a logging backend and override
>         default logback configuration also. [2][3]
>
>         [1] http://stackoverflow.com/a/9003191
>         [2]
>         http://ci.apache.org/projects/flink/flink-docs-master/internals/logging.html
>         [3] http://stackoverflow.com/a/3810936
>
>         Regards,
>         Chiwan Park
>
>
>         > On Jun 19, 2015, at 8:05 PM, Juan Fumero
>         <[hidden email]> wrote:
>         >
>         > Hi,
>         >  is there any option (from API level) to redirect the log
>         messages to a
>         > file instead of stdout?
>         >
>         > Thanks
>         >
>         >
>         > log4j:WARN No appenders could be found for logger
>         > (org.apache.flink.api.java.ExecutionEnvironment).
>         > log4j:WARN Please initialize the log4j system properly.
>         > log4j:WARN See
>         http://logging.apache.org/log4j/1.2/faq.html#noconfig for
>         > more info.
>         > 06/19/2015 12:14:12     Job execution switched to status
>         RUNNING.
>         > ....
>         > 06/19/2015 12:14:12     DataSink (collect() sink)(1/1)
>         switched to
>         > DEPLOYING
>         > 06/19/2015 12:14:12     DataSink (collect() sink)(1/1)
>         switched to
>         > RUNNING
>         > 06/19/2015 12:14:12     DataSink (collect() sink)(1/1)
>         switched to
>         > FINISHED
>         > 06/19/2015 12:14:12     Job execution switched to status
>         FINISHED.
>         >
>         >
>
>
>
>
> ​