Flink Application Log :Custom Log properties

[raikarsunil]

Hi ,

Is there any way we can configure application logging by passing the custom log4j properties file in flink run command? The aim is to write all the application logs in a specific file rather than the default yarn/flink log file.

Currently I am using the below command to run my application :

flink run -m yarn-cluster -yst -yn 2 -ynm MyAppName -yd -yt CustomFileLocation JARFILE ARGS

NOTE:There are many applications running on the EMR ,looking for solution without tweaking the log properties in the flink directory.

Thanks,
Sunil

[Fabian Hueske-2]

Hi Sunil,

I have not much experience with logging on Yarn, but Robert (in CC) might be able to help you.

Cheers, Fabian

2017-02-28 14:54 GMT+01:00 raikarsunil <[hidden email]>:

Hi ,

Is there any way we can configure application logging by passing the custom
log4j properties file in flink run command? The aim is to write all the
application logs in a specific file rather than the default yarn/flink log
file.

Currently I am using the below command to run my application :

flink run -m yarn-cluster -yst -yn 2 -ynm MyAppName -yd -yt
CustomFileLocation JARFILE ARGS

*NOTE:There are many applications running on the EMR ,looking for solution
without tweaking the log properties in the flink directory.*

Thanks,
Sunil



--
View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-Application-Log-Custom-Log-properties-tp11960.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.

[Sathi Chowdhury]

Thanks Sunil, this is also a question I wanted to ask to the forum. How to separate application log (aggregated to a file ) just similar to yarn log aggregation but capture flink process related logs from all workers and get it in one central location.
I would love to hear suggestion and stories around if someone is running flink pipeline in production, how are they instrumenting the error logs etcc.
Thanks
Sathi
On 2/28/17, 5:54 AM, "raikarsunil" <[hidden email]> wrote:

Hi ,

Is there any way we can configure application logging by passing the custom
log4j properties file in flink run command? The aim is to write all the
application logs in a specific file rather than the default yarn/flink log
file.

Currently I am using the below command to run my application :

flink run -m yarn-cluster -yst -yn 2 -ynm MyAppName -yd -yt
CustomFileLocation JARFILE ARGS

*NOTE:There are many applications running on the EMR ,looking for solution
without tweaking the log properties in the flink directory.*

Thanks,
Sunil



--
View this message in context: https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fapache-flink-user-mailing-list-archive.2336050.n4.nabble.com%2FFlink-Application-Log-Custom-Log-properties-tp11960.html&data=01%7C01%7C%7C6850dd45302942f18a7208d45fe21776%7C0d009d13c2cd47d891dd2ae838b00d4b%7C0&sdata=4WgG%2BccM4JQrMdYTkSLxChGairkXrLwDBajVZfbklRk%3D&reserved=0
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.


=============Notice to Recipient: This e-mail transmission, and any documents, files or previous e-mail messages attached to it may contain information that is confidential or legally privileged, and intended for the use of the individual or entity named above. If you are not the intended recipient, or a person responsible for delivering it to the intended recipient, you are hereby notified that you must not read this transmission and that any disclosure, copying, printing, distribution or use of any of the information contained in or attached to this transmission is STRICTLY PROHIBITED. If you have received this transmission in error, please immediately notify the sender by telephone or return e-mail and delete the original transmission and its attachments without reading or saving in any manner. Thank you. =============