cleaning yarn logs for long-running applications

[Soheil Pourbafrani]

Hi, I want to use Yarn as cluster manager for running Flink applications, but I'm worried about how Flink or Yarn handle local logs in each machine. Does they clean aged logs for a long-running application? If not, it's possible the local storage get full!!!

[Julio Biason]

Hi Soheil,

That's something you probably want to take a look at logrotate: https://linux.die.net/man/8/logrotate

(We didn't reach the point of generate massive number of logs yet here, but that's the default route we take to solve this kind of issue.)

On Sun, Jan 14, 2018 at 8:12 AM, Soheil Pourbafrani <[hidden email]> wrote:

Hi, I want to use Yarn as cluster manager for running Flink applications, but I'm worried about how Flink or Yarn handle local logs in each machine. Does they clean aged logs for a long-running application? If not, it's possible the local storage get full!!!

--