Deleting files in continuous processing

[Mohit Anchlia]

Is there a way to delete a file once it has been processed?

streamEnv

.readFile(format, args[0], FileProcessingMode.PROCESS_CONTINUOUSLY, 2000)

[Mohit Anchlia]

Just checking to see if there is a way to purge files after it's processed.

On Tue, Aug 15, 2017 at 5:11 PM, Mohit Anchlia <[hidden email]> wrote:

Is there a way to delete a file once it has been processed?

streamEnv

.readFile(format, args[0], FileProcessingMode.PROCESS_CONTINUOUSLY, 2000)

[Till Rohrmann]

Hi Mohit,

Flink does not support this behaviour out of the box afaik. I think you have to write your own source function or extend ContinuousFileMonitoringFunction in order to do that.

Cheers,
Till

​

On Mon, Aug 21, 2017 at 11:07 PM, Mohit Anchlia <[hidden email]> wrote:

Just checking to see if there is a way to purge files after it's processed.

On Tue, Aug 15, 2017 at 5:11 PM, Mohit Anchlia <[hidden email]> wrote:

Is there a way to delete a file once it has been processed?

streamEnv

.readFile(format, args[0], FileProcessingMode.PROCESS_CONTINUOUSLY, 2000)