Monitoring folder in flink

Posted by flinkuser101 on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Monitoring-folder-in-flink-tp16327.html

I have folder where new files arrive at schedule. Why is my flink readfile not reading new files. I have used but *PROCESS_ONCE* and *PROCESS_CONTINUOUSLY*. When I use *PROCESS_CONTINUOUSLY* it reads the same file but the execution does not terminate whereas for PROCESS_ONCE it terminates in IDE.

    String path = "C:\\test";
   
StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
   
    TextInputFormat format = new TextInputFormat(new org.apache.flink.core.fs.Path(path));
   
    DataStream<String> inputStream = env.readFile(format, path, FileProcessingMode.PROCESS_ONCE, 100);