Re: Existing files and directories are not overwritten in NO_OVERWRITE mode. Use OVERWRITE mode to overwrite existing files and directories.

Posted by Jörn Franke on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Existing-files-and-directories-are-not-overwritten-in-NO-OVERWRITE-mode-Use-OVERWRITE-mode-to-overwr-tp21130p21146.html

import org.apache.flink.core.fs.FileSystem

On 3. Jul 2018, at 08:13, Mich Talebzadeh <[hidden email]> wrote:

thanks Hequn.

When I use as suggested, I am getting this error

error] /home/hduser/dba/bin/flink/md_streaming/src/main/scala/myPackage/md_streaming.scala:30: not found: value FileSystem
[error]                  .writeAsText("/tmp/md_streaming.txt", FileSystem.WriteMode.OVERWRITE)
[error]                                                        ^
[error] one error found

On Tue, 3 Jul 2018 at 03:16, Hequn Cheng <[hidden email]> wrote:

Hi Mich, 

It seems the writeMode has not been set correctly. Have you ever tried 

.writeAsText("/tmp/md_streaming.txt", FileSystem.WriteMode.OVERWRITE);

On Mon, Jul 2, 2018 at 10:44 PM, Mich Talebzadeh <[hidden email]> wrote:

Flink 1.5

This streaming data written to a file

    val stream = env
                 .addSource(new FlinkKafkaConsumer09[String]("md", new SimpleStringSchema(), properties))
                 .writeAsText("/tmp/md_streaming.txt")
 env.execute("Flink Kafka Example")

The error states

Caused by: java.io.IOException: File or directory /tmp/md_streaming.txt already exists. Existing files and directories are not overwritten in NO_OVERWRITE mode. Use OVERWRITE mode to overwrite existing files and directories.

Is there any append in writeAsText? I tried OVERWRITE but  did not work.

Thanks