TableFactory support for WriteMode.OVERWRITE

[Flavio Pompermaier]

Hi to all,

I was trying to use the new Table API to write into a CSV sink but I discovered that I can't set the write mode to WriteMode.OVERWRITE. Is this a missing feature or is it specifically designed? Is there any good reason to not allow overwrite of file system connectors?

Best,

Flavio

[Jark Wu-3]

Hi Flavio,

That's true. The OVERWRITE mode s not exposed to csv table sink. There is already a JIRA issue discussed this FLINK-15066 [1].

I think we may support INSERT OVERWRITE for csv sink in 1.11 instead of passing a connector property to the sink factory.

Best,

Jark 

[1]: https://issues.apache.org/jira/browse/FLINK-15066

On Thu, 13 Feb 2020 at 21:56, Flavio Pompermaier <[hidden email]> wrote:

Hi to all,

I was trying to use the new Table API to write into a CSV sink but I discovered that I can't set the write mode to WriteMode.OVERWRITE. Is this a missing feature or is it specifically designed? Is there any good reason to not allow overwrite of file system connectors?

Best,

Flavio

[Jingsong Li]

Hi Flavio,

Csv has many problems...

- Old csv is not standard csv, we need use new csv..

- FileSystem in SQL, insert into need be append, insert overwrite need be overwrite.

- FileSystem with csv need support partition.

Hope these things can be resolved in 1.11.

Best,

Jingsong Lee

On Fri, Feb 14, 2020 at 10:44 AM Jark Wu <[hidden email]> wrote:

Hi Flavio,

That's true. The OVERWRITE mode s not exposed to csv table sink. There is already a JIRA issue discussed this FLINK-15066 [1].

I think we may support INSERT OVERWRITE for csv sink in 1.11 instead of passing a connector property to the sink factory.

Best,

Jark 

[1]: https://issues.apache.org/jira/browse/FLINK-15066

On Thu, 13 Feb 2020 at 21:56, Flavio Pompermaier <[hidden email]> wrote:

Hi to all,

I was trying to use the new Table API to write into a CSV sink but I discovered that I can't set the write mode to WriteMode.OVERWRITE. Is this a missing feature or is it specifically designed? Is there any good reason to not allow overwrite of file system connectors?

Best,

Flavio

--