Exactly once semantics for hdfs sink

[Nick Bendtner]

Hi all,

I want to use the StreamingFile sink for writing data to hdfs. Can I achieve exactly once semantics with this sink ? 

Best,

HW. 

[r_khachatryan]

Hi Vishwas,

Yes, Streaming File Sink does support exactly-once semantics and can be used with HDFS.

On Mon, Feb 10, 2020 at 5:20 PM Vishwas Siravara <[hidden email]> wrote:

Hi all,

I want to use the StreamingFile sink for writing data to hdfs. Can I achieve exactly once semantics with this sink ? 

Best,

HW. 

[Nick Bendtner]

Hi Khachatryan,

Thanks for your reply. Can you help me understand how it works with hdfs specifically , even a link to a document will help. 

Best,

Vishwas 

On Mon, Feb 10, 2020 at 10:32 AM Khachatryan Roman <[hidden email]> wrote:

Hi Vishwas,

Yes, Streaming File Sink does support exactly-once semantics and can be used with HDFS.

Regards,
Roman

On Mon, Feb 10, 2020 at 5:20 PM Vishwas Siravara <[hidden email]> wrote:

Hi all,

I want to use the StreamingFile sink for writing data to hdfs. Can I achieve exactly once semantics with this sink ? 

Best,

HW. 

[Zhijiang(wangzhijiang999)]

Hi Vishwas,

I guess this link [1] can help understand how it works and how to use in practice for StreamingFileSink.

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.10/dev/connectors/streamfile_sink.html

Best,

Zhijiang

------------------------------------------------------------------

From:Vishwas Siravara <[hidden email]>

Send Time:2020 Feb. 12 (Wed.) 05:19

To:Khachatryan Roman <[hidden email]>

Cc:user <[hidden email]>

Subject:Re: Exactly once semantics for hdfs sink

Hi Khachatryan,

Thanks for your reply. Can you help me understand how it works with hdfs specifically , even a link to a document will help. 

Best,

Vishwas 

On Mon, Feb 10, 2020 at 10:32 AM Khachatryan Roman <[hidden email]> wrote:

Hi Vishwas,

Yes, Streaming File Sink does support exactly-once semantics and can be used with HDFS.

Regards,
Roman

On Mon, Feb 10, 2020 at 5:20 PM Vishwas Siravara <[hidden email]> wrote:

Hi all,

I want to use the StreamingFile sink for writing data to hdfs. Can I achieve exactly once semantics with this sink ? 

Best,

HW. 

[r_khachatryan]

Hi Vishwas,

Please let me know if you have any specific questions about the StreamingFile sink.

On Wed, Feb 12, 2020 at 4:45 AM Zhijiang <[hidden email]> wrote:

Hi Vishwas,

I guess this link [1] can help understand how it works and how to use in practice for StreamingFileSink.

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.10/dev/connectors/streamfile_sink.html

Best,

Zhijiang

------------------------------------------------------------------

From:Vishwas Siravara <[hidden email]>

Send Time:2020 Feb. 12 (Wed.) 05:19

To:Khachatryan Roman <[hidden email]>

Cc:user <[hidden email]>

Subject:Re: Exactly once semantics for hdfs sink

Hi Khachatryan,

Thanks for your reply. Can you help me understand how it works with hdfs specifically , even a link to a document will help. 

Best,

Vishwas 

On Mon, Feb 10, 2020 at 10:32 AM Khachatryan Roman <[hidden email]> wrote:

Hi Vishwas,

Yes, Streaming File Sink does support exactly-once semantics and can be used with HDFS.

Regards,
Roman

On Mon, Feb 10, 2020 at 5:20 PM Vishwas Siravara <[hidden email]> wrote:

Hi all,

I want to use the StreamingFile sink for writing data to hdfs. Can I achieve exactly once semantics with this sink ? 

Best,

HW.