Re: Can JDBCSinkFunction support exectly once?

Posted by Fabian Hueske-2 on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Can-JDBCSinkFunction-support-exectly-once-tp24636p24641.html

Hi,

JDBCSinkFunction is a simple wrapper around the JDBCOutputFormat (the DataSet / Batch API output interface).

Dominik is right, that JDBCSinkFunction does not support exactly-once output.

It is not strictly required that an exactly-once sink implements TwoPhaseCommitFunction.

TPCF is a convenience interface that internally use the CheckpointListener interface. It makes it easier to implement exactly-once sinks but is not the only way to do it.

Implementing an exactly-once JDBC sink by extending TwoPhaseCommitFunction would require to be able to recover (and commit) an open transaction after a task was restarted.

Not sure if the JDBC interface supports this.

Best, Fabian

Am Mi., 21. Nov. 2018 um 11:27 Uhr schrieb Dominik Wosiński <[hidden email]>:

Hey, 

As far as I know, the function needs to implement the TwoPhaseCommitFunction and not the CheckpointListener. JDBCSinkFunction does not implement the two-phase commit, so currently it does not support exactly once.

Best Regards,

Dom.

śr., 21 lis 2018 o 11:07 Jocean shi <[hidden email]> napisał(a):

Hi,

Can JDBCSinkFunction support exectly once? Is it that The JDBCSinkFunction dont't implement CheckpointListener meaning JDBCSinkFunction dont't support exectly once?

cheers

Jocean