Re: Can JDBCSinkFunction support exectly once?
Posted by
Jocean shi on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Can-JDBCSinkFunction-support-exectly-once-tp24636p24645.html
Hi,
Thanks for help.
I find that many sink dont't support stream exectly-once. I need use exectly-once Hbase sink and Mysql sink in my work.
I try to contribute to Hbase Sink firstly
Best Regards,
Jocean
Hi,
JDBCSinkFunction is a simple wrapper around the JDBCOutputFormat (the DataSet / Batch API output interface).
Dominik is right, that JDBCSinkFunction does not support exactly-once output.
It is not strictly required that an exactly-once sink implements TwoPhaseCommitFunction.
TPCF is a convenience interface that internally use the CheckpointListener interface. It makes it easier to implement exactly-once sinks but is not the only way to do it.
Implementing an exactly-once JDBC sink by extending TwoPhaseCommitFunction would require to be able to recover (and commit) an open transaction after a task was restarted.
Not sure if the JDBC interface supports this.
Best, Fabian
Am Mi., 21. Nov. 2018 um 11:27 Uhr schrieb Dominik Wosiński <
[hidden email]>:
Hey,
As far as I know, the function needs to implement the
TwoPhaseCommitFunction and not the
CheckpointListener. JDBCSinkFunction does not implement the two-phase commit, so currently it does not support exactly once.
Best Regards,
Dom.
Hi,
Can JDBCSinkFunction support exectly once? Is it that The JDBCSinkFunction dont't implement CheckpointListener meaning JDBCSinkFunction dont't support exectly once?
cheers
Jocean