JDBC sink for streams API

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

JDBC sink for streams API

Eduardo Winpenny Tejedor
Hi all,

Could someone point me to the current advised way of adding a JDBC sink? 

Online I've seen one can DataStream.writeUsingOutputFormat() however I see the OutputFormatSinkFunction is deprecated. 

I can also see a JDBCSinkFunction (or JDBCUpsertSinkFunction) but that is "package private" so I can't use it myself. JDBCAppendSinkTable makes use of it but that doesn't work with the Streams API...I tried simply copying its code into a class of my own alas the flush method in JDBCOutputFormat is also "package private"...

JDBC does not receive a lot of attention in Fabian's and Vasili's Stream Processing with Apache Flink or in the online documentation - is there a reason for this? Is it bad practice?


Regards,
Eduardo
Reply | Threaded
Open this post in threaded view
|

Re: JDBC sink for streams API

tison
Hi Eduardo,

JDBCSinkFunction is a package-private class which you can make use of
by JDBCAppendTableSink. A typical statement could be

new JDBCAppendTableSink.builder()
                       . ...
                       .build()
                       .consumeDataStream(upstream);

Also JDBCUpsertTableSink and JDBCTableSourceSinkFactory are worth to give a look.

The successor of OutputFormatSinkFunction is StreamingFileSink, but it seems out of
your requirements.

Best,
tison.


Eduardo Winpenny Tejedor <[hidden email]> 于2019年8月13日周二 上午7:59写道:
Hi all,

Could someone point me to the current advised way of adding a JDBC sink? 

Online I've seen one can DataStream.writeUsingOutputFormat() however I see the OutputFormatSinkFunction is deprecated. 

I can also see a JDBCSinkFunction (or JDBCUpsertSinkFunction) but that is "package private" so I can't use it myself. JDBCAppendSinkTable makes use of it but that doesn't work with the Streams API...I tried simply copying its code into a class of my own alas the flush method in JDBCOutputFormat is also "package private"...

JDBC does not receive a lot of attention in Fabian's and Vasili's Stream Processing with Apache Flink or in the online documentation - is there a reason for this? Is it bad practice?


Regards,
Eduardo