reading from jdbc connection

[Arran Duff]

Hi, 

I'm quite new to flink and I'm trying to create an application, which reads ID's from a kinesis stream and then uses these to read from a mysql database. I expect that I would just be doing a join of the id's onto the table

I'm struggling to understand from the documentation how to actually connect to jdbc from flink using java.  For example - the code shown here

doesn't give any information about what to provide as arguments in the connect method. Reading the javadoc I can see that it needs to be a ConnectorDescriptor object. And I can see that the known subclasses include a HBase, Kafka and Hbase connector. But I don't see one for JDBC. Should I just be using the CustomConnectorDescriptor and adding JDBC connection options? Will this work out of the box or am I going down a rabbit hole?

I also note that all of the examples that I see for the jdbc connector are written in SQL, or DDL or yaml - for example here https://ci.apache.org/projects/flink/flink-docs-release-1.12/dev/table/connect.html#jdbc-connector
I'm not quite sure how I would go about getting that working in java. Any help would be greatly appreciated

Thanks,

Arran

[rmetzger0]

Hey Arran,

It seems that the preferred way, even in the Java API is to use a DDL statement: https://github.com/apache/flink/blob/master/flink-table/flink-table-api-java-bridge/src/main/java/org/apache/flink/table/api/bridge/java/StreamTableEnvironment.java#L602-L639

Hope this helps!

Best,

Robert

On Thu, Mar 25, 2021 at 2:40 PM Arran Duff <[hidden email]> wrote:

Hi, 

I'm quite new to flink and I'm trying to create an application, which reads ID's from a kinesis stream and then uses these to read from a mysql database. I expect that I would just be doing a join of the id's onto the table

I'm struggling to understand from the documentation how to actually connect to jdbc from flink using java.  For example - the code shown here

doesn't give any information about what to provide as arguments in the connect method. Reading the javadoc I can see that it needs to be a ConnectorDescriptor object. And I can see that the known subclasses include a HBase, Kafka and Hbase connector. But I don't see one for JDBC. Should I just be using the CustomConnectorDescriptor and adding JDBC connection options? Will this work out of the box or am I going down a rabbit hole?

I also note that all of the examples that I see for the jdbc connector are written in SQL, or DDL or yaml - for example here https://ci.apache.org/projects/flink/flink-docs-release-1.12/dev/table/connect.html#jdbc-connector
I'm not quite sure how I would go about getting that working in java. Any help would be greatly appreciated

Thanks,

Arran