Support of custom JDBC dialects in flink-jdbc

[Dongwon Kim-2]

Hi, I tried flink-jdbc [1] to read data from Druid because Druid implements Calcite Avatica [2], but the connection string, jdbc:avatica:remote:url=http://BROKER:8082/druid/v2/sql/avatica/, is not supported by any of JDBCDialects [3].

I implement custom JDBCDialect [4], custom StreamTableSourceFactory [5], etc in my own repository [6], but it seems overkill.

How about supporting custom JDBC dialects in flink-jdbc based on SPI?

Best,

- Dongwon

[1] https://ci.apache.org/projects/flink/flink-docs-stable/dev/table/connect.html#jdbc-connector

[2] https://druid.apache.org/docs/latest/querying/sql.html#jdbc

[3] https://github.com/apache/flink/blob/master/flink-connectors/flink-jdbc/src/main/java/org/apache/flink/api/java/io/jdbc/dialect/JDBCDialects.java

[4] https://github.com/eastcirclek/flink-avatica-source/blob/master/src/main/java/io/github/eastcirclek/AvaticaJDBCDialect.java

[5] https://github.com/eastcirclek/flink-avatica-source/blob/master/src/main/java/io/github/eastcirclek/AvaticaJDBCSourceFactory.java

[6] https://github.com/eastcirclek/flink-avatica-source

[Jark Wu-3]

Hi Dongwon,

I saw many requirements on this and I'm big +1 for this. 

I created https://issues.apache.org/jira/browse/FLINK-16833 to track this effort. Hope this can be done before 1.11 release. 

Best,

Jark

On Fri, 27 Mar 2020 at 22:22, Dongwon Kim <[hidden email]> wrote:

Hi, I tried flink-jdbc [1] to read data from Druid because Druid implements Calcite Avatica [2], but the connection string, jdbc:avatica:remote:url=http://BROKER:8082/druid/v2/sql/avatica/, is not supported by any of JDBCDialects [3].

I implement custom JDBCDialect [4], custom StreamTableSourceFactory [5], etc in my own repository [6], but it seems overkill.

How about supporting custom JDBC dialects in flink-jdbc based on SPI?

Best,

- Dongwon

[1] https://ci.apache.org/projects/flink/flink-docs-stable/dev/table/connect.html#jdbc-connector

[2] https://druid.apache.org/docs/latest/querying/sql.html#jdbc

[3] https://github.com/apache/flink/blob/master/flink-connectors/flink-jdbc/src/main/java/org/apache/flink/api/java/io/jdbc/dialect/JDBCDialects.java

[4] https://github.com/eastcirclek/flink-avatica-source/blob/master/src/main/java/io/github/eastcirclek/AvaticaJDBCDialect.java

[5] https://github.com/eastcirclek/flink-avatica-source/blob/master/src/main/java/io/github/eastcirclek/AvaticaJDBCSourceFactory.java

[6] https://github.com/eastcirclek/flink-avatica-source

[Dongwon Kim-2]

Hi Jark,

Many thanks for creating the issue on Jira and nice summarization :-)

Best,

Dongwon

On Sat, Mar 28, 2020 at 12:37 AM Jark Wu <[hidden email]> wrote:

Hi Dongwon,

I saw many requirements on this and I'm big +1 for this. 

I created https://issues.apache.org/jira/browse/FLINK-16833 to track this effort. Hope this can be done before 1.11 release. 

Best,

Jark

On Fri, 27 Mar 2020 at 22:22, Dongwon Kim <[hidden email]> wrote:

Hi, I tried flink-jdbc [1] to read data from Druid because Druid implements Calcite Avatica [2], but the connection string, jdbc:avatica:remote:url=http://BROKER:8082/druid/v2/sql/avatica/, is not supported by any of JDBCDialects [3].

I implement custom JDBCDialect [4], custom StreamTableSourceFactory [5], etc in my own repository [6], but it seems overkill.

How about supporting custom JDBC dialects in flink-jdbc based on SPI?

Best,

- Dongwon

[1] https://ci.apache.org/projects/flink/flink-docs-stable/dev/table/connect.html#jdbc-connector

[2] https://druid.apache.org/docs/latest/querying/sql.html#jdbc

[3] https://github.com/apache/flink/blob/master/flink-connectors/flink-jdbc/src/main/java/org/apache/flink/api/java/io/jdbc/dialect/JDBCDialects.java

[4] https://github.com/eastcirclek/flink-avatica-source/blob/master/src/main/java/io/github/eastcirclek/AvaticaJDBCDialect.java

[5] https://github.com/eastcirclek/flink-avatica-source/blob/master/src/main/java/io/github/eastcirclek/AvaticaJDBCSourceFactory.java

[6] https://github.com/eastcirclek/flink-avatica-source