JDBC connection pools

[Marco Villalobos-2]

Currently, my jobs that require JDBC initialize a connection in the open method directly via JDBC driver.

1. What are the established best practices for this?

2. Is it better to use a connection pool that can validate the connection and reconnect?

3. Would each operator require its own connection pool?  

I'd like the communities thought on this topic.

[Matthias]

Hi Marco,

have you had a look into the connector documentation ([1] for the regular connector or [2] for the SQL connector)? Maybe, discussions about connection pooling in [3] and [4] or the code snippets provided in the JavaDoc of JdbcInputFormat [5] help as well.

Best,

Matthias

[1] https://ci.apache.org/projects/flink/flink-docs-stable/dev/connectors/jdbc.html

[2] https://ci.apache.org/projects/flink/flink-docs-stable/dev/table/connectors/jdbc.html

[3] http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/How-to-handle-JDBC-connections-in-a-topology-td29004.html

[4] http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Should-I-use-static-database-connection-pool-td30537.html

[5] https://github.com/apache/flink/blob/c6997c97c575d334679915c328792b8a3067cfb5/flink-connectors/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/JdbcInputFormat.java#L55

On Thu, Jan 21, 2021 at 8:19 PM Marco Villalobos <[hidden email]> wrote:

Currently, my jobs that require JDBC initialize a connection in the open method directly via JDBC driver.

1. What are the established best practices for this?

2. Is it better to use a connection pool that can validate the connection and reconnect?

3. Would each operator require its own connection pool?  

I'd like the communities thought on this topic.