Why is Hive dependency flink-sql-connector-hive not available on Maven Central?

[Yik San Chan]

The question is cross-posted in StackOverflow https://stackoverflow.com/questions/66914119/flink-why-is-hive-dependency-flink-sql-connector-hive-not-available-on-maven-ce

According to [Flink SQL Hive: Using bundled hive jar](https://ci.apache.org/projects/flink/flink-docs-stable/dev/table/connectors/hive/#using-bundled-hive-jar):

> The following tables list all available bundled hive jars. You can pick one to the /lib/ directory in Flink distribution.
> - flink-sql-connector-hive-1.2.2 (download link)
> - flink-sql-connector-hive-2.2.0 (download link)
> ...

However, these dependencies are not available from Maven central. As a work around, I use [user defined dependencies](https://ci.apache.org/projects/flink/flink-docs-stable/dev/table/connectors/hive/#user-defined-dependencies), but this is not recommended:

> the recommended way to add dependency is to use a bundled jar. Separate jars should be used only if bundled jars don’t meet your needs.

I wonder why the bundle jars are not available in Maven central?

Follow-up: Since they are not available from Maven central, I wonder how to include them in pom.xml in order to run `mvn package`?

Thanks!

[Tzu-Li (Gordon) Tai]

Hi,

I'm pulling in Rui Li (cc'ed) who might be able to help you here as he actively maintains the hive connectors.

Cheers,
Gordon

On Fri, Apr 2, 2021 at 11:36 AM Yik San Chan <[hidden email]> wrote:

The question is cross-posted in StackOverflow https://stackoverflow.com/questions/66914119/flink-why-is-hive-dependency-flink-sql-connector-hive-not-available-on-maven-ce

According to [Flink SQL Hive: Using bundled hive jar](https://ci.apache.org/projects/flink/flink-docs-stable/dev/table/connectors/hive/#using-bundled-hive-jar):

> The following tables list all available bundled hive jars. You can pick one to the /lib/ directory in Flink distribution.
> - flink-sql-connector-hive-1.2.2 (download link)
> - flink-sql-connector-hive-2.2.0 (download link)
> ...

However, these dependencies are not available from Maven central. As a work around, I use [user defined dependencies](https://ci.apache.org/projects/flink/flink-docs-stable/dev/table/connectors/hive/#user-defined-dependencies), but this is not recommended:

> the recommended way to add dependency is to use a bundled jar. Separate jars should be used only if bundled jars don’t meet your needs.

I wonder why the bundle jars are not available in Maven central?

Follow-up: Since they are not available from Maven central, I wonder how to include them in pom.xml in order to run `mvn package`?

Thanks!

[Yik San Chan]

Hi,

I am able to find the jar from Maven central. See updates in the StackOverflow post.

Thank you!

Best,

Yik San

On Tue, Apr 6, 2021 at 4:05 PM Tzu-Li (Gordon) Tai <[hidden email]> wrote:

Hi,

I'm pulling in Rui Li (cc'ed) who might be able to help you here as he actively maintains the hive connectors.

Cheers,
Gordon

On Fri, Apr 2, 2021 at 11:36 AM Yik San Chan <[hidden email]> wrote:

The question is cross-posted in StackOverflow https://stackoverflow.com/questions/66914119/flink-why-is-hive-dependency-flink-sql-connector-hive-not-available-on-maven-ce

According to [Flink SQL Hive: Using bundled hive jar](https://ci.apache.org/projects/flink/flink-docs-stable/dev/table/connectors/hive/#using-bundled-hive-jar):

> The following tables list all available bundled hive jars. You can pick one to the /lib/ directory in Flink distribution.
> - flink-sql-connector-hive-1.2.2 (download link)
> - flink-sql-connector-hive-2.2.0 (download link)
> ...

However, these dependencies are not available from Maven central. As a work around, I use [user defined dependencies](https://ci.apache.org/projects/flink/flink-docs-stable/dev/table/connectors/hive/#user-defined-dependencies), but this is not recommended:

> the recommended way to add dependency is to use a bundled jar. Separate jars should be used only if bundled jars don’t meet your needs.

I wonder why the bundle jars are not available in Maven central?

Follow-up: Since they are not available from Maven central, I wonder how to include them in pom.xml in order to run `mvn package`?

Thanks!

[Rui Li]

Hi Yik San,

Glad to know you've found the jar. Another option to locate the jar is to just use maven dependency plugin like this:

mvn dependency:get -Dartifact=org.apache.flink:flink-sql-connector-hive-2.3.6_2.12:1.12.2

On Tue, Apr 6, 2021 at 4:10 PM Yik San Chan <[hidden email]> wrote:

Hi,

I am able to find the jar from Maven central. See updates in the StackOverflow post.

Thank you!

Best,

Yik San

On Tue, Apr 6, 2021 at 4:05 PM Tzu-Li (Gordon) Tai <[hidden email]> wrote:

Hi,

I'm pulling in Rui Li (cc'ed) who might be able to help you here as he actively maintains the hive connectors.

Cheers,
Gordon

On Fri, Apr 2, 2021 at 11:36 AM Yik San Chan <[hidden email]> wrote:

The question is cross-posted in StackOverflow https://stackoverflow.com/questions/66914119/flink-why-is-hive-dependency-flink-sql-connector-hive-not-available-on-maven-ce

According to [Flink SQL Hive: Using bundled hive jar](https://ci.apache.org/projects/flink/flink-docs-stable/dev/table/connectors/hive/#using-bundled-hive-jar):

> The following tables list all available bundled hive jars. You can pick one to the /lib/ directory in Flink distribution.
> - flink-sql-connector-hive-1.2.2 (download link)
> - flink-sql-connector-hive-2.2.0 (download link)
> ...

However, these dependencies are not available from Maven central. As a work around, I use [user defined dependencies](https://ci.apache.org/projects/flink/flink-docs-stable/dev/table/connectors/hive/#user-defined-dependencies), but this is not recommended:

> the recommended way to add dependency is to use a bundled jar. Separate jars should be used only if bundled jars don’t meet your needs.

I wonder why the bundle jars are not available in Maven central?

Follow-up: Since they are not available from Maven central, I wonder how to include them in pom.xml in order to run `mvn package`?

Thanks!

--

[Yik San Chan]

Thanks for the tip! 

On Tue, Apr 6, 2021 at 4:25 PM Rui Li <[hidden email]> wrote:

Hi Yik San,

Glad to know you've found the jar. Another option to locate the jar is to just use maven dependency plugin like this:

mvn dependency:get -Dartifact=org.apache.flink:flink-sql-connector-hive-2.3.6_2.12:1.12.2

On Tue, Apr 6, 2021 at 4:10 PM Yik San Chan <[hidden email]> wrote:

Hi,

I am able to find the jar from Maven central. See updates in the StackOverflow post.

Thank you!

Best,

Yik San

On Tue, Apr 6, 2021 at 4:05 PM Tzu-Li (Gordon) Tai <[hidden email]> wrote:

Hi,

I'm pulling in Rui Li (cc'ed) who might be able to help you here as he actively maintains the hive connectors.

Cheers,
Gordon

On Fri, Apr 2, 2021 at 11:36 AM Yik San Chan <[hidden email]> wrote:

The question is cross-posted in StackOverflow https://stackoverflow.com/questions/66914119/flink-why-is-hive-dependency-flink-sql-connector-hive-not-available-on-maven-ce

According to [Flink SQL Hive: Using bundled hive jar](https://ci.apache.org/projects/flink/flink-docs-stable/dev/table/connectors/hive/#using-bundled-hive-jar):

> The following tables list all available bundled hive jars. You can pick one to the /lib/ directory in Flink distribution.
> - flink-sql-connector-hive-1.2.2 (download link)
> - flink-sql-connector-hive-2.2.0 (download link)
> ...

However, these dependencies are not available from Maven central. As a work around, I use [user defined dependencies](https://ci.apache.org/projects/flink/flink-docs-stable/dev/table/connectors/hive/#user-defined-dependencies), but this is not recommended:

> the recommended way to add dependency is to use a bundled jar. Separate jars should be used only if bundled jars don’t meet your needs.

I wonder why the bundle jars are not available in Maven central?

Follow-up: Since they are not available from Maven central, I wonder how to include them in pom.xml in order to run `mvn package`?

Thanks!

--

Best regards!

Rui Li