Classes missing from jar

[Jason Kania]

Hello,

I am just getting started with Flink and am attempting to use the kafka connector.

In particular I am attempting to use the jar flink-connector-kafka-0.11_2.11-1.4.0.jar downloaded from:

https://repo1.maven.org/maven2/org/apache/flink/flink-connector-kafka-0.11_2.11/1.4.0/

with the latest flink-dist_2.11-1.4.0.jar that was downloaded from the main Flink site.

I attempted to use the constructor:

public FlinkKafkaConsumer011(String topic, KeyedDeserializationSchema<T> deserializer, Properties props)

but the IDE indicated that the KeyedDeserializationSchema interface was not defined.

I see that it should be defined as it is contained in the directory:

https://github.com/apache/flink/tree/release-1.4/flink-connectors/flink-connector-kafka-base/src/main/java/org/apache/flink/streaming/util/serialization

and the flink-dist_2.11-1.4.0.jar does contain that path but only contains the classes:

I am guessing I have a mismatch of some sort, but the versions match so I am not sure what to check.

Is the flink connector class in the maven repo old or is there a different location from which it should be retrieved?

Any other suggestions?

Thanks,

Jason

[Tzu-Li (Gordon) Tai]

Hi Jason,

The KeyedDeserializationSchema is located in the flink-connector-kafka-base
module, so you'll need to include the jar for that too [1].

Cheers,
Gordon

[1]
https://repo1.maven.org/maven2/org/apache/flink/flink-connector-kafka-base_2.11/1.4.0/





--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[Jason Kania]

Thanks. That resolved it. Also had to pull in the kafka 10 and 9 versions of the connector jars. Once the base jar is in the mvn repository, this won't be as problematic.

On Friday, January 12, 2018, 9:46:22 AM EST, Tzu-Li (Gordon) Tai <[hidden email]> wrote:

Hi Jason,

The KeyedDeserializationSchema is located in the flink-connector-kafka-base
module, so you'll need to include the jar for that too [1].

Cheers,
Gordon

[1]

https://repo1.maven.org/maven2/org/apache/flink/flink-connector-kafka-base_2.11/1.4.0/





--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[Tzu-Li (Gordon) Tai]

Hi Jason,

You actually should not be adding the flink-dist jar as a dependency in your application.

It seems like you are not using a build tool for your application, but adding dependencies manually.

In general, I would recommend build management tools like Maven / Gradle for building Java applications.

That enables dependencies to be automatically included when building the application.

Flink also ships a quickstart Maven archetype, which has all the project build configurations setup properly out-of-the-box.

See here [1] for details. It’s a nice walkthrough of setting up your Flink project using the Java API :)

Cheers,

Gordon

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.4/quickstart/java_api_quickstart.html

On 13 January 2018 at 1:28:54 AM, Jason Kania ([hidden email]) wrote:

Thanks. That resolved it. Also had to pull in the kafka 10 and 9 versions of the connector jars. Once the base jar is in the mvn repository, this won't be as problematic.

On Friday, January 12, 2018, 9:46:22 AM EST, Tzu-Li (Gordon) Tai <[hidden email]> wrote:

Hi Jason,

The KeyedDeserializationSchema is located in the flink-connector-kafka-base
module, so you'll need to include the jar for that too [1].

Cheers,
Gordon

[1]

https://repo1.maven.org/maven2/org/apache/flink/flink-connector-kafka-base_2.11/1.4.0/





--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/