Zeppelin: Flink Kafka Connector

[Neil Derraugh]

This is really a Zeppelin question, and I’ve already posted to the user list there.  I’m just trying to draw in as many relevant eyeballs as possible.  If you can help please reply on the Zeppelin mailing list.

In my Zeppelin notebook I’m having a problem importing the Kafka streaming library for Flink.

I added org.apache.flink:flink-connector-kafka_2.11:0.10.2 to the Dependencies on the Flink interpreter.

The Flink interpreter runs code, just not if I have the following import.
import org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer010

I get this error:
<console>:72: error: object FlinkKafkaConsumer010 is not a member of package org.apache.flink.streaming.connectors.kafka
       import org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer010

Am I doing something wrong here?

Neil

[Timo Walther]

You are using an old version of Flink (0.10.2). A FlinkKafkaConsumer010
was not present at that time. You need to upgrade to Flink 1.2.

Timo


Am 17/01/17 um 15:58 schrieb Neil Derraugh:

> This is really a Zeppelin question, and I’ve already posted to the user list there.  I’m just trying to draw in as many relevant eyeballs as possible.  If you can help please reply on the Zeppelin mailing list.
>
> In my Zeppelin notebook I’m having a problem importing the Kafka streaming library for Flink.
>
> I added org.apache.flink:flink-connector-kafka_2.11:0.10.2 to the Dependencies on the Flink interpreter.
>
> The Flink interpreter runs code, just not if I have the following import.
> import org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer010
>
> I get this error:
> <console>:72: error: object FlinkKafkaConsumer010 is not a member of package org.apache.flink.streaming.connectors.kafka
>         import org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer010
>
> Am I doing something wrong here?
>
> Neil

[Fabian Hueske-2]

One thing to add: Flink 1.2.0 has not been release yet.
The FlinkKafkaConsumer010 is only available in a SNAPSHOT release or the first release candidate (RC0).

Best, Fabian

2017-01-17 16:08 GMT+01:00 Timo Walther <[hidden email]>:

You are using an old version of Flink (0.10.2). A FlinkKafkaConsumer010 was not present at that time. You need to upgrade to Flink 1.2.

Timo


Am 17/01/17 um 15:58 schrieb Neil Derraugh:

This is really a Zeppelin question, and I’ve already posted to the user list there.  I’m just trying to draw in as many relevant eyeballs as possible.  If you can help please reply on the Zeppelin mailing list.

In my Zeppelin notebook I’m having a problem importing the Kafka streaming library for Flink.

I added org.apache.flink:flink-connector-kafka_2.11:0.10.2 to the Dependencies on the Flink interpreter.

The Flink interpreter runs code, just not if I have the following import.
import org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer010

I get this error:
<console>:72: error: object FlinkKafkaConsumer010 is not a member of package org.apache.flink.streaming.connectors.kafka
        import org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer010

Am I doing something wrong here?

Neil

[Foster, Craig]

Are connectors being included in the 1.2.0 release or do you mean Kafka specifically?

 

From: Fabian Hueske <[hidden email]>
Reply-To: "[hidden email]" <[hidden email]>
Date: Tuesday, January 17, 2017 at 7:10 AM
To: "[hidden email]" <[hidden email]>
Subject: Re: Zeppelin: Flink Kafka Connector

 

One thing to add: Flink 1.2.0 has not been release yet.
The FlinkKafkaConsumer010 is only available in a SNAPSHOT release or the first release candidate (RC0).

Best, Fabian

 

2017-01-17 16:08 GMT+01:00 Timo Walther <[hidden email]>:

You are using an old version of Flink (0.10.2). A FlinkKafkaConsumer010 was not present at that time. You need to upgrade to Flink 1.2.

Timo


Am 17/01/17 um 15:58 schrieb Neil Derraugh:

 

This is really a Zeppelin question, and I’ve already posted to the user list there.  I’m just trying to draw in as many relevant eyeballs as possible.  If you can help please reply on the Zeppelin mailing list.

In my Zeppelin notebook I’m having a problem importing the Kafka streaming library for Flink.

I added org.apache.flink:flink-connector-kafka_2.11:0.10.2 to the Dependencies on the Flink interpreter.

The Flink interpreter runs code, just not if I have the following import.
import org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer010

I get this error:
<console>:72: error: object FlinkKafkaConsumer010 is not a member of package org.apache.flink.streaming.connectors.kafka
        import org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer010

Am I doing something wrong here?

Neil

 

 

[Fabian Hueske-2]

The connectors are included in the release and available as individual Maven artifacts.

So Flink 1.2.0 will provide a flink-connector-kafka-0.10 artifact (with version 1.2.0).

2017-01-17 16:22 GMT+01:00 Foster, Craig <[hidden email]>:

Are connectors being included in the 1.2.0 release or do you mean Kafka specifically?

 

From: Fabian Hueske <[hidden email]>
Reply-To: "[hidden email]" <[hidden email]>
Date: Tuesday, January 17, 2017 at 7:10 AM
To: "[hidden email]" <[hidden email]>
Subject: Re: Zeppelin: Flink Kafka Connector

 

One thing to add: Flink 1.2.0 has not been release yet.
The FlinkKafkaConsumer010 is only available in a SNAPSHOT release or the first release candidate (RC0).

Best, Fabian

 

2017-01-17 16:08 GMT+01:00 Timo Walther <[hidden email]>:

You are using an old version of Flink (0.10.2). A FlinkKafkaConsumer010 was not present at that time. You need to upgrade to Flink 1.2.

Timo


Am 17/01/17 um 15:58 schrieb Neil Derraugh:

 

This is really a Zeppelin question, and I’ve already posted to the user list there.  I’m just trying to draw in as many relevant eyeballs as possible.  If you can help please reply on the Zeppelin mailing list.

In my Zeppelin notebook I’m having a problem importing the Kafka streaming library for Flink.

I added org.apache.flink:flink-connector-kafka_2.11:0.10.2 to the Dependencies on the Flink interpreter.

The Flink interpreter runs code, just not if I have the following import.
import org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer010

I get this error:
<console>:72: error: object FlinkKafkaConsumer010 is not a member of package org.apache.flink.streaming.connectors.kafka
        import org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer010

Am I doing something wrong here?

Neil

 

 

[Neil Derraugh]

Hi Timo & Fabian,

Thanks for replying.  I'm using Zeppelin built off master.  And Flink 1.2 built off the release-1.2 branch.  Is that the right branch?

Neil

[Neil Derraugh]

I re-read that enough times and it finally made sense. I wasn’t paying attention and thought 0.10.2 was the Kafka version —which hasn’t been released yet either - ha :(.  I switched to a recent version and it’s all good. :)

Thanks !
Neil

> On Jan 17, 2017, at 11:14 AM, Neil Derraugh <[hidden email]> wrote:
>
> Hi Timo & Fabian,
>
> Thanks for replying.  I'm using Zeppelin built off master.  And Flink 1.2
> built off the release-1.2 branch.  Is that the right branch?
>
> Neil
>
>
>
> --
> View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Zeppelin-Flink-Kafka-Connector-tp11113p11119.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.

[Fabian Hueske-2]

Ah, OK :-)

Thanks for reporting back!

Cheers, Fabian

2017-01-17 17:50 GMT+01:00 Neil Derraugh <[hidden email]>:

I re-read that enough times and it finally made sense. I wasn’t paying attention and thought 0.10.2 was the Kafka version —which hasn’t been released yet either - ha :(.  I switched to a recent version and it’s all good. :)

Thanks !
Neil

> On Jan 17, 2017, at 11:14 AM, Neil Derraugh <[hidden email]> wrote:
>
> Hi Timo & Fabian,
>
> Thanks for replying.  I'm using Zeppelin built off master.  And Flink 1.2
> built off the release-1.2 branch.  Is that the right branch?
>
> Neil
>
>
>
> --
> View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Zeppelin-Flink-Kafka-Connector-tp11113p11119.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.