Unable to Read from Kafka [ zookeeper.connect error]

[subashbasnet]

Hello all,

I ran the WriteIntoKafka.java/WikipediaAnalysis.java from flink-streaming examples and able to view the output stream in the terminal via following command:
>bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic wiki-result
But I am unable to read the same stream written to Kafka via ReadFromKafKa.java.
I pass the following as my arguments while running ReadFromKafKa.java:

--topic wiki-result --bootstrap.servers localhost:9092 --group.id myconsumer

I get the following error:

Exception in thread "main" java.lang.IllegalArgumentException: Required property 'zookeeper.connect' has not been set in the properties

at org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer08.validateZooKeeperConfig(FlinkKafkaConsumer08.java:581)

at org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer08.<init>(FlinkKafkaConsumer08.java:215)

at org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer08.<init>(FlinkKafkaConsumer08.java:193)

at org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer08.<init>(FlinkKafkaConsumer08.java:160)

at org.apache.flink.streaming.examples.kafka.ReadFromKafka.main(ReadFromKafka.java:54)

How to set property 'zookeeper.connect', or what am I doing wrong?

Best Regards,
Subash Basnet

[rmetzger0]

Hi,

you can set the property by passing it as an argument when starting the job: --zookeeper.connect localhost:2181

On Wed, Mar 16, 2016 at 3:17 PM, subash basnet <[hidden email]> wrote:

Hello all,

I ran the WriteIntoKafka.java/WikipediaAnalysis.java from flink-streaming examples and able to view the output stream in the terminal via following command:
>bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic wiki-result
But I am unable to read the same stream written to Kafka via ReadFromKafKa.java.
I pass the following as my arguments while running ReadFromKafKa.java:

--topic wiki-result --bootstrap.servers localhost:9092 --group.id myconsumer

I get the following error:

Exception in thread "main" java.lang.IllegalArgumentException: Required property 'zookeeper.connect' has not been set in the properties

at org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer08.validateZooKeeperConfig(FlinkKafkaConsumer08.java:581)

at org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer08.<init>(FlinkKafkaConsumer08.java:215)

at org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer08.<init>(FlinkKafkaConsumer08.java:193)

at org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer08.<init>(FlinkKafkaConsumer08.java:160)

at org.apache.flink.streaming.examples.kafka.ReadFromKafka.main(ReadFromKafka.java:54)

How to set property 'zookeeper.connect', or what am I doing wrong?

Best Regards,
Subash Basnet

[stefanobaghino]

Hi Subash,

you just have to add the following parameter: --zookeeper.connect localhost:2181

Let us know if it works before presumably the usage string for the ReadFromKafka example can by incorrect.

Hope I've been helpful.

On Wed, Mar 16, 2016 at 3:17 PM, subash basnet <[hidden email]> wrote:

Hello all,

I ran the WriteIntoKafka.java/WikipediaAnalysis.java from flink-streaming examples and able to view the output stream in the terminal via following command:
>bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic wiki-result
But I am unable to read the same stream written to Kafka via ReadFromKafKa.java.
I pass the following as my arguments while running ReadFromKafKa.java:

--topic wiki-result --bootstrap.servers localhost:9092 --group.id myconsumer

I get the following error:

Exception in thread "main" java.lang.IllegalArgumentException: Required property 'zookeeper.connect' has not been set in the properties

at org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer08.validateZooKeeperConfig(FlinkKafkaConsumer08.java:581)

at org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer08.<init>(FlinkKafkaConsumer08.java:215)

at org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer08.<init>(FlinkKafkaConsumer08.java:193)

at org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer08.<init>(FlinkKafkaConsumer08.java:160)

at org.apache.flink.streaming.examples.kafka.ReadFromKafka.main(ReadFromKafka.java:54)

How to set property 'zookeeper.connect', or what am I doing wrong?

Best Regards,
Subash Basnet

--

[subashbasnet]

Hello Stefano, 
Thank you, it's working :). 

Regards,
Subash Basnet

On Wed, Mar 16, 2016 at 3:24 PM, Stefano Baghino <[hidden email]> wrote:

Hi Subash,

you just have to add the following parameter: --zookeeper.connect localhost:2181

Let us know if it works before presumably the usage string for the ReadFromKafka example can by incorrect.

Hope I've been helpful.

On Wed, Mar 16, 2016 at 3:17 PM, subash basnet <[hidden email]> wrote:

Hello all,

I ran the WriteIntoKafka.java/WikipediaAnalysis.java from flink-streaming examples and able to view the output stream in the terminal via following command:
>bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic wiki-result
But I am unable to read the same stream written to Kafka via ReadFromKafKa.java.
I pass the following as my arguments while running ReadFromKafKa.java:

--topic wiki-result --bootstrap.servers localhost:9092 --group.id myconsumer

I get the following error:

Exception in thread "main" java.lang.IllegalArgumentException: Required property 'zookeeper.connect' has not been set in the properties

at org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer08.validateZooKeeperConfig(FlinkKafkaConsumer08.java:581)

at org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer08.<init>(FlinkKafkaConsumer08.java:215)

at org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer08.<init>(FlinkKafkaConsumer08.java:193)

at org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer08.<init>(FlinkKafkaConsumer08.java:160)

at org.apache.flink.streaming.examples.kafka.ReadFromKafka.main(ReadFromKafka.java:54)

How to set property 'zookeeper.connect', or what am I doing wrong?

Best Regards,
Subash Basnet

--

BR,

Stefano Baghino

Software Engineer @ Radicalbit