local dev using kafka consumer by docker got wrong cluster node id

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

local dev using kafka consumer by docker got wrong cluster node id

andy
So I want to run flink in my local. Kafka docker and its zookeeper has been work great for local dev of other projects, I want to try this kafka with new flink project in local.
I have problem of first, the connect from my kafka consumer source is created but then it try to connect with a different node while the flink job running.
Example:
- First it connect to node 0.0.0.0:9092(id: -1 rack: null)
- Then It try to connect nodes = [4c34977feb35:9092 (id: 1001 rack: null)]

+ the value 4c34977feb35 is the docker container name of my kafka docker
+ the value "0.0.0.0:9092” is provided by flink job code

I already try to setup another kafka cluster with different name and the flink consumer is still somehow can find it which end up can not connect to it

My env:
- docker for mac
- flink 1.7.0
- scala 2.12

Log file is include here:

log.txt (6K) Download Attachment