Docker: multi kafka brokers failed
I would like to use docker with several kafka brokers.
So I started some tests with this docker image => https://github.com/ches/docker-kafka
For one broker all work fine:
I start my zookeeper:
docker run -d –name zookeeper jplock/zookeeper:3.4.6
Then I start a kafka instance:
docker run -d –name kafka –link zookeeper:zookeeper ches/kafka
When I create topic and messages, all work.
Now I create a second kafka instance:
docker run -d –name kafka2 –link zookeeper:zookeeper –expose 9093 –env-file env ches/kafka
I change the exposed port to 9093 and environment variables:
The broker starts well and I can create a topic with replication:
docker -D run –rm ches/kafka kafka-topics.sh –create –topic test2 –replication-factor 2 –partitions 1 –zookeeper $ZK_IP:2181
Now when I want to send some messages:
docker run –rm –interactive ches/kafka kafka-console-producer.sh –topic test2 –broker-list $KAFKA_IP:9092
I get this error:
ERROR Producer connection to 172.17.0.17:9093 unsuccessful
(kafka.producer.SyncProducer) java.net.ConnectException: Connection
A docker ps give me that:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES d7bba0f3d0db ches/kafka:latest "kafka-console-produ About a minute ago Up About a minute 9092/tcp, 7203/tcp sick_shockley 9c475a659383 ches/kafka:latest "/start.sh" 4 minutes ago Up 4 minutes 7203/tcp, 9092/tcp, 9093/tcp kafka2 63aae4c539ab ches/kafka:latest "/start.sh" 28 minutes ago Up 28 minutes 7203/tcp, 9092/tcp kafka ab560690e0e7 jplock/zookeeper:3.4.6 "/opt/zookeeper/bin/ 28 minutes ago Up 28 minutes 2181/tcp, 2888/tcp, 3888/tcp zookeeper
So kafka2 seems started on 9093 port.
Why I get this error ?
One Solution collect form web for “Docker: multi kafka brokers failed”
Check in the zookeeper docker what is the advertised host name of the kafka dockers. It’s very possible that they registered their docker hash as host name (as it’s the result of getInetAdress() within a docker container) instead of a resolvable address.
If that’s the case, editing your standard kafka config to change advertised.host.name should solve your problem (it’s a bit annoying because you have to change it at start, but you can for example fetch it from the /etc/hosts file of the docker container at startup, it should be the first half of the first line in it).