Docker: multi kafka brokers failed

I would like to use docker with several kafka brokers.

So I started some tests with this docker image => https://github.com/ches/docker-kafka

  • Docker Setup with a Mysql Container for a Python App
  • Get 502 Bad Gateway when use fastcgi_pass: 127.0.0.1:9000
  • accessing a docker container from another another container
  • Inconsistent runtime kernel parameters in DOCKER container and on host
  • How i can configure dockerized WildFly with HornetQ?
  • In Docker for Windows, permissions denied for mkdir/chown
  • For one broker all work fine:

    I start my zookeeper:

    docker run -d –name zookeeper jplock/zookeeper:3.4.6

    Then I start a kafka instance:

    docker run -d –name kafka –link zookeeper:zookeeper ches/kafka

    When I create topic and messages, all work.

    Now I create a second kafka instance:

    docker run -d –name kafka2 –link zookeeper:zookeeper –expose 9093 –env-file env ches/kafka

    I change the exposed port to 9093 and environment variables:

    PORT=9093

    EXPOSED_PORT=9093

    BROKER_ID=2

    The broker starts well and I can create a topic with replication:

    docker -D run –rm ches/kafka kafka-topics.sh –create –topic test2 –replication-factor 2 –partitions 1 –zookeeper $ZK_IP:2181

    Now when I want to send some messages:

    docker run –rm –interactive ches/kafka kafka-console-producer.sh –topic test2 –broker-list $KAFKA_IP:9092

    I get this error:

    ERROR Producer connection to 172.17.0.17:9093 unsuccessful
    (kafka.producer.SyncProducer) java.net.ConnectException: Connection
    refused

    A docker ps give me that:

    CONTAINER ID        IMAGE                    COMMAND                CREATED              STATUS              PORTS                          NAMES
    d7bba0f3d0db        ches/kafka:latest        "kafka-console-produ   About a minute ago   Up About a minute   9092/tcp, 7203/tcp             sick_shockley
    9c475a659383        ches/kafka:latest        "/start.sh"            4 minutes ago        Up 4 minutes        7203/tcp, 9092/tcp, 9093/tcp   kafka2
    63aae4c539ab        ches/kafka:latest        "/start.sh"            28 minutes ago       Up 28 minutes       7203/tcp, 9092/tcp             kafka
    ab560690e0e7        jplock/zookeeper:3.4.6   "/opt/zookeeper/bin/   28 minutes ago       Up 28 minutes       2181/tcp, 2888/tcp, 3888/tcp   zookeeper
    

    So kafka2 seems started on 9093 port.

    Why I get this error ?

    Thanks.

  • Run Docker in daemon mode
  • Can I set Neo4j on Docker to import CSV file on the first run, not after that?
  • How to get JSF working with spring-boot app in docker container?
  • jenkins exectue shell docker in slave node without sudo
  • 'docker-compose' not found error when command was executed in Jenkins job
  • REST API for Docker Hub Registry
  • One Solution collect form web for “Docker: multi kafka brokers failed”

    Check in the zookeeper docker what is the advertised host name of the kafka dockers. It’s very possible that they registered their docker hash as host name (as it’s the result of getInetAdress() within a docker container) instead of a resolvable address.

    If that’s the case, editing your standard kafka config to change advertised.host.name should solve your problem (it’s a bit annoying because you have to change it at start, but you can for example fetch it from the /etc/hosts file of the docker container at startup, it should be the first half of the first line in it).

    Docker will be the best open platform for developers and sysadmins to build, ship, and run distributed applications.