Kafka not interacting with kaka-python code in docker

So I started a kafka and zookeeper instance on the host. Now I want to interact with it via two dockers – producer and consumer

The code in the PRODUCER docker is:

  • GitLab CI ERROR: Preparation failed: Post https…docker%3Adind: net/http: timeout awaiting response headers
  • Docker container not starting a second time (iptables)
  • “Error reading data” while executing a RabbitMQ consumer in Symfony
  • Containers network error when use docker with live-restore
  • Docker container stops running only when volume is attached
  • Docker Service updatedAt timestamp
  • from kafka import KafkaProducer
    import time
    producer = KafkaProducer(bootstrap_servers=['localhost:9092'])
    
    i = 0
    while 1:
      # "kafkaesque" is the name of our topic
      producer.send("stupid", str(i))
      i += 1
      time.sleep(1)
    

    The code for CONSUMER docker is:

    from kafka import KafkaConsumer
    
    consumer = KafkaConsumer(bootstrap_servers=['localhost:9092'])
    consumer.subscribe(['stupid'])
    
    for message in consumer:
        print (message.value)
    

    The Dockerfile for CONSUMER is:

    FROM debian:jessie
    RUN apt-get update && apt-get -y upgrade
    RUN apt-get install -y python-setuptools
    RUN apt-get install -y python-pip
    RUN pip install kafka-python
    ADD . /Consumer
    WORKDIR /Consumer
    CMD ["python", "consumer.py"]
    

    The Dockerfile for PRODUCER is:

    FROM debian:jessie
    RUN apt-get update && apt-get -y upgrade
    RUN apt-get install -y python-setuptools
    RUN apt-get install -y python-pip
    RUN pip install kafka-python
    ADD . /Producer
    WORKDIR /Producer
    CMD ["python","counter.py"]
    

    Now i built both of them, and when I run it, like this:

    docker run consumer 
    docker run producer
    

    Then for each of then I get this error:

    Traceback (most recent call last):
      File "consumer.py", line 3, in <module>
        consumer = KafkaConsumer(bootstrap_servers=['localhost:9092'])
      File "/usr/local/lib/python2.7/dist-packages/kafka/consumer/group.py", line 284, in __init__
        self._client = KafkaClient(metrics=self._metrics, **self.config)
      File "/usr/local/lib/python2.7/dist-packages/kafka/client_async.py", line 202, in __init__
        self.config['api_version'] = self.check_version(timeout=check_timeout)
      File "/usr/local/lib/python2.7/dist-packages/kafka/client_async.py", line 791, in check_version
        raise Errors.NoBrokersAvailable()
    kafka.errors.NoBrokersAvailable: NoBrokersAvailable
    

    **

    But when I run each of them on the host machine, they work perfectly.

    **

    Can someone point out, how to solve this. I have some idea, as in the network port of the docker is not able to interact with the network port of the host, but i have tried EXPOSE and -p each of which is not helping me.

  • DNS not working on Docker image running in Google Compute Engine node
  • Dockerrun.aws.json file for private dockerhub image
  • Split Django project
  • How to change the resources allocated to a container at run time?
  • Exposing dynamically opened ports inside docker container
  • Execute mysql command from the host to container running mysql server
  • 2 Solutions collect form web for “Kafka not interacting with kaka-python code in docker”

    check KAFKA_ADVERTISED_HOST_NAME plese, run docker with:

    docker run - KAFKA_ADVERTISED_HOST_NAME=
    

    Try:

    (bootstrap_servers=[], api_version=(0, 9)))
    
    Docker will be the best open platform for developers and sysadmins to build, ship, and run distributed applications.