How to run HDFS cluster without DNS

I’m building a local HDFS dev environment (actually hadoop + mesos + zk + kafka) to ease development of Spark jobs and facilitate local integrated testing.
All other components are working fine but I’m having issues with HDFS. When the Data Node tries to connect to the name node, I get a DisallowedDataNodeException:

org.apache.hadoop.hdfs.server.protocol.DisallowedDatanodeException: Datanode denied communication with namenode

Most questions related to the same issue boil down to name resolution of the data node at the name node either static through the etc/hosts files or using dns. Static resolution is not an option with docker, as I don’t know the data nodes when the name node container is created. I would like to avoid creating and maintaining an additional DNS service. Ideally, I would like to wire everything using the --link feature of docker.

  • How to auto start docker containers in Docker for mac?
  • Docker permission issue after copying files
  • Is docker-machine required on linux?
  • Docker 1.4.1 fails to start on Fedora 21: FATA[0000] EOF
  • Could not save file in the docker image after “docker commit”
  • Docker link container
  • Is there a way to configure HDFS in such a way that it only uses IP addresses to work?

    I found this property and set to false, but it didn’t do the trick:

    dfs.namenode.datanode.registration.ip-hostname-check (default: true)

    Is there a way to have a multi-node local HDFS cluster working only using IP addresses and without using DNS?

  • SSH tunneling to remote server with docker
  • Docker swarm node cannot see service
  • How to setup development environment easily?
  • Symfony app/console command is not running with cronjob inside Docker container
  • Docker container with entrypoint variable expansion and CMD parameters
  • Window version of Docker not able to mount local directory to ubuntu image on container?
  • One Solution collect form web for “How to run HDFS cluster without DNS”

    I would look at reconfiguring your Docker image to use a different hosts file [1]. In particular:

    1. In the Dockerfile(s), do the switch-a-roo [1]
    2. Bring up the master node
    3. Bring up the data nodes, linked
    4. Before starting the datanode, copy over the /etc/hosts to the new location, /tmp/hosts
    5. Append the master node’s name and master node ip to the new hosts file

    Hope this works for you!

    [1] https://github.com/dotcloud/docker/issues/2267#issuecomment-40364340

    Docker will be the best open platform for developers and sysadmins to build, ship, and run distributed applications.