How to set up Hadoop in Docker Swarm?

I would like to be able to start a Hadoop cluster in Docker, distributing the Hadoop nodes to the different physical nodes, using swarm.

I have found the sequenceiq image that lets me run hadoop in a docker container, but this doesn’t allow me to use multiple nodes. I have also looked at the cloudbreak project, but it seems to need an openstack installation, which seems a bit overkill, because it seems to me like swarm alone should be enough to do what we need.

  • Do ruby gi lmitations apply to docker containers
  • Docker + NGINX + SSL
  • dockerizing wordpress with mysql installed on the host machine
  • Using Gitlab CD and Docker
  • Bluemix create container group linking to another container
  • how load mysql extensions dockerfile with Docker, Compose with Pfm and Nginx and Mysql
  • Also I found this Stackoverflow question+answer which relies on weave, which needs sudo-rights, which our admin won’t give to everyone.

    Is there a solution so that starting the hadoop cluster comes down to starting a few containers via swarm?

  • Jenkins Docker container not copying pre-installed plugins to JENKINS_HOME on start
  • Docker Compose - How reference many schemas in one mysql container
  • Docker composer project can't find/ping or connect to internal hostnames
  • Docker container start error throw input/output error
  • How to set docker run arguments on marathon spec
  • Maven WAR Build for Tomcat: “Could not open ServletContext resource [/WEB-INF/applicationContext.xml]”
  • Docker will be the best open platform for developers and sysadmins to build, ship, and run distributed applications.