HaProxy for service discovery on a marathon mesos docker linked containers

Please this is not asked anywhere I have checked. Here is what I have done. I am able to deploy single instance of mesos, marathon and docker. Moving next step ahead I want to have 2 mesos slave(docker containers) linked to each other. Just using docker the same can be achieved by using the docker link feature. But while using the orchestration(mesos) and scheduler(marathon)it seems u need to use service discovery.

My setup up is simple and runnning on a single host. So I will have 2 docker containers one running a simple pub/sub and one running rabbitmq. How can I use HA PRoxy in this setup. I have seen some documents provided by mesosphere
http://mesosphere.com/docs/getting-started/service-discovery/ but it is not clear how to go about it.

  • How to copy multiple files into a docker data volume
  • Salt: 'dockerng' __virtual__ returned False
  • Can't access mapped port (can't connect from host to sshd running inside docker container)
  • Cannot login to private docker registry
  • apt-add-repository: command not found error in Dockerfile
  • Docker - traffic mirroring
  • Docker network - Why docker container responds on 127.0.0.1?
  • Cannot install RWeka package in docker image
  • Docker instead of multiple VMs
  • Export of locally available Docker image to local file system
  • mount a folder in docker machine using docker for mac
  • Dockerfile error: /bin/sh: 1: [“python”,: not found
  • 4 Solutions collect form web for “HaProxy for service discovery on a marathon mesos docker linked containers”

    The canonical approach for service discovery with Mesos + Marathon + Docker is currently what is described in the document you linked.

    I’m assuming you’re able to get the two applications running in Marathon already.

    Typically what happens is:

    1) Configure your application definition to include the ports that your application requires.

    2) You set up the provided haproxy-marathon-bridge script to run periodically using a utility like cron. This script scrapes Marathon’s API to figure out what host and port the application instances are running on and what the known “friendly” port is.

    In the example in the service discovery article, the first application has friendly ports of 80 and 443, whilst the second has a friendly port of 8081.

    The script then generates a haproxy.cfg configuration that has rules mapping localhost:friendly_port to actual_host:actual_port.

    3) Configure your applications to look for each other on localhost:friendly_port. HAProxy will route connections appropriately.

    Hope this helps your understanding!

    I created a haproxy service discovery docker container that you can run in mesos. It’s not production ready but I am using it in my development environment doing exactly what you’re trying to do. The reason I prefer this over what comes with marathon is I haven’t found a good way to do complicated haproxy configurations with haproxy-marathon-bridge. With spiderweb you can create a template for the haproxy configuration which enables you to do things such as acl routing etc. It doesn’t support health checks yet which is something that will need to be done before its production ready. You can see the project here https://github.com/SBRDevelopment/spiderweb.

    We have combined Mesos and Marathon with consul and registartor,
    so in the end you have haproxy configuration auto-generated with consul-template.

    try https://github.com/eBayClassifiedsGroup/PanteraS
    All in one container.

    With Mesos-DNS you can also do the following:

    1. Setup mesos-dns as in this guide: http://programmableinfrastructure.com/guides/service-discovery/mesos-dns-haproxy-marathon/ (you can skip HAProxy steps they are not required)
    2. When you start your docker containers make sure that they have “namespace %slave_ip_with_mesos_dns%” (replace string with IP address) in their /etc/resolv.conf files.
    3. if lets say name of an app is “peek” it should be reachable from other applications at peek.marathon.mesos
    Docker will be the best open platform for developers and sysadmins to build, ship, and run distributed applications.