Docker Swarm – strategy doesn't seem to have any effect

I have provisioned a swarm master and swarm nodes with Docker Machine (as described here). Everything is working fine; all the machines are created and running, they have all been discovered and accepts containers.

The output from ‘docker-machine ls’ is:

  • Accessing RDS from within a Docker container not getting through security group?
  • Proper method to run gulp build on jenkins for AWS ECS
  • Does docker stores all its files as “memory image”, as part of image, not disk file?
  • Can't interact with java process when attaching
  • Change “hosts” / “-h” Docker for Windows in daemon.json
  • Create container from local file
  • $ docker-machine ls
    NAME         ACTIVE      DRIVER         STATE     URL                          SWARM               DOCKER    ERRORS
    default      -           virtualbox     Stopped                                                    Unknown   
    local        -           virtualbox     Stopped                                                    Unknown   
    my-swarm     * (swarm)   digitalocean   Running   tcp://104.131.161.197:2376   my-swarm (master)   v1.11.1   
    node0        -           digitalocean   Running   tcp://104.236.29.169:2376    my-swarm            v1.11.1   
    node1        -           digitalocean   Running   tcp://104.236.216.164:2376   my-swarm            v1.11.1   
    

    The problem I’m having is with the distribution of containers. No matter which strategy I set for the swarm, it only seems to distribute containers to one of the nodes at a time. I.e. I run a bunch of containers, and the are all started on the same node, as shown below (with strategy Spread):

    $ docker ps
    5c075d7ccddc        stress              "/bin/sh -c /stress.s"   32 seconds ago       Up 31 seconds                           node0/elated_goldstine
    5bae22a15829        stress              "/bin/sh -c /stress.s"   46 seconds ago       Up 44 seconds                           node0/cocky_booth
    dc52b3dfa0e6        stress              "/bin/sh -c /stress.s"   About a minute ago   Up About a minute                       node0/goofy_kalam
    3b9e69c694da        stress              "/bin/sh -c /stress.s"   About a minute ago   Up About a minute                       node0/focused_fermat
    ef0e006ff3e0        stress              "/bin/sh -c /stress.s"   About a minute ago   Up About a minute                       node0/stoic_engelbart
    53e46b19ab33        stress              "/bin/sh -c /stress.s"   About a minute ago   Up About a minute                       node0/condescending_rosalind
    e9e126c7f4c6        stress              "/bin/sh -c /stress.s"   About a minute ago   Up About a minute                       node0/sleepy_jang
    f9c0003d509d        stress              "/bin/sh -c /stress.s"   About a minute ago   Up About a minute                       node0/amazing_bhaskara
    

    What I would expect here is for the containers to be distributed roughly evenly on the 3 nodes, especially as the script I’m running in the containers is designed to take as much CPU as possible. But instead all of them are on node0 (which I would only expect with Binpack). The Random strategy has the exact same behaviour.

    The output from ‘docker info’ with the swarm master set as active seems correct:

    $ docker info
    Containers: 15
     Running: 4
     Paused: 0
     Stopped: 11
    Images: 5
    Server Version: swarm/1.2.1
    Role: primary
    Strategy: spread
    Filters: health, port, containerslots, dependency, affinity, constraint
    Nodes: 3 
     my-swarm: 104.131.161.197:2376
      └ ID: L2HK:F6S3:WWIM:BHNI:M4XL:KLEA:4U22:J6CE:ZHZI:OGGT:76KF:MTQU
      └ Status: Healthy
      └ Containers: 2
      └ Reserved CPUs: 0 / 1
      └ Reserved Memory: 0 B / 513.4 MiB
      └ Labels: executiondriver=, kernelversion=4.2.0-27-generic, operatingsystem=Ubuntu 15.10, provider=digitalocean, storagedriver=aufs
      └ Error: (none)
      └ UpdatedAt: 2016-05-09T10:25:24Z
      └ ServerVersion: 1.11.1
     node0: 104.236.29.169:2376
      └ ID: I3TQ:5BMS:TM2P:GLL4:64OH:BDMY:SWBU:3QG4:TOZ2:LEDW:A6SQ:X34H
      └ Status: Healthy
      └ Containers: 12
      └ Reserved CPUs: 0 / 1
      └ Reserved Memory: 0 B / 513.4 MiB
      └ Labels: executiondriver=, kernelversion=4.2.0-27-generic, operatingsystem=Ubuntu 15.10, provider=digitalocean, storagedriver=aufs
      └ Error: (none)
      └ UpdatedAt: 2016-05-09T10:25:02Z
      └ ServerVersion: 1.11.1
     node1: 104.236.216.164:2376
      └ ID: OTQH:UBSV:2HKE:ZVHL:2K7Z:BYGC:ZX25:Y6BQ:BN5J:UWEB:65KE:DABM
      └ Status: Healthy
      └ Containers: 1
      └ Reserved CPUs: 0 / 1
      └ Reserved Memory: 0 B / 513.4 MiB
      └ Labels: executiondriver=, kernelversion=4.2.0-27-generic, operatingsystem=Ubuntu 15.10, provider=digitalocean, storagedriver=aufs
      └ Error: (none)
      └ UpdatedAt: 2016-05-09T10:25:10Z
      └ ServerVersion: 1.11.1
    Plugins: 
     Volume: 
     Network: 
    Kernel Version: 4.2.0-27-generic
    Operating System: linux
    Architecture: amd64
    CPUs: 3
    Total Memory: 1.504 GiB
    Name: my-swarm
    Docker Root Dir: 
    Debug mode (client): false
    Debug mode (server): false
    WARNING: No kernel memory limit support
    

    Is there some piece I have missed that is necessary for this type of automatic distribution to work the way I’m expecting?

  • Issue when joining serf nodes located in different Docker containers
  • Connection refused : Nginx HTTPS reverse proxy in docker container
  • Is it possible to devep web applications (PHP/Ruby) using Docker without PHP/Ruby on host?
  • How to find the CMD command of a docker image?
  • Scripts with dialog from Docker container
  • Meteor, docker and SSL on localhost
  • One Solution collect form web for “Docker Swarm – strategy doesn't seem to have any effect”

    Issue

    It is most likely that you Docker Agent does not have the image you try to run on them.

    Note that swarm build and swarm pull will not build, or pull, on each Docker agent.

    Solution A

    Run the pull manually on each Docker Agent

    Solution B (not tested)

    Run a local Docker registry wherein the Docker agent will pull the images from.


    Have fun with Swarm! Also, if I may, I suggest you to read this answer that details a whole step-by-step tutorial on Swarm.

    Docker will be the best open platform for developers and sysadmins to build, ship, and run distributed applications.