OpenStack with Docker fails to spawn instances

I am trying to integrate Docker into OpenStack (Icehouse 2014.1.3) but OpenStack fails to start the instances.

On compute node, in /var/log/nova/compute.log:

  • Visual Studio Team Service fails Task Docker Build
  • Docker container interact mode auto closing issue in docker-machine
  • Docker is changing files ownership and permission, why?
  • Not able to use .env file with docker-compose
  • Docker: creating multiple containers from only one image
  • Docker undo rm container
  • 2014-12-02 08:34:23.167 22606 ERROR nova.compute.manager [req-ce2cd284-0ab7-48a8-ba17-d18ecb59e0a4 2f33bf612258455b86d28f0841d9b9fb 1bfa9d3fcdc8464ba3bca4f636813ea6] [instance: 67c886e1-71f7-4d94-a099-b4d485a12125] Instance failed to spawn
    2014-12-02 08:34:23.167 22606 TRACE nova.compute.manager [instance: 67c886e1-71f7-4d94-a099-b4d485a12125] Traceback (most recent call last):
    2014-12-02 08:34:23.167 22606 TRACE nova.compute.manager [instance: 67c886e1-71f7-4d94-a099-b4d485a12125]   File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 1737, in _spawn
    2014-12-02 08:34:23.167 22606 TRACE nova.compute.manager [instance: 67c886e1-71f7-4d94-a099-b4d485a12125]     block_device_info)
    2014-12-02 08:34:23.167 22606 TRACE nova.compute.manager [instance: 67c886e1-71f7-4d94-a099-b4d485a12125]   File "/usr/lib/python2.6/site-packages/novadocker/virt/docker/driver.py", line 289, in spawn
    2014-12-02 08:34:23.167 22606 TRACE nova.compute.manager [instance: 67c886e1-71f7-4d94-a099-b4d485a12125]     image = self._pull_missing_image(context, image_meta, instance)
    2014-12-02 08:34:23.167 22606 TRACE nova.compute.manager [instance: 67c886e1-71f7-4d94-a099-b4d485a12125]   File "/usr/lib/python2.6/site-packages/novadocker/virt/docker/driver.py", line 245, in _pull_missing_image
    2014-12-02 08:34:23.167 22606 TRACE nova.compute.manager [instance: 67c886e1-71f7-4d94-a099-b4d485a12125]     fileutils.ensure_tree(snapshot_directory)
    2014-12-02 08:34:23.167 22606 TRACE nova.compute.manager [instance: 67c886e1-71f7-4d94-a099-b4d485a12125]   File "/usr/lib/python2.6/site-packages/nova/openstack/common/fileutils.py", line 37, in ensure_tree
    2014-12-02 08:34:23.167 22606 TRACE nova.compute.manager [instance: 67c886e1-71f7-4d94-a099-b4d485a12125]     os.makedirs(path)
    2014-12-02 08:34:23.167 22606 TRACE nova.compute.manager [instance: 67c886e1-71f7-4d94-a099-b4d485a12125]   File "/usr/lib64/python2.6/os.py", line 150, in makedirs
    2014-12-02 08:34:23.167 22606 TRACE nova.compute.manager [instance: 67c886e1-71f7-4d94-a099-b4d485a12125]     makedirs(head, mode)
    2014-12-02 08:34:23.167 22606 TRACE nova.compute.manager [instance: 67c886e1-71f7-4d94-a099-b4d485a12125]   File "/usr/lib64/python2.6/os.py", line 157, in makedirs
    2014-12-02 08:34:23.167 22606 TRACE nova.compute.manager [instance: 67c886e1-71f7-4d94-a099-b4d485a12125]     mkdir(name, mode)
    2014-12-02 08:34:23.167 22606 TRACE nova.compute.manager [instance: 67c886e1-71f7-4d94-a099-b4d485a12125] OSError: [Errno 13] Permission denied: '/usr/lib/python2.6/site-packages/instances'
    2014-12-02 08:34:23.167 22606 TRACE nova.compute.manager [instance: 67c886e1-71f7-4d94-a099-b4d485a12125] 
    2014-12-02 08:34:26.128 22606 ERROR nova.compute.manager [req-ce2cd284-0ab7-48a8-ba17-d18ecb59e0a4 2f33bf612258455b86d28f0841d9b9fb 1bfa9d3fcdc8464ba3bca4f636813ea6] [instance: 67c886e1-71f7-4d94-a099-b4d485a12125] Error: [Errno 13] Permission denied: '/usr/lib/python2.6/site-packages/instances'
    2014-12-02 08:34:26.128 22606 TRACE nova.compute.manager [instance: 67c886e1-71f7-4d94-a099-b4d485a12125] Traceback (most recent call last):
    2014-12-02 08:34:26.128 22606 TRACE nova.compute.manager [instance: 67c886e1-71f7-4d94-a099-b4d485a12125]   File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 1328, in _build_instance
    2014-12-02 08:34:26.128 22606 TRACE nova.compute.manager [instance: 67c886e1-71f7-4d94-a099-b4d485a12125]     set_access_ip=set_access_ip)
    2014-12-02 08:34:26.128 22606 TRACE nova.compute.manager [instance: 67c886e1-71f7-4d94-a099-b4d485a12125]   File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 393, in decorated_function
    2014-12-02 08:34:26.128 22606 TRACE nova.compute.manager [instance: 67c886e1-71f7-4d94-a099-b4d485a12125]     return function(self, context, *args, **kwargs)
    2014-12-02 08:34:26.128 22606 TRACE nova.compute.manager [instance: 67c886e1-71f7-4d94-a099-b4d485a12125]   File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 1740, in _spawn
    2014-12-02 08:34:26.128 22606 TRACE nova.compute.manager [instance: 67c886e1-71f7-4d94-a099-b4d485a12125]     LOG.exception(_('Instance failed to spawn'), instance=instance)
    2014-12-02 08:34:26.128 22606 TRACE nova.compute.manager [instance: 67c886e1-71f7-4d94-a099-b4d485a12125]   File "/usr/lib/python2.6/site-packages/nova/openstack/common/excutils.py", line 68, in __exit__
    2014-12-02 08:34:26.128 22606 TRACE nova.compute.manager [instance: 67c886e1-71f7-4d94-a099-b4d485a12125]     six.reraise(self.type_, self.value, self.tb)
    2014-12-02 08:34:26.128 22606 TRACE nova.compute.manager [instance: 67c886e1-71f7-4d94-a099-b4d485a12125]   File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 1737, in _spawn
    2014-12-02 08:34:26.128 22606 TRACE nova.compute.manager [instance: 67c886e1-71f7-4d94-a099-b4d485a12125]     block_device_info)
    2014-12-02 08:34:26.128 22606 TRACE nova.compute.manager [instance: 67c886e1-71f7-4d94-a099-b4d485a12125]   File "/usr/lib/python2.6/site-packages/novadocker/virt/docker/driver.py", line 289, in spawn
    2014-12-02 08:34:26.128 22606 TRACE nova.compute.manager [instance: 67c886e1-71f7-4d94-a099-b4d485a12125]     image = self._pull_missing_image(context, image_meta, instance)
    2014-12-02 08:34:26.128 22606 TRACE nova.compute.manager [instance: 67c886e1-71f7-4d94-a099-b4d485a12125]   File "/usr/lib/python2.6/site-packages/novadocker/virt/docker/driver.py", line 245, in _pull_missing_image
    2014-12-02 08:34:26.128 22606 TRACE nova.compute.manager [instance: 67c886e1-71f7-4d94-a099-b4d485a12125]     fileutils.ensure_tree(snapshot_directory)
    2014-12-02 08:34:26.128 22606 TRACE nova.compute.manager [instance: 67c886e1-71f7-4d94-a099-b4d485a12125]   File "/usr/lib/python2.6/site-packages/nova/openstack/common/fileutils.py", line 37, in ensure_tree
    2014-12-02 08:34:26.128 22606 TRACE nova.compute.manager [instance: 67c886e1-71f7-4d94-a099-b4d485a12125]     os.makedirs(path)
    2014-12-02 08:34:26.128 22606 TRACE nova.compute.manager [instance: 67c886e1-71f7-4d94-a099-b4d485a12125]   File "/usr/lib64/python2.6/os.py", line 150, in makedirs
    2014-12-02 08:34:26.128 22606 TRACE nova.compute.manager [instance: 67c886e1-71f7-4d94-a099-b4d485a12125]     makedirs(head, mode)
    2014-12-02 08:34:26.128 22606 TRACE nova.compute.manager [instance: 67c886e1-71f7-4d94-a099-b4d485a12125]   File "/usr/lib64/python2.6/os.py", line 157, in makedirs
    2014-12-02 08:34:26.128 22606 TRACE nova.compute.manager [instance: 67c886e1-71f7-4d94-a099-b4d485a12125]     mkdir(name, mode)
    2014-12-02 08:34:26.128 22606 TRACE nova.compute.manager [instance: 67c886e1-71f7-4d94-a099-b4d485a12125] OSError: [Errno 13] Permission denied: '/usr/lib/python2.6/site-packages/instances'
    2014-12-02 08:34:26.128 22606 TRACE nova.compute.manager [instance: 67c886e1-71f7-4d94-a099-b4d485a12125] 
    

    Docker was installed following https://wiki.openstack.org/wiki/Docker#Installing_Docker_for_OpenStack with a modification

    git checkout -b pre-i18n 9045ca43b645e72751099491bf5f4f9e4bddbb91
    

  • Docker Container Networking one container on one network
  • akka-http application running inside docker
  • Monitoring node processes within docker containers
  • AWS Container Service - what is the logging best practice?
  • Application served by uWSGI with Supervisord from Docker
  • Container NAMES when deploying with Docker
  • One Solution collect form web for “OpenStack with Docker fails to spawn instances”

    Turns out I needed a docker image I wanted to start container from on a compute node, not on a node with Glance service.
    Got everything to work now.

    Docker will be the best open platform for developers and sysadmins to build, ship, and run distributed applications.