How to speed up CI build times when using docker?

I currently use docker + travis CI to test/ deploy my app. This works great locally because I have data volumes for things like node_modules etc, and docker’s layers provide caching speeding up builds.

However, when I push the code to travis it has to rebuild and install everything from scratch and it takes forever! Travis doesn’t support caching docker layers atm. Is there some other way to speed up my builds, or another similar tool that allows docker layer caching?

  • Howto detect in Docker the container kernel version
  • Do I want a container handling multiple requests?
  • Cannot delete files on docker host
  • Dockerfile not using cache in RUN composer install command
  • How to put docker container for database on a different host in production?
  • How to estimate the VM resources (CPU, RAM, HDD etc) with respect to Docker image size? My images are huge, 5-10 GB
  • Docker - Jenkins Image
  • Duplicating docker container for debugging
  • kubernetes volume hostPath
  • How to test the docker containers startup time
  • Setting up Ruby along with android in docker
  • Naming Docker Containers on start ECS
  • One Solution collect form web for “How to speed up CI build times when using docker?”

    You might want to investigate how i3wm has solved a similar problem.

    The main developer has written on the design behind his Travis CI workflow. Quoting the relevant part:

    The basic idea is to build a Docker container based on Debian testing
    and then run all build/test commands inside that container. Our
    Dockerfile installs compilers, formatters and other development tools
    first, then installs all build dependencies for i3 based on the
    debian/control file, so that we don’t need to duplicate build
    dependencies for Travis and for Debian.

    This solves the immediate issue nicely, but comes at a significant
    cost: building a Docker container adds quite a bit of wall clock time
    to a Travis run, and we want to give our contributors quick feedback.
    The solution to long build times is caching: we can simply upload the
    Docker container to the Docker Hub and make subsequent builds use the
    cached version.

    We decided to cache the container for a month, or until inputs to the
    build environment (currently the Dockerfile and debian/control)
    change. Technically, this is implemented by a little shell script
    called ha.sh (get it? hash!) which prints the SHA-256 hash of the
    input files. This hash, appended to the current month, is what we use
    as tag for the Docker container, e.g. 2016-03-3d453fe1.

    See our .travis.yml for how to plug it all together.

    Docker will be the best open platform for developers and sysadmins to build, ship, and run distributed applications.