Docker how to run pip requirements.txt only if there was a change?

In a Dockerfile I have a layer which installs requirements.txt:

FROM python:2.7
RUN pip install -r requirements.txt

When I build the docker image it runs the whole process regardless of any changes made to this file.

  • See full command of running/stopped container in Docker
  • Variable cgroups parametres in Docker
  • Docker /etc/apache2/sites-enabled/000-default.conf not updating
  • Can I mix linux-based and windows-based containers in same docker engine host?
  • how to securely connect to a mysql server running on a virtual machine
  • Write to /etc/hosts in docker container when running docker container
  • How do I make sure Docker only runs pip install -r requirements.txt if there has been a change to the file?

    Removing intermediate container f98c845d0f05
    Step 3 : RUN pip install -r requirements.txt
     ---> Running in 8ceb63abaef6
    Collecting https://github.com/tomchristie/django-rest-framework/archive/master.zip (from -r requirements.txt (line 30))
      Downloading https://github.com/tomchristie/django-rest-framework/archive/master.zip
    Collecting Django==1.8.7 (from -r requirements.txt (line 1))
    

  • Connect tomcat container to mysql container
  • How to test HTML pages served by docker with Windows vitrualbox guest?
  • Troubleshooting API timeout from Django+Celery in Docker Container
  • Shell variables in dockerfile missing in docker image labels
  • Error building syntaxnet using Dockerfile
  • How best to use Docker in continuous delivery?
  • 2 Solutions collect form web for “Docker how to run pip requirements.txt only if there was a change?”

    I’m assuming that at some point in your build process, you’re copying your entire application into the Docker image with COPY or ADD:

    COPY . /opt/app
    WORKDIR /opt/app
    RUN pip install -r requirements.txt
    

    The problem is that you’re invalidating the Docker build cache every time you’re copying the entire application into the image. This will also invalidate the cache for all subsequent build steps.

    To prevent this, I’d suggest copying only the requirements.txt file in a separate build step before adding the entire application into the image:

    COPY requirements.txt /opt/app/requirements.txt
    WORKDIR /opt/app
    RUN pip install -r requirements.txt
    COPY . /opt/app
    # continue as before...
    

    As the requirements file itself probably changes only rarely, you’ll be able to use the cached layers up until the point that you add your application code into the image.

    This is directly mentioned in Docker’s own “Best practices for writing Dockerfiles”:

    If you have multiple Dockerfile steps that use different files from
    your context, COPY them individually, rather than all at once. This
    will ensure that each step’s build cache is only invalidated (forcing
    the step to be re-run) if the specifically required files change.

    For example:

    COPY requirements.txt /tmp/
    RUN pip install --requirement /tmp/requirements.txt
    COPY . /tmp/
    

    Results in fewer cache invalidations for the RUN step, than if you put
    the COPY . /tmp/ before it.

    Docker will be the best open platform for developers and sysadmins to build, ship, and run distributed applications.