Docker how to run pip requirements.txt only if there was a change?

In a Dockerfile I have a layer which installs requirements.txt:

FROM python:2.7
RUN pip install -r requirements.txt

When I build the docker image it runs the whole process regardless of any changes made to this file.

  • Removing docker image errors “No such id” with a different image ID
  • How do I list all containers in a user-defined docker network?
  • Connection refused when running mongo DB command in docker
  • running nodejs app in docker with redirecting standard output to a file
  • Do pushes to docker hub always increment the “pull” count?
  • There is a circular dependency in the target dependency graph involving target “DockerBuildServiceReferences”
  • How do I make sure Docker only runs pip install -r requirements.txt if there has been a change to the file?

    Removing intermediate container f98c845d0f05
    Step 3 : RUN pip install -r requirements.txt
     ---> Running in 8ceb63abaef6
    Collecting https://github.com/tomchristie/django-rest-framework/archive/master.zip (from -r requirements.txt (line 30))
      Downloading https://github.com/tomchristie/django-rest-framework/archive/master.zip
    Collecting Django==1.8.7 (from -r requirements.txt (line 1))
    

  • Can you explain roles of swarm machine?
  • Docker container, how to use host proxy
  • Why can't I pipe a file to “tar” from “curl” with a Docker image for Debian?
  • Finding the layers and layer sizes for each Docker image
  • Is it possible to execute CMD at the middle of docker file?
  • Docker machine timeout - how to fix without destroying the machine?
  • 2 Solutions collect form web for “Docker how to run pip requirements.txt only if there was a change?”

    I’m assuming that at some point in your build process, you’re copying your entire application into the Docker image with COPY or ADD:

    COPY . /opt/app
    WORKDIR /opt/app
    RUN pip install -r requirements.txt
    

    The problem is that you’re invalidating the Docker build cache every time you’re copying the entire application into the image. This will also invalidate the cache for all subsequent build steps.

    To prevent this, I’d suggest copying only the requirements.txt file in a separate build step before adding the entire application into the image:

    COPY requirements.txt /opt/app/requirements.txt
    WORKDIR /opt/app
    RUN pip install -r requirements.txt
    COPY . /opt/app
    # continue as before...
    

    As the requirements file itself probably changes only rarely, you’ll be able to use the cached layers up until the point that you add your application code into the image.

    This is directly mentioned in Docker’s own “Best practices for writing Dockerfiles”:

    If you have multiple Dockerfile steps that use different files from
    your context, COPY them individually, rather than all at once. This
    will ensure that each step’s build cache is only invalidated (forcing
    the step to be re-run) if the specifically required files change.

    For example:

    COPY requirements.txt /tmp/
    RUN pip install --requirement /tmp/requirements.txt
    COPY . /tmp/
    

    Results in fewer cache invalidations for the RUN step, than if you put
    the COPY . /tmp/ before it.

    Docker will be the best open platform for developers and sysadmins to build, ship, and run distributed applications.