“Caching” intermediate Docker build

I’m learning to use Docker and I’ve come across a minor annoyance. Whenever I make a change to the Dockerfile,I run docker build -t tag . which goes through the entire Dockerfile as it should. This takes a good 5-6 minutes due to the dependencies in my project. Sometimes a command that I run will cause an error, or there will be a mistake in the Dockerfile. While the fix may take a couple seconds, I have to rebuild the entire thing which decreases my productivity. Is there a way to “continue from where the build last failed” after editing the Dockerfile? Thanks.

  • What command can I use to find out the version of docker swarm I am using?
  • Queries about LoadBalancing in Kubernetes cluster
  • Setting docker environment variable to existing environment variable with different name
  • Permissions error when using cli in Jboss wildfly and docker
  • How to change the ownership of Docker-Tomcat log files?
  • Is it redundant in a Dockfile to run USER root since you're already root?
  • How to mount container directory to another docker container?
  • Using two docker files - dev and prod
  • Cloning GitHub private repository with Docker container non-root user?
  • Why can't I ping docker container?
  • How to create docker machine with specific version?
  • Running Nodejs Application in Amazon EC2 to serve my static content in nodejs using nginx as a reverse proxy
  • 2 Solutions collect form web for ““Caching” intermediate Docker build”

    This is called the “build cache” and it is already a feature of Docker. Docker’s builder will only use the cache up until the point where your Dockerfile has changed. There are some edge cases when using COPY or ADD directives that will cause the build cache to be invalidated (since it hashes files to determine if any have changed, and invalidates the cache if so). This means that if you are using COPY foo /foo and you have changed that file, the build cache will be invalidated. Also, if you do COPY . /opt/bar/ (meaning, you copy the entire directory to somewhere), even some small change like a Vim swap file or Dockerfile change will invalidate the cache!

    The behavior of not using the build cache at all is invoked using --no-cache in your docker build command.

    So basically, it’s there, and you’re using it, just that you’re probably changing the Dockerfile at a very early point or hitting that lesser known edge case with a COPY/ADD directive, and the builder is invalidating everything after that point. And just to answer the question before you ask it, it would be very hard or impossible to continue using the cache after a change has invalidated the cache. Meaning, if you change your first Dockerfile line and invalidate the build cache, it is basically impossible to use the build cache past that point.

    Is there a way to “continue from where the build last failed” after editing the Dockerfile?

    No (as L0j1k’s answer explains well)

    That is why the best practice is to organize your Dockerfile from the stablest commands (the one which will never have to be changed/modified) to the most specific commands (the ones you might have to change quite a bit).

    That way, your modifications will trigger only a build on the last few lines of your Dockerfile, instead of going through everything again, because you changed one of the first lines.

    Docker will be the best open platform for developers and sysadmins to build, ship, and run distributed applications.