Docker build & deploy on Jenkins takes increasingly long amount of time

I have a small .NET Core web application running in a Docker container that is built and deployed in Jenkins. The docker repository is hosted via Quay.

Initially, it took less than a minute to build & deploy the container. But over time each build & deploy has taken an increasingly long amount of time, to where it is now up to over an hour. Here are some hopefully relevant portions from the beginning of the log:

  • Docker-Compose Up Works but Eb Local Run does not
  • Cannot read docker container logs: Permission denied
  • How can I preinstall software on travis-ci?
  • Can't run Docker container from image after commit
  • Remote debugging node app in Docker
  • Installing composer in a docker container
  • checking out out the repository from git:

    ...
    + ./build.sh
    + REPO=quay.io/my/dir
    + IMAGE=quay.io/my/dir:dddaca1fd347b1fabea342a6a65431b68f793ad4
    + docker pull quay.io/my/dir
    Using default tag: latest
    latest: Pulling from my/dir
    Digest: sha256:9a654ad205764a668c1fc531c0972d1273b0920dc8282a8c7b6498482cee7975
    Status: Image is up to date for quay.io/my/dir:latest
    + docker build -t quay.io/my/dir:dddaca1fd347b1fabea342a6a65431b68f793ad4 .
    Sending build context to Docker daemon 121.9 kB
    Sending build context to Docker daemon 121.9 kB
    ...
    

    pushing the docker image, I think:

    ...
    + '[' true = true ']'
    + docker tag -f quay.io/my/dir:dddaca1fd347b1fabea342a6a65431b68f793ad4 quay.io/my/dir:latest
    + '[' true = true ']'
    + docker tag -f quay.io/my/dir:dddaca1fd347b1fabea342a6a65431b68f793ad4 quay.io/my/dir:latest-master
    + docker tag -f quay.io/my/dir:dddaca1fd347b1fabea342a6a65431b68f793ad4 quay.io/my/dir:master-62
    + docker push quay.io/my/dir
    The push refers to a repository [quay.io/my/dir] (len: 26)
    ...
    

    Then the time is taken up mostly by reams of the following:

    ...
    7db029aae55d: Preparing
    7db029aae55d: Pushing
    7db029aae55d: Pushed
    55d4937daea6: Preparing
    55d4937daea6: Pushing
    55d4937daea6: Pushed
    7ed278cf0e04: Preparing
    7ed278cf0e04: Pushing
    7ed278cf0e04: Pushed
    master-49: digest: sha256:8ce1e...<more of the key>...90503 size: 21007
    4c16f41d1cc1: Image already exists
    8f1aae847e7a: Image already exists
    28a352fdaf13: Image already exists
    80151bfc7fc5: Image already exists
    54c7eb98ce04: Preparing
    54c7eb98ce04: Pushing
    54c7eb98ce04: Pushed
    b1131d0b29bb: Preparing
    ... more of the same
    

    I’m a newbie to Docker and would appreciate some kind help. Here is my build.sh and Dockerfile:

    build.sh

    #!/bin/bash -ex
    
    REPO=quay.io/my/dir
    IMAGE=$REPO:$GIT_COMMIT
    
    docker pull $REPO
    docker build -t $IMAGE .
    docker run --rm $IMAGE /bin/bash -c "cd /app/test/Project.Tests && dotnet test"
    
    if [ "$TAG_LATEST" = true ]; then
        docker tag -f $IMAGE $REPO:latest
    fi
    
    if [ "$TAG_LATEST_MASTER" = true ]; then
        docker tag -f $IMAGE $REPO:latest-master
    fi
    
    # $TAG defined prior to running script
    docker tag -f $IMAGE $REPO:$TAG
    
    docker push $REPO
    

    Dockerfile

    FROM microsoft/dotnet:1.1.1-sdk
    
    EXPOSE 5000
    
    COPY . /app
    WORKDIR /app
    RUN cd src/Project && dotnet restore --configfile NuGet.config --no-cache
    RUN cd test/Project.Tests && dotnet restore --no-cache
    RUN dotnet build
    
    WORKDIR /app/src/Project
    CMD ["dotnet", "run", "--server.urls", "https://0.0.0.0:5000"]
    

    Image History

    $ sudo docker history quay.io/my/dir
    IMAGE               CREATED             CREATED BY                                      SIZE                COMMENT
    0e58e5457605        4 days ago          /bin/sh -c #(nop) CMD ["dotnet" "run" "--s...   0B
    <missing>           4 days ago          /bin/sh -c #(nop) WORKDIR /app/src/Project    0B
    <missing>           4 days ago          /bin/sh -c dotnet build                         782kB
    <missing>           4 days ago          /bin/sh -c cd test/Project.Tests && dotn...   15.8MB
    <missing>           4 days ago          /bin/sh -c cd src/Project && dotnet rest...   2.4MB
    <missing>           4 days ago          /bin/sh -c #(nop) WORKDIR /app                  0B
    <missing>           4 days ago          /bin/sh -c #(nop) COPY dir:c9cbed0051facc2...   62kB
    <missing>           3 weeks ago         /bin/sh -c #(nop) EXPOSE 5000/tcp               0B
    <missing>           4 weeks ago         /bin/sh -c mkdir warmup     && cd warmup  ...   297MB
    <missing>           4 weeks ago         /bin/sh -c #(nop)  ENV NUGET_XMLDOC_MODE=skip   0B
    <missing>           4 weeks ago         /bin/sh -c curl -SL $DOTNET_SDK_DOWNLOAD_U...   290MB
    <missing>           4 weeks ago         /bin/sh -c #(nop)  ENV DOTNET_SDK_DOWNLOAD...   0B
    <missing>           4 weeks ago         /bin/sh -c #(nop)  ENV DOTNET_SDK_VERSION=...   0B
    <missing>           4 weeks ago         /bin/sh -c apt-get update     && apt-get i...   1.64MB
    <missing>           4 weeks ago         /bin/sh -c apt-get update && apt-get insta...   123MB
    <missing>           4 weeks ago         /bin/sh -c apt-get update && apt-get insta...   44.6MB
    <missing>           4 weeks ago         /bin/sh -c #(nop)  CMD ["/bin/bash"]            0B
    <missing>           4 weeks ago         /bin/sh -c #(nop) ADD file:f4e6551ac34ab44...   124MB
    

  • Understanding a docker entrypoint script
  • Service host/port undefined, Kubernetes/Google Container Engine
  • Docker build inside kubernetes pod fails with “could not find bridge docker0”
  • Docker containers clustering
  • Can't push docker image to internal Artifactory
  • How to export the data in docker container?
  • One Solution collect form web for “Docker build & deploy on Jenkins takes increasingly long amount of time”

    I solved this by changing the last line in my build.sh file:

    docker push $REPO
    

    to

    docker push $REPO:$TAG
    

    I was pushing the entire repository back to the quay server on each build, instead of only the image I just created. Each build would take longer than the last because the repository grows with each image that is added to it.

    Docker will be the best open platform for developers and sysadmins to build, ship, and run distributed applications.