Docker: Layering on top of shared volume

I want to use a shared volume in a container but the updates to data in the shared volume cannot go to the underlying directory on the host and these updates should be discarded when the container goes away.

For example:

  • Share docker images between hosts with NFS
  • Docker intermittently failing when building image
  • After stopping docker containers , previously running containers cannot be started
  • How to clean docker devicemapper folder properly ?
  • How does device mapper “union” docker images
  • Deleting devicemapper after changing storage driver
    • Create shared directory /var/data with one file /var/data/random

      mkdir /var/data;
      dd if=/dev/urandom bs=1M count=100 of=/var/data/random

    • Start a container using this directory as shared volume

      docker run -v /var/data:/data -t -i –name deb debian /bin/sh

    • Inside the container modify the file

      dd if=/dev/zero bs=4k count=1 of=/data/random conv=notrunc

    Here is what I would like to happen:

    1) Within the container the /var/data/random should look like 4K of zero followed by random data

    2) Outside the container /var/data/random would remain unmodified

    Mounting the container as read-only doesn’t solve the option. Copying the data and making it part of the docker image is not a solution.

    I would like to use the same layering that docker uses for the root filesystem of the container to make this work.

    Thanks!

  • How can I restart docker service using puppet…?
  • Pycharm 2016.1.4 (newest) and Docker issue
  • Can't connect to docker-machine container on digitalocean
  • Empty response while making a GET http request to the localhost server using Go
  • How do I create a PID trap for multiple commands separated by &&
  • How to check whether python package is installed or not in Docker?
  • One Solution collect form web for “Docker: Layering on top of shared volume”

    This can be archived using data containers.

    Take a look at this bash script:

    #!/bin/bash
    
    docker kill data-test-container > /dev/null 2>&1
    docker rm data-test-container  > /dev/null 2>&1
    docker rmi data-test-image  > /dev/null 2>&1
    docker build -t data-test-image - <<END_DOCKERFILE
    FROM busybox
    RUN mkdir /data && dd if=/dev/urandom bs=1K count=100 of=/data/persistent
    VOLUME /data
    END_DOCKERFILE
    
    docker run -d --name=data-test-container data-test-image true
    echo File checksum in image:
    docker run --rm --volumes-from=data-test-container busybox md5sum /data/persistent
    docker run --rm --volumes-from=data-test-container busybox dd if=/dev/urandom bs=1K count=100 of=/data/persistent
    echo File checksum after change:
    docker run --rm --volumes-from=data-test-container busybox md5sum /data/persistent
    
    docker rm data-test-container
    docker run -d --name=data-test-container data-test-image true
    
    echo File checksum after restore:
    docker run --rm --volumes-from=data-test-container busybox md5sum /data/persistent
    

    Here is the output:

    Sending build context to Docker daemon 2.048 kB
    Sending build context to Docker daemon 
    Step 0 : FROM busybox
     ---> 4986bf8c1536
    Step 1 : RUN mkdir /data && dd if=/dev/urandom bs=1K count=100 of=/data/persistent
     ---> Running in 117d1884d448
    100+0 records in
    100+0 records out
     ---> ac22e944c73c
    Removing intermediate container 117d1884d448
    Step 2 : VOLUME /data
     ---> Running in 52cf4b446255
     ---> d7f3494b1f72
    Removing intermediate container 52cf4b446255
    Successfully built d7f3494b1f72
    b7c7fe133479afed41f0af31df7a4535d19353bc976b25393a2c6fe68344ac6b
    File checksum in image:
    029ae64fe02ebfb6a3ebf0af31965826  /data/persistent
    100+0 records in
    100+0 records out
    File checksum after change:
    767260fa798a55d639de0f775e18595c  /data/persistent
    data-test-container
    19b9e3a5d3286c7a9bee101b1984ac033a2dd8f0bd7a6648f6c451b5e0cded32
    File checksum after restore:
    029ae64fe02ebfb6a3ebf0af31965826  /data/persistent
    

    If you want to get live data into an image (in stead of having to build it from scratch), snapshots should be the way to go, but I haven’t tried this and I’ve been told that there is a nasty bug in the Docker image repository server code (if you set up a local server to host the snapshots) that makes it impossible to delete snapshots. This bug may have been fixed by now, though.

    Docker will be the best open platform for developers and sysadmins to build, ship, and run distributed applications.