Docker build failing when using gcsfuse to mount google storage

I have been trying to mount SQL and a storage bucket to my docker WordPress container. It appears to succeeding in mounting SQL, but failing mounting the bucket. The instance is based of of this post.

I have attached the Docker file and error below, as well as my build command.

  • AWS ECS container exiting without specific reason
  • docker - Error response from daemon: rpc error: code = 2 desc = name conflicts with an existing object
  • Background/foreground with docker-compose
  • Upgrading to docker 1.9 on Centos 7.1
  • In dockerized python3 ftp command RETR throws a ConnectionResetError
  • can't authenticate user in postgres docker container
  • Build command:

    docker build -t ic/spm .

    Dockerfile:

    FROM wordpress
    MAINTAINER Gareth Williams <gareth@itinerateconsulting.com>
    
    # Move login creds locally
    ADD ./creds.json /creds.json
    
    # install sudo, wget and gcsfuse
    ENV GCSFUSE_REPO=gcsfuse-jessie
    RUN   apt-get update && \
          apt-get -y install sudo && \
          apt-get install -y curl ca-certificates && \
          echo "deb http://packages.cloud.google.com/apt $GCSFUSE_REPO main" > /etc/apt/sources.list.d/gcsfuse.list && \
          curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - && \
          apt-get update && \
          apt-get install -y gcsfuse wget && \
          apt-get remove -y curl --purge && \
          apt-get autoremove -y && \
          rm -rf /var/lib/apt/lists/*
    
    # Config fuse
    RUN chmod a+r /etc/fuse.conf
    RUN perl -i -pe 's/#user_allow_other/user_allow_other/g' /etc/fuse.conf
    
    # Setup sql proxy
    RUN sudo mkdir /cloudsql
    RUN sudo chmod 777 /cloudsql
    ADD https://dl.google.com/cloudsql/cloud_sql_proxy.linux.amd64 cloud_sql_proxy.linux.amd64
    RUN mv cloud_sql_proxy.linux.amd64 cloud_sql_proxy && chmod +x ./cloud_sql_proxy
    RUN ./cloud_sql_proxy -dir=/cloudsql -fuse -credential_file=/creds.json &
    # mysql -u icroot -S /cloudsql/[INSTANCE_CONNECTION_NAME]
    
    # Perform Cloud Storage FUSE mounting for uploads folder
    RUN mkdir /mnt/uploads
    RUN chmod a+w /mnt/uploads
    #RUN chown www-data:www-data -R /mnt && groupadd fuse && gpasswd -a www-data fuse && chmod g+rw /dev/fuse
    USER www-data
    RUN gcsfuse --key-file /creds.json \
      --debug_gcs --debug_http --debug_fuse --debug_invariants \
      --dir-mode "777" -o allow_other spm-bucket /mnt/uploads
    

    Error:

    Step 17 : RUN gcsfuse --key-file /creds.json   --foreground --debug_gcs --debug_http --debug_fuse --debug_invariants   --dir-mode "777" -o allow_other spm-bucket /mnt/uploads
     ---> Running in 7e3f31221bee
    Using mount point: /mnt/uploads
    Opening GCS connection...
    Opening bucket...
    gcs: Req              0x0: <- ListObjects()
    http: ========== REQUEST:
    GET http://www.googleapis.com/storage/v1/b/spm-bucket/o?maxResults=1&projection=full HTTP/1.1
    Host: www.googleapis.com
    User-Agent: gcsfuse/0.0
    Authorization: Bearer ya29.ElrQAw8oxClKt8YGvtmxhc7z2Y2LufvL0fBueq1UESjYYjRrdxukNTQqO1qfM8e8h-rqfbOWNSjVK2rCRXVrEDla-CiUVhHwT6X71Y1Djb0jDJg7z3KblgNQPrc
    Accept-Encoding: gzip
    
    http: ========== RESPONSE:
    HTTP/2.0 200 OK
    Content-Length: 31
    Alt-Svc: quic=":443"; ma=2592000; v="35,34"
    Cache-Control: private, max-age=0, must-revalidate, no-transform
    Content-Type: application/json; charset=UTF-8
    Date: Wed, 11 Jan 2017 09:19:05 GMT
    Expires: Wed, 11 Jan 2017 09:19:05 GMT
    Server: UploadServer
    Vary: Origin
    Vary: X-Origin
    X-Guploader-Uploadid: AEnB2UpTqXhtHW906FFDTRsz4FjHjFu_E84wYhvt0zhaVFuMpqSY1fsd1XcrEcpsYBBwX1mqf0ZXRVWJH05ThtDQIfFKHd4PFw
    
    {
     "kind": "storage#objects"
    }
    http: ====================
    gcs: Req              0x0: -> ListObjects() (1.793169206s): OK
    Mounting file system...
    mountWithArgs: mountWithConn: Mount: mount: running fusermount: exit status 1
    
    stderr:
    fusermount: failed to open /dev/fuse: Operation not permitted
    

  • Laravel POST TokenMismatchException for files bigger than 8kb/16KB (and 2MB) when jwilder/nginx-proxy
  • Gunicorn does not find main module with docker
  • Can not install MariaDB through Docker and Vagrant
  • PHP api requests outgoing IP on docker swarm & HAproxy
  • What is the correct way to have Tensorflow available in a docker container or docker image?
  • How to display the docker current configuration?
  • 2 Solutions collect form web for “Docker build failing when using gcsfuse to mount google storage”

    If you’re running your container on GKE, and you want to use gcsfuse, permissions should automatically be inherited in your account locally. Also…there is a caveat that you need to make sure that the cluster your running needs to have storage access. So make sure your cluster has the storage permissions set to full access. That way gcsfuse can mount your buckets on GCS within the container without having to worry about passing credential files and all that stuff…making the implementation pretty straight forward.

    In your docker file…make sure you’re doing your apt commands to get and install the gcsfuse application.

    I personally made a shell script that I call once the instance is up, that mounts my directories that I needed.

    Something like this…

    Docker Entry

    ENTRYPOINT ["/opt/entry.sh"]

    entry.sh script example

    gcsfuse [gcs bucket name] [local folder to mount as]

    When generating your GKE cluster, make sure to add the storage scope

    gcloud container clusters create [your cluster name] --scopes storage-full

    Hope this helps you.

    your www-data have permission problem in the dockerfile:

    #RUN chown www-data:www-data -R /mnt && groupadd fuse && gpasswd -a www-data fuse && chmod g+rw /dev/fuse
    

    uncomment this line

    Docker will be the best open platform for developers and sysadmins to build, ship, and run distributed applications.