How to access my local system Postgres from Docker

When I use Docker on my own machine, I want to access my system’s Postgres. This is prior to deploying my images to AWS, where I want to use RDS.

I have followed the steps on Docker’s Quickstart: Docker Compose and Rails page with the exception that, because I was working with an existing application, I didn’t create the initial minimal Gemfile that just loads Rails.

  • How do I attach multiple containers directly to a physical interface?
  • Query to dataframe very slow on Zeppelin
  • How can I periodically backup a Postgres database in a Docker container, when volume mounts aren't working?
  • bin mounted to alpine container fails to run with /gogs not found
  • How to workaround “the input device is not a TTY” when using grunt-shell to invoke a script that calls docker run?
  • Dockerizing an existing Project
  • Outside of the Docker container, Postgres is installed and working correctly:

    $ which postgres
    /Applications/Postgres.app/Contents/Versions/latest/bin/postgres
    $ which pg_restore
    /Applications/Postgres.app/Contents/Versions/latest/bin/pg_restore
    $ which pg_dump
    /Applications/Postgres.app/Contents/Versions/latest/bin/pg_dump
    

    The app has been working fine without Docker for years. To test the app outside of the Docker container once more, I have comment out one line of Docker’s recommended docker-compose.yml file as follows:

    development: &default
      adapter: postgresql
      encoding: unicode
      database: postgres
      pool: 5
      username: postgres
      password:
      # host: db
    
    test:
      <<: *default
      database: myapp_test
    

    When I do that, I can run rake db:test:clone (for example) outside of the container no worries:

    But when I uncomment that line and connect to a TTY in the container, I’m in all sorts of pain:

    /myapp# rake db:test:clone                                          
    pg_dump -s -x -O -f /myapp/db/structure.sql postgres
    Please check the output above for any errors and make sure that `pg_dump` is installed in your PATH and has proper permissions.
    /myapp# which postgres
    /myapp# which pg_restore
    /myapp# which pg_dump
    /myapp# 
    

    What can I do?

    The Dockerfile, based on Docker’s instructions with a few extra binaries required by my existing app, is:

    FROM ruby:2.3.1-slim
    
    RUN echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selections
    RUN apt-get update
    RUN apt-get install -qq -y --no-install-recommends --fix-missing apt-utils
    RUN apt-get install -qq -y --no-install-recommends --fix-missing build-essential
    RUN apt-get install -qq -y --no-install-recommends --fix-missing git
    RUN apt-get install -qq -y --no-install-recommends --fix-missing xvfb
    RUN apt-get install -qq -y --no-install-recommends --fix-missing qt5-default
    RUN apt-get install -qq -y --no-install-recommends --fix-missing libqt5webkit5-dev
    RUN apt-get install -qq -y --no-install-recommends --fix-missing gstreamer1.0-plugins-base
    RUN apt-get install -qq -y --no-install-recommends --fix-missing gstreamer1.0-tools
    RUN apt-get install -qq -y --no-install-recommends --fix-missing gstreamer1.0-x
    RUN apt-get install -qq -y --no-install-recommends --fix-missing nodejs
    RUN apt-get install -qq -y --no-install-recommends --fix-missing libpq-dev
    
    RUN mkdir /myapp
    WORKDIR /myapp
    
    ADD Gemfile /myapp/Gemfile
    ADD Gemfile.lock /myapp/Gemfile.lock
    RUN bundle install
    
    ADD . /myapp
    

    Related: I can see redis-cli outside the container but not inside it. This will be an issue too once I solve the Postgres situation.

    Update

    When I go in like this…

    docker-compose run web rake db:create
    docker-compose run web rake db:test:clone
    

    …it seems I can access Postgres, but not pg_dump and other utilities required for rake db:test:clone, rake db:test:load or similar.

    Update II

    I added the following to the Dockerfile:

    RUN apt-get install -qq -y --no-install-recommends --fix-missing postgresql-client
    

    Now, instead of giving me a “can’t be found” error, it says:

    pg_dump: server version: 9.5.4; pg_dump version: 9.4.9
    pg_dump: aborting because of server version mismatch
    

    This is an issue you can find discussed elsewhere, but not in the context of Docker, which is crucial here.

    Would it help if I downgraded my Mac’s Postgres to the version fetched by apt-get? Upgrading the version installed by apt-get appears not to be an option:

    Step 15 : RUN apt-get install -qq -y --no-install-recommends --fix-missing postgresql-client-9.5
     ---> Running in 614b88701710
    E: Unable to locate package postgresql-client-9.5
    E: Couldn't find any package by regex 'postgresql-client-9.5'
    

  • What docker image use for ionic CD / e2e testing? Ubuntu or just node?
  • Can docker-swarm be configured to auto scale containers according to load
  • Docker Container fails to connect to remote SQL
  • Docker: tail log file without hanging
  • Dynamically start new Docker containers when needed?
  • Compile Tensorflow from source with Docker to get CPU speed up
  • One Solution collect form web for “How to access my local system Postgres from Docker”

    This isn’t “yer doin it wrong”, this is tricky. I ran through the docker intro you linked and I couldn’t get it to work.

    Here’s what I did for our app. I built our app off of a base image named after the app. The base image is named after the app. So using your example it’d be called myapp_base. I don’t see your docker-compose.yml file here so I’ll make some assumptions.


    web:
    image: your_company/myapp_base
    links:
    - db
    volumes:
    - .:/myapp
    # ports:
    # - "3002:3002"
    # env_file:
    # - .myapp.env

    Port and env_file are not needed but they might come in handy later. Myapp_base is built with a weird workflow that pulls in deps but basically throws in postgres-client if the app is declared to need postgres. It’s in chef but it’s basically myapp runlist needs postgresql. We’re actually moving away from this and trying to put the setup stuff in scripts and the Dockerfile itself.

    So … basically just add the apt-get install postgres-client to your Dockerfile, tag it as myapp_base and use that in your docker-compose.yml file. Then run the docker-compose run web rake db stuff and psql will be found.

    Docker will be the best open platform for developers and sysadmins to build, ship, and run distributed applications.