How to access my local system Postgres from Docker

When I use Docker on my own machine, I want to access my system’s Postgres. This is prior to deploying my images to AWS, where I want to use RDS.

I have followed the steps on Docker’s Quickstart: Docker Compose and Rails page with the exception that, because I was working with an existing application, I didn’t create the initial minimal Gemfile that just loads Rails.

  • Put docker image into user local directory
  • Connect many Jenkins servers with different branches of the same github repo
  • Error “No module named MySQLdb” on docker
  • How can I modify Docker's internal IP on a Mac?
  • How to include dependencies in .NET Core app docker image?
  • Xdebug IP changed when VPN Is active
  • Outside of the Docker container, Postgres is installed and working correctly:

    $ which postgres
    $ which pg_restore
    $ which pg_dump

    The app has been working fine without Docker for years. To test the app outside of the Docker container once more, I have comment out one line of Docker’s recommended docker-compose.yml file as follows:

    development: &default
      adapter: postgresql
      encoding: unicode
      database: postgres
      pool: 5
      username: postgres
      # host: db
      <<: *default
      database: myapp_test

    When I do that, I can run rake db:test:clone (for example) outside of the container no worries:

    But when I uncomment that line and connect to a TTY in the container, I’m in all sorts of pain:

    /myapp# rake db:test:clone                                          
    pg_dump -s -x -O -f /myapp/db/structure.sql postgres
    Please check the output above for any errors and make sure that `pg_dump` is installed in your PATH and has proper permissions.
    /myapp# which postgres
    /myapp# which pg_restore
    /myapp# which pg_dump

    What can I do?

    The Dockerfile, based on Docker’s instructions with a few extra binaries required by my existing app, is:

    FROM ruby:2.3.1-slim
    RUN echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selections
    RUN apt-get update
    RUN apt-get install -qq -y --no-install-recommends --fix-missing apt-utils
    RUN apt-get install -qq -y --no-install-recommends --fix-missing build-essential
    RUN apt-get install -qq -y --no-install-recommends --fix-missing git
    RUN apt-get install -qq -y --no-install-recommends --fix-missing xvfb
    RUN apt-get install -qq -y --no-install-recommends --fix-missing qt5-default
    RUN apt-get install -qq -y --no-install-recommends --fix-missing libqt5webkit5-dev
    RUN apt-get install -qq -y --no-install-recommends --fix-missing gstreamer1.0-plugins-base
    RUN apt-get install -qq -y --no-install-recommends --fix-missing gstreamer1.0-tools
    RUN apt-get install -qq -y --no-install-recommends --fix-missing gstreamer1.0-x
    RUN apt-get install -qq -y --no-install-recommends --fix-missing nodejs
    RUN apt-get install -qq -y --no-install-recommends --fix-missing libpq-dev
    RUN mkdir /myapp
    WORKDIR /myapp
    ADD Gemfile /myapp/Gemfile
    ADD Gemfile.lock /myapp/Gemfile.lock
    RUN bundle install
    ADD . /myapp

    Related: I can see redis-cli outside the container but not inside it. This will be an issue too once I solve the Postgres situation.


    When I go in like this…

    docker-compose run web rake db:create
    docker-compose run web rake db:test:clone

    …it seems I can access Postgres, but not pg_dump and other utilities required for rake db:test:clone, rake db:test:load or similar.

    Update II

    I added the following to the Dockerfile:

    RUN apt-get install -qq -y --no-install-recommends --fix-missing postgresql-client

    Now, instead of giving me a “can’t be found” error, it says:

    pg_dump: server version: 9.5.4; pg_dump version: 9.4.9
    pg_dump: aborting because of server version mismatch

    This is an issue you can find discussed elsewhere, but not in the context of Docker, which is crucial here.

    Would it help if I downgraded my Mac’s Postgres to the version fetched by apt-get? Upgrading the version installed by apt-get appears not to be an option:

    Step 15 : RUN apt-get install -qq -y --no-install-recommends --fix-missing postgresql-client-9.5
     ---> Running in 614b88701710
    E: Unable to locate package postgresql-client-9.5
    E: Couldn't find any package by regex 'postgresql-client-9.5'

  • Proper way of automatically adding indexes docker service with Gitlab CI
  • Expose my docker container to my external IP on GCP
  • Knife Container issue
  • How to use volume in docker compose for postgres?
  • Why node_modules is empty after docker build?
  • How to run docker image , take ssh of container and bind port to “49” in single cmd?
  • One Solution collect form web for “How to access my local system Postgres from Docker”

    This isn’t “yer doin it wrong”, this is tricky. I ran through the docker intro you linked and I couldn’t get it to work.

    Here’s what I did for our app. I built our app off of a base image named after the app. The base image is named after the app. So using your example it’d be called myapp_base. I don’t see your docker-compose.yml file here so I’ll make some assumptions.

    image: your_company/myapp_base
    - db
    - .:/myapp
    # ports:
    # - "3002:3002"
    # env_file:
    # - .myapp.env

    Port and env_file are not needed but they might come in handy later. Myapp_base is built with a weird workflow that pulls in deps but basically throws in postgres-client if the app is declared to need postgres. It’s in chef but it’s basically myapp runlist needs postgresql. We’re actually moving away from this and trying to put the setup stuff in scripts and the Dockerfile itself.

    So … basically just add the apt-get install postgres-client to your Dockerfile, tag it as myapp_base and use that in your docker-compose.yml file. Then run the docker-compose run web rake db stuff and psql will be found.

    Docker will be the best open platform for developers and sysadmins to build, ship, and run distributed applications.