configuration management in Docker Containers

Currently I have configurations files stored in GitHub. I have a single json file with format as below
{
DEV:
{ key1 : val1, key2 : val2 },
PROD:
{ key1 : val1, key2 : val2 }
}.

My build system clones the git repo, builds the projects and creates a Docker image and stores in private Docker registry. I have the jar files and configuration files copied into Docker image. Whenever I spin up a container I inject an environment variable (ENV=DEV/PROD) which my code uses to read configs based on environment.

  • Using Gitlab CD and Docker
  • Jenkins Pipeline: Postman Test script fails with “connect ECONNREFUSED ip:port”
  • How do I run a salt state on a running docker container
  • Consul cluster automatic bootstrap on docker
  • How can I create a Docker image based on a git tag in the public Registry?
  • how to setup continuos deployment from docker-hub to AWS ECS?
  • I have couple of questions here:

    • What is the best way to maintain environment specific configuration?

    • Configuration files may contain sensitive data such as api keys and secrets how can I encrypt and store and decrypt when I build Docker image?

    • If I want to change some configuration I need to trigger build because my configuration file is placed inside Docker image. Can I place config files outside Docker container as a volume so that I can replace config file and restart container so that code reads updated configs? If I want to place outside of Docker container can I still use any cluster management tools for container orchestration/management (Kubernetes/ECS)?
    • What is the way to make apps running in Docker containers read updated configs by just restarting the container instead of building a new docker image and deploying a new container?

  • Push image vs building on system itself, docker
  • Dockerfile entrypoint script arguments in docker run
  • How to use Docker in the development/deployment workflow?
  • docker swarm mode multiple services same port
  • Simple Continuous Deliver procedure for docker in aws
  • I've started my Docker container but it is not staying up
  • 2 Solutions collect form web for “configuration management in Docker Containers”

    1. ENV variables are fine. IF you have really complex system you can spin Consul and/or Vault to manage secure configuration, but if your sistem is small, the mess do not worth the effort.
    2. You can encrypt all configuration into hash, and decrypt it with the key only know to application.
    3. Use docker-compose. It allows perfect mechanism – the one you actually requesting here, example of the docker-compose.yml:

      version: "2"  
      services:   
        server:
          hostname: server
          container_name: server
          image: serverBla
          build: ./server
          env_file:
            - ./config/config.env
      

    The file ./config/config.env is your “dynamic” configuration. You need just recreate the container to refresh the values inside of your app.
    As far as I know docker-compose works well with anything you mentioned (I am not familiar with Kubernetes, but I am sure there is such thing, it’s basic for container api).

    My answer to most of your questions would be “Vault by Hashicorp”

    You could spin up a Vault cluster in your environment and have your container connect to, and retrieve secrets from Vault at boot-up and runtime. The most trivial way of implementing the secret retrieval is to have the secrets read in to the container at boot up time and pushed in to the container’s ENV.

    If you go one step further, you can use a language level vault client library to retrieve and refresh secrets as your application is running, removing the necessity to restart your container to retrieve new secrets.

    How does this fit in to what you asked?

    • What is the best way to maintain environment specific configuration?

    I would say Vault. You can write secrets out to a file or folder using Puppet with Hiera if you wanted, but that means writing secrets to disk which is not ideal. TMK K8s stores secrets unencrypted at rest, which is also not ideal. I would prefer Vault here. Each environment has its own Vault cluster, and there’s your ‘environment specific configuration’.

    • Configuration files may contain sensitive data such as api keys and
      secrets how can I encrypt and store and decrypt when I build Docker
      image?

    See above. Vault stores secrets encrypted at rest. Use your Vault token (which you can mount in to your container through a volume if you want) to access the secrets and decrypt them. You can then push them to your ENV before starting your containers main process, or do a tighter integration and make Vault part of your application’s logic.

    • If I want to change some configuration I need to trigger build
      because my configuration file is placed inside Docker image. Can I
      place config files outside Docker container as a volume so that I can
      replace config file and restart container so that code reads updated
      configs? If I want to place outside of Docker container can I still
      use any cluster management tools for container
      orchestration/management (Kubernetes/ECS)?

    The answer to this is highly dependent on what the configuration is containing. If it’s specific to the code revision, I’d highly recommend bundling it with your code in order to prevent any mismatches in code and config.

    If it’s effectively independent of code (e.g: tweaking values or secrets), I’d throw them in Vault – why put them in a file when you can store them in an encrypted secret management system?

    However if your heart is set on files, you can easily mount them in from a known volume to a known path, and all the scheduling systems (including Nomad now) allow volume mounting with deployed containers.

    • What is the way to make apps running in Docker containers read
      updated configs by just restarting the container instead of building
      a new docker image and deploying a new container?

    With Vault: envconsul is your friend if you want the secrets only changed at boot up time.

    See above for my opinion on run-time changes to config values.

    The tl;dr of the actual execution of this is to create a ‘startup’ script which is the entrypoint for your container. The startup script will retrieve the secrets from whatever management system you choose (volume mounts, Vault, a file, whatever), and ready them for your application to read.

    The startup script then starts your main app’s run process.

    Hope this helps!

    PS: I don’t work for Hashicorp, but I do like their software for working with containers. A lot of what you can get by buying in to Hashicorp’s disparate software you can also get from K8s as a single solution. Single apps (Hashicorp) allow you to pick and mix the bits of the solution you want, but also requires a higher maintenance overhead. K8s is a pretty heavy lock-in, but does most of the things you’ll likely want without needing to learn anything else.

    Good luck

    Docker will be the best open platform for developers and sysadmins to build, ship, and run distributed applications.