Managing resources (database, elasticsearch, redis, etc) for tests using Docker and Jenkins

We need to use Jenkins to test some web apps that each need:

  • a database (postgres in our case)
  • a search service (ElasticSearch in our case, but only sometimes)
  • a cache server, such as redis

So far, we’ve just had these services running on the Jenkins master, but this causes problems when we want to upgrade Postgres, ES or Redis versions. Not all apps can move in lock step, and we want to run the tests on new versions before committing to move an app in production.

  • Exposing local Docker containers on internet (There are two containers linked to each other).
  • Dockerfile Dynamic Registry for Image
  • mkdir .ssh in a Dockerfile, folder is not there?
  • Dockerfile : not able to use add to map local file to container
  • issue with Docker registry having ngnix authentication
  • Docker vs. Rocket in development
  • What we’d like to do is have these services provided on a per-job-run basis, each one running in its own container.

    What’s the best way to orchestrate these containers?

    • How do you start up these ancillary containers and tear them down, regardless of whether to job succeeds or not?

    • how do you prevent port collisions between, say, the database in a run of a job for one web app and the database in the job for another web app?

  • Upload to s3 from docker container running inside ECS not working
  • Running mysql_upgrade in Docker?
  • Issue while indexing data in solr in docker cluster setup
  • docker on ubuntu 16.04 OCI error when launching container
  • Asked password when ssh to container
  • docker compose variable substitution
  • One Solution collect form web for “Managing resources (database, elasticsearch, redis, etc) for tests using Docker and Jenkins”

    Check docker-compose and write a docker-compose file for your tests.
    The latest network features of Docker (private network) will help you to isolate builds running in parallel.

    However, start learning docker-compose as if you only had one build at the same time. When confident with this, look further for advanced docker documentation around networking.

    Docker will be the best open platform for developers and sysadmins to build, ship, and run distributed applications.