Running Jenkins tests in Docker containers build from dockerfile in codebase

I want to deploy a continuous integration platform based on Jenkins. As I have various kinds of projects (PHP / Symfony, node, angular, …) and as I want these tests to run both locally and on Jenkins, I was thinking about using Dockers containers.

The process I’m aiming for is :

  • How do I cleanly shut down a mesos-slave?
  • Can't reach ActiveMQ Port on Docker from my host
  • Docker container has no TMPDIR environmental variable
  • Docker pull error
  • Install gems on docker
  • How to create populated MySQL Docker Image on build time
    • A merge request is opened on Github / Gitlab
    • A webhook notifies Jenkins of the merge request
    • Jenkins pulls the repo, builds the containers and runs a shell script to execute the tests
    • Once the tests are finished, Jenkins retrieves the results from one of the containers (through a shared volume) and process the results.

    I do not want Jenkins to be in a container.

    With this kind of process, I’m hoping to be able to run very easily the tests on each developer machine with something like a docker-composer up and then in one of the container ./tests all.

    I’m not very familiar with Jenkins. I’ve read a lot of documentation, but most of them suggested to define Jenkins slaves for each kind of projects beforehand. I would like everything to be as dynamic as possible and require as less configuration on Jenkins as possible.

    I would appreciate a description of your test process if you have ever implemented something similar. If you think what I’m aiming for is impossible, I would also appreciate if you could explain to me why.

  • Load balance docker containers in Mesos cluster with HAproxy
  • Can not ping docker container from another host which is in the same LAN
  • OpenShift 3.1 - Prevent Docker from caching curl resource
  • Is there any way to disable a service in docker-compose.yml
  • Mapping an existing local neo4j database to a neo4j docker container
  • Dockerfile with an external arguments file
  • One Solution collect form web for “Running Jenkins tests in Docker containers build from dockerfile in codebase”

    A setup I suggest is Docker in Docker.

    The base is a derived Docker image, which extends the jenkins:2.x image by adding a Docker commandline client.
    The Jenkins is started as a container with its home folder (a folder e.g. /var/jenkins_home mounted from the Docker host) and the Docker socket file to be able to start Docker containers from Jenkins build jobs.

    docker run -d --name jenkins -v /var/jenkins_home:/var/jenkins_home -v /var/run/docker.sock:/var/run/docker.sock ... <yourDerivedJenkinsImage>
    

    To check, if this setup is working just execute following command after starting the Jenkins container:

    docker exec jenkins docker version
    

    If the “docker version” output does NOT show:

    Is the docker daemon running on this host?

    Everythin is fine.

    In your build jobs, you could configure the process you mentioned above. Let Jenkins simply check out the repository. The repository should contain your build and test scripts.

    Use a freestyle build job with a shell execution. A shell execution could look like this:

    docker run --rm --volumes-from jenkins <yourImageToBuildAndTestTheProject> bash $WORKSPACE/<pathToYourProjectWithinTheGitRepository>/build.sh 
    

    This command simply starts a new container (to build and/or test your project) with the volumes from jenkins. Which means that the cloned repository will be available under $WORKSPACE. So if you run “bash $WORKSPACE/<pathToYourProjectWithinTheGitRepository>/build.sh” your project will be built within a container of “yourImageToBuildAndTestTheProject”. After running this, you could start other containers for integration tests or combine this with “docker-compose” by installing it on the derived Jenkins image.

    Advantages are the minimal configuration affort you have within Jenkins – only the SCM configuration for cloning the GIT repository is required. Since each Jenkins job uses the Docker client directly you could use for each project one or Docker image to build and/or test, WITHOUT further Jenkins configuration.

    If you need additional configuration e.g. SSH keys or Maven settings, just put them on the Docker host and start the Jenkins container with the additional volumes, which contain those configuration files.

    Using this Docker option within the shell execution of your build jobs:

    --volumes-from jenkins
    

    Automatically adds workspace and configuration files to each of your build jobs.

    Docker will be the best open platform for developers and sysadmins to build, ship, and run distributed applications.