How to force pull docker images in DC OS?

For docker orchestration, we are currently using mesos and chronos to schedule job runs.

Now, we dropped chronos and try to set it up via DCOs, using mesos and metronome.

  • Spring boot connection reset after a few minutes in docker container
  • How can I exclude a sub-directory with a .dockerignore file
  • Docker IBM SDK-8 Image - Getting thread/memory dump
  • How to route to host in docker container during 'docker build'
  • Ansible error: KeyError: 'ApiVersion'
  • Docker Compose - Image name
  • In chronos, I could activate force pulling a docker image via its yml config:

    container:
      type: docker
      image: registry.example.com:5001/the-app:production
      forcePullImage: true
    

    Now, in DC/OS using metronome and mesos, I also want it to force it to always pull the up-to-date image from the registry, instead of relying on its cached version.

    Yet the json config for docker seems limited:

    "docker": {
      "image": "registry.example.com:5001/the-app:production"
    },
    

    If I push a new image to the production tag, the old image is used for the job run on mesos.

    Just for the sake of it, I tried adding the flag:

    "docker": {
      "image": "registry.example.com:5001/my-app:staging",
      "forcePullImage": true
    },
    

    yet on the put request, I get an error:

    http PUT example.com/service/metronome/v1/jobs/the-app < app-config.json 
    
    HTTP/1.1 422 Unprocessable Entity
    Connection: keep-alive
    Content-Length: 147
    Content-Type: application/json
    Date: Fri, 12 May 2017 09:57:55 GMT
    Server: openresty/1.9.15.1
    
    {
        "details": [
            {
                "errors": [
                    "Additional properties are not allowed but found 'forcePullImage'."
                ],
                "path": "/run/docker"
            }
        ],
        "message": "Object is not valid"
    }
    

    How can I achieve that the DC OS always pulls the up-to-date image? Or do I have to always update the job definition via a unique image tag?

  • Nginx not serving static files with docker-compose and django (13: Permission denied)
  • In a Docker Compose multi-container application, how to prevent Postgres from running statements previously issued
  • How can I map a volume in a running docker container?
  • Get internal IP address of Docker container using Ansible
  • Unable to connect to mongodb container using MongoJavaDriver
  • How to use Nomad with Nvidia Docker?
  • 3 Solutions collect form web for “How to force pull docker images in DC OS?”

    As this doesn’t currently seem possible, I created a quick and dirty solution to easily update the image tag for all the registered jobs using typescript and request-promise library.

    Basically I fetch all the jobs from the metronome api, filter them by id starting with my app name, and then change the docker image, and issue for each changed job a PUT request to the metronome api to update the config.

    Here’s my solution:

    const targetTag = 'stage-build-1501'; // currently hardcoded, should be set via jenkins run
    const app = 'my-app';
    const dockerImage = `registry.example.com:5001/${app}:${targetTag}`;
    
    interface JobConfig {
        id: string;
        description: string;
        labels: object;
        run: {
            cpus: number,
            mem: number,
            disk: number,
            cmd: string,
            env: any,
            placement: any,
            artifacts: any[];
            maxLaunchDelay: 3600;
            docker: { image: string };
            volumes: any[];
            restart: any;
        };
    }
    
    const rp = require('request-promise');
    
    const BASE_URL = 'http://example.com';
    const METRONOME_URL = '/service/metronome/v1/jobs';
    const JOBS_URL = BASE_URL + METRONOME_URL;
    
    const jobsOptions = {
        uri: JOBS_URL,
        headers: {
            'User-Agent': 'Request-Promise',
        },
        json: true,
    };
    
    const createJobUpdateOptions = (jobConfig: JobConfig) => {
        return {
            method: 'PUT',
            body: jobConfig,
            uri: `${JOBS_URL}/${jobConfig.id}`,
            headers: {
                'User-Agent': 'Request-Promise',
            },
            json: true,
        };
    };
    
    rp(jobsOptions).then((jobs: JobConfig[]) => {
        const filteredJobs = jobs.filter((job: any) => {
            return job.id.includes('job-prefix.'); // I don't want to change the image of all jobs, only for the same application
        });
    
        filteredJobs.map((job: JobConfig) => {
            job.run.docker.image = dockerImage;
        });
    
        filteredJobs.map((updateJob: JobConfig) => {
            console.log(`${updateJob.id} to be updated!`);
            const requestOption = createJobUpdateOptions(updateJob);
            rp(requestOption).then((response: any) => {
                console.log(`Updated schedule for ${updateJob.id}`);
            });
        });
    
    });
    

    I think the "forcePullImage": true should work with the docker dictionary.

    Check:
    https://mesosphere.github.io/marathon/docs/native-docker.html

    Look at the “force pull option”.

    Docker will be the best open platform for developers and sysadmins to build, ship, and run distributed applications.