Prometheus – how to monitor other docker containers

I want to use Prometheus to monitor my docker containers. I can run Prometheus with Grafana but I do not know how to instruct it to monitor other docker containers.
If possible I would like to see some example. For instance I have Ubuntu container running on my host as well as Gentoo container. How can I tell Prometheus to monitor them, or perhaps to monitor some application(s) running inside?
Thanks in advance for your time and patience with me.

  • Getting docker pull to default pull from a private registry?
  • Nginx docker container returning corrupt JSON file, renamed file
  • Sonatype Nexus3 docker image - How to setup an internal docker repository
  • kubernetes stateful set connection
  • npm install error with docker - kurento application
  • Kubernetes - container communication within a pod using names instead of 'localhost'?
  • How can kubernetes dynamically expose my docker port?
  • docker run complains file not found
  • Jenkins build parameter not working in quotes
  • How does rancher schedule services to specific hosts?
  • Parsing Docker image tag into component parts [duplicate]
  • Unknown filesystem type on /dev/mapper/docker-202
  • One Solution collect form web for “Prometheus – how to monitor other docker containers”

    You could use cadvisor, which provides container users an understanding of the resource usage and performance characteristics of their running containers.

    A very good article about setting up Prometheus to monitor Docker is using this architecture:

    cAdvisor with prometheus

    Briefly, the idea is to collect information about containers using cAdvisor and put them into Prometheus database. Grafana will query the Prometheus database and render monitoring charts / values.

    To collect data from cAdvisor to Prometheus, you will have to edit a configuration file (prometheus.yml):

    scrape_configs:
      - job_name: 'cadvisor'
        scrape_interval: 5s
        static_configs:
          - targets: ['cadvisor:8080']
    

    When you have some data in Prometheus, the you have to use Grafana to view it. A (short) example of monitoring json queries you could import into Grafana is as follow:

    Get % of user CPU :

      "targets": [
        {
          "expr": "sum(rate(container_cpu_user_seconds_total{image!=\"\"}[1m])) / count(node_cpu{mode=\"system\"}) * 100",
          "interval": "10s",
          "intervalFactor": 1,
          "legendFormat": "",
          "refId": "A",
          "step": 10
        }
      ]
    

    Get % of RAM used :

      "targets": [
        {
          "expr": "(sum(node_memory_MemTotal) - sum(node_memory_MemFree+node_memory_Buffers+node_memory_Cached) ) / sum(node_memory_MemTotal) * 100",
          "interval": "10s",
          "intervalFactor": 2,
          "legendFormat": "",
          "refId": "A",
          "step": 20
        }
      ]
    

    For complete json data (too long to be posted here), you can clone this repository :

    git clone https://github.com/stefanprodan/dockprom
    

    And try to import this Grafana json.

    I’m currently using this architecture to monitor a docker swarm mode cluster in production, and here is the output of the monitoring that you can find in the github repository.

    prometheus with grafana

    Docker will be the best open platform for developers and sysadmins to build, ship, and run distributed applications.