How to aggregate logs for all docker containers in mesos

I have multiple microservices written in node and microservices are installed into the docker container and we are using Mesos+Marathon for clustering.

How can I aggregate the logs of all the containers(microservices) on different instance.?

  • error during connect: … http: server gave HTTP response to HTTPS client on any docker command with remote host
  • Docker add warfile to official Tomcat image
  • Change hostname after running a container
  • Provide only GRANT EXECUTE 'procedure' to a user (No select or insert permissions)
  • Backup Mongo in a Docker container
  • Dockerized selenium browser cannot access Capybara test url
  • Best way to execute command line type tasks from a Web App [closed]
  • Docker Port forwarding doesn't call css and images
  • LogStash container restarts on Kube pod
  • Docker does not have default machine in Windows 10
  • How to modify the `--registry-mirror` of a running docker machine?
  • docker-maven-plugin: How to define which tests are run for which image?
  • 3 Solutions collect form web for “How to aggregate logs for all docker containers in mesos”

    This is a broad question but I suggest you setup an Elastic Search, Logstash, Kibana stack (ELK)

    https://www.elastic.co/products/elasticsearch

    https://www.elastic.co/products/logstash

    https://www.elastic.co/products/kibana

    Then on each one of your containers you can run the logstash forwarder/shipper to send logs to your logstash frontend.

    Logs get stored in Elastic Search and then you search for them using Kibana or the Elastic Search API

    Hope it helps.

    We’re using Docker+Mesos as well and are shipping all logs to a log analytics service (it’s the service the company I work for offers, http://logz.io). There a couple of ways to achieve that:
    – Have a log shipper agent within each docker – an agent like rsyslog, nxlog, logstash, logstash-forwarder – that agent would ship data to a central logging solution
    – Create a Docker that is running the shipper agent (like rsyslog, nxlog, logstash, logstash-forwarder) and that agent reads logs from all Dockers on each machine and ships them to a central location – this is the path we’re taking

    I am also doing some Docker + Mesos + Marathon work so, I guess, I am going to face same doubt that you have.

    I don’t know if there’s any native solution yet. But there’s a blog by the folks at elastic.io on how they went about solving this issue.

    Here’s the link – Log aggregation for Docker containers in Mesos / Marathon cluster

    Docker will be the best open platform for developers and sysadmins to build, ship, and run distributed applications.