How to aggregate logs for all docker containers in mesos
I have multiple microservices written in node and microservices are installed into the docker container and we are using Mesos+Marathon for clustering.
How can I aggregate the logs of all the containers(microservices) on different instance.?
3 Solutions collect form web for “How to aggregate logs for all docker containers in mesos”
This is a broad question but I suggest you setup an Elastic Search, Logstash, Kibana stack (ELK)
Then on each one of your containers you can run the logstash forwarder/shipper to send logs to your logstash frontend.
Logs get stored in Elastic Search and then you search for them using Kibana or the Elastic Search API
Hope it helps.
We’re using Docker+Mesos as well and are shipping all logs to a log analytics service (it’s the service the company I work for offers, http://logz.io). There a couple of ways to achieve that:
– Have a log shipper agent within each docker – an agent like rsyslog, nxlog, logstash, logstash-forwarder – that agent would ship data to a central logging solution
– Create a Docker that is running the shipper agent (like rsyslog, nxlog, logstash, logstash-forwarder) and that agent reads logs from all Dockers on each machine and ships them to a central location – this is the path we’re taking
I am also doing some Docker + Mesos + Marathon work so, I guess, I am going to face same doubt that you have.
I don’t know if there’s any native solution yet. But there’s a blog by the folks at elastic.io on how they went about solving this issue.
Here’s the link – Log aggregation for Docker containers in Mesos / Marathon cluster