Fluentd capture stack traces from Docker

I’m trying to get Fluentd to parse Java stack traces, coming from the Docker logging driver, using in_tail and emit them as a single messages.

For the life of me, can’t figure out why it’s still splitting them up.

  • how to run two web apps in the same localhost with same port in docker?
  • How to quickly update a running docker-compose container
  • docker nginx stream balancer 404
  • Firefox in a docker container accessible from selenium in another
  • How to create a MongoDB docker and save the DB's configuration?
  • Docker hub automated build fails but locally not
  • This is a sample input, that’s being written to a file:

    2015-12-17T19:19:47+00:00   docker.java.ubuntu:15.10    {"log":"Exception in thread main java.lang.NullPointerException\r","container_id":"5a064eb23465350a11fe00b1f7787f5bd3e9f0182dd44c09516a72ab4006bd54","container_name":"/src-test_1.0.0.353_989549167.1","source":"stdout"}
    2015-12-17T19:19:47+00:00   docker.java.ubuntu:15.10    {"container_id":"5a064eb23465350a11fe00b1f7787f5bd3e9f0182dd44c09516a72ab4006bd54","container_name":"/src-test_1.0.0.353_989549167.1","source":"stdout","log":"        at com.example.myproject.Book.getTitle(Book.java:16)\r"}
    2015-12-17T19:19:47+00:00   docker.java.ubuntu:15.10    {"container_name":"/src-test_1.0.0.353_989549167.1","source":"stdout","log":"        at com.example.myproject.Author.getBookTitles(Author.java:25)\r","container_id":"5a064eb23465350a11fe00b1f7787f5bd3e9f0182dd44c09516a72ab4006bd54"}
    2015-12-17T19:19:47+00:00   docker.java.ubuntu:15.10    {"container_id":"5a064eb23465350a11fe00b1f7787f5bd3e9f0182dd44c09516a72ab4006bd54","container_name":"/src-test_1.0.0.353_989549167.1","source":"stdout","log":"        at com.example.myproject.Bootstrap.main(Bootstrap.java:14)\r"}
    2015-12-17T19:19:47+00:00   docker.java.ubuntu:15.10    {"container_id":"5a064eb23465350a11fe00b1f7787f5bd3e9f0182dd44c09516a72ab4006bd54","container_name":"/src-test_1.0.0.353_989549167.1","source":"stdout","log":"test\r"}
    

    This is the config I’m using for in_tail:

    <source>
      @type tail
      tag docker.multiline
      path /tmp/fluent/java*
      pos_file /tmp/fluent/log.pos
      refresh_interval 10
      format multiline
      format first_line /.*\"log\":\"[^\s].*/
      format /\"log\":\"(?<message>.+)\\r/
    </source>
    

    The regexes look correct to me and when I plug them into a regex tester, the first_line regex only matches the first and last lines of my sample, while the format regex matches every line, but only captures the stack trace info, as I’m expecting. However, they’re all coming out as separate messages, almost like first_line is matching every line, instead of the first and last.

  • docker build failing with Could not resolve 'archive.ubuntu.com'
  • aws ecs perfomance doesn't increase on single instance
  • Does docker stores all its files as “memory image”, as part of image, not disk file?
  • What are the disadvantages of a Docker container using the host network?
  • Docker-compose Daemon mode logs
  • Redis: ERR Rewriting config file: Permission denied when using Docker data volume
  • Docker will be the best open platform for developers and sysadmins to build, ship, and run distributed applications.