Any hook for Docker killed by out of memory

I’m running a docker for a daemon job. And the container will be killed every several hours. I’d like to add some hook (callback), such as:

restart the container and then run some commands on the restarted container

  • Share Existing Host Directory with Docker Container?
  • Docker and netstat: netstat is not showing ports, exposed by docker containers
  • How to change docker bridge address space (subnet)?
  • How to use --volume option with Docker Toolbox on Windows?
  • Docker delete images issue
  • Unable to push windows docker image to Artifactory
  • Is it possible to do that with Docker?

    Otherwise, is there any better approach to detect the behaviour with Python or Ruby?

    java invoked oom-killer: gfp_mask=0xd0, order=0, oom_score_adj=0
    java cpuset=bcb33ac552c23cfa531814fbc3a64ae5cd8d85aa19245e1560e0ce3e3310c798 mems_allowed=0
    CPU: 3 PID: 14182 Comm: java Not tainted 4.1.0-x86_64-linode59 #1
     0000000000000000 ffff8800dc520800 ffffffff8195b396 ffff880002cf5ac0
     ffffffff81955e58 ffff8800a2918c38 ffff8800f43c3e78 0000000000000000
     ffff8800b5f687f0 000000000000000d ffffea0002d7da30 ffff88005bebdec0
    Call Trace:
     [<ffffffff8195b396>] ? dump_stack+0x40/0x50
     [<ffffffff81955e58>] ? dump_header+0x7b/0x1fe
     [<ffffffff8119655d>] ? __do_fault+0x3f/0x79
     [<ffffffff811789d6>] ? find_lock_task_mm+0x2c/0x7b
     [<ffffffff81961c55>] ? _raw_spin_unlock_irqrestore+0x2d/0x3e
     [<ffffffff81178dee>] ? oom_kill_process+0xc5/0x387
     [<ffffffff811789d6>] ? find_lock_task_mm+0x2c/0x7b
     [<ffffffff811b76be>] ? mem_cgroup_oom_synchronize+0x3ad/0x4c7
     [<ffffffff811b6c92>] ? mem_cgroup_is_descendant+0x29/0x29
     [<ffffffff811796e7>] ? pagefault_out_of_memory+0x1c/0xc1
     [<ffffffff81963e58>] ? page_fault+0x28/0x30
    Task in /docker/bcb33ac552c23cfa531814fbc3a64ae5cd8d85aa19245e1560e0ce3e3310c798 killed as a result of limit of /docker/bcb33ac552c23cfa531814fbc3a64ae5cd8d85aa19245e1560e0ce3e3310c798
    memory: usage 524288kB, limit 524288kB, failcnt 14716553
    memory+swap: usage 0kB, limit 9007199254740988kB, failcnt 0
    kmem: usage 0kB, limit 9007199254740988kB, failcnt 0
    Memory cgroup stats for /docker/bcb33ac552c23cfa531814fbc3a64ae5cd8d85aa19245e1560e0ce3e3310c798: cache:72KB rss:524216KB rss_huge:0KB mapped_file:64KB writeback:0KB inactive_anon:262236KB active_anon:262044KB inactive_file:4KB active_file:4KB unevictable:0KB
    [ pid ]   uid  tgid total_vm      rss nr_ptes nr_pmds swapents oom_score_adj name
    [14097]  1000 14097     5215       20      17       3       47             0
    [14146]     0 14146    11960        0      30       3      101             0 sudo
    [14150]  1000 14150     1112        7       8       3       22             0 xvfb-run
    [14162]  1000 14162    51929    11220      90       3       95             0 Xvfb
    [14169]  1000 14169   658641    18749     120       6        0             0 java
    [14184]  1000 14184    28364      555      58       3        0             0 fluxbox
    [24639]  1000 24639     5212       59      16       3        0             0 bash
    Memory cgroup out of memory: Kill process 14169 (java) score 96 or sacrifice child
    Killed process 14169 (java) total-vm:2634564kB, anon-rss:74996kB, file-rss:0kB

  • Containers dependency with etcd keys
  • Can Docker map multiple container ports to one host port?
  • How to add `--registry-mirror` when starting docker from “Docker quickstart terminal”?
  • How to pass parameters from docker to angular js?
  • When and when not to use tty in Docker Remote API
  • Is jwilder/nginx-proxy still actual since embedded DNS server in user-defined networks was introduced?
  • One Solution collect form web for “Any hook for Docker killed by out of memory”

    Docker itself doesn’t have any such mechanism. All you can do is pass the --restart flag to tell Docker when it should try to bring a failed container back.

    However, most places where you want to keep a container up you’ll want something more complex than the --restart flag anyway. Once you’re using runit or systemd to manage your containers it’s easy to add in a little extra shell code to figure out why the last invocation crashed and take some special actions based on that.

    Docker will be the best open platform for developers and sysadmins to build, ship, and run distributed applications.