Docker SIGTERM not being delivered to node.js/coffee app when started with flags
I’ve set up listeners in my application to catch SIGTERM, SIGINT and SIGUSR2:
# kill process.on 'SIGTERM', -> killExecutors 'SIGTERM' # ctrl + c process.on 'SIGINT', -> killExecutors 'SIGINT' # nodemon signal process.on 'SIGUSR2', -> killExecutors 'SIGUSR2'
It works as expected. When I run it inside a docker instance:
FROM node:4.4.7 MAINTAINER Newborns <firstname.lastname@example.org> COPY . /src EXPOSE 7733 WORKDIR /src RUN npm install CMD ["./node_modules/.bin/coffee", "feeder.coffee"]
Everything works fine, too. BUT, when I add a node flag to the execution
FROM node:4.4.7 MAINTAINER Newborns <email@example.com> COPY . /src EXPOSE 7733 WORKDIR /src RUN npm install CMD ["./node_modules/.bin/coffee", "--nodejs", "--max_old_space_size=384", "feeder.coffee"]
it stops catching the signals. I’ve tried to change de CMD exec form to
CMD ./node_modules/.bin/coffee --nodejs --max_old_space_size=384 feeder.coffee
but still doesn’t work. What changes between the execution with and without flags?
Actually, what happens is that docker starts one process when no flags are passed
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 1 4.2 1.0 960940 85424 ? Ssl 20:21 0:01 node ./node_modules/.bin/coffee feeder.coffee root 16 0.1 0.0 20220 2884 ? Ss 20:22 0:00 bash root 20 0.0 0.0 17500 2064 ? R+ 20:22 0:00 ps -aux
and two processes when flags are passed
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 1 0.1 0.3 707704 25272 ? Ssl 20:17 0:00 node ./node_modules/.bin/coffee --nodejs --max_old_space_size=384 feeder.coffee root 10 1.7 1.1 965900 90068 ? Sl 20:17 0:01 /usr/local/bin/node --max_old_space_size=384 /src/node_modules/.bin/coffee feeder.coffee
question is: why?
One Solution collect form web for “Docker SIGTERM not being delivered to node.js/coffee app when started with flags”
coffee executable when you need to use extended
node options to avoid the technicalities of signals with forked processes under Docker.
node --max_old_space_size=384 app.js
Now, onto the technicalities…
Docker and signals
The initial process in a container is PID 1 in the containers namespace. PID 1 (or the init process) is treated as a special case by the kernel with regards to signal handling.
- If the init process does not install a signal handler, that signal won’t be sent to it.
- Signals do not propagate automatically from an init process, the process must manage this.
So a docker process is expected to handle signals itself.
As you have noted,
coffee will fork a child
node process when it has the
--nodejs option to be able to pass the extra options on.
This initially presents some odd behaviour outside of docker with signal handling (on osx at least). A
SIGTERM will be forwarded onto the child but also kill the parent
coffee process immediately, no matter how you handle the signal in your code (which is running in the child).
A quick example
process.on 'SIGTERM', -> console.log 'SIGTERM' process.on 'SIGINT', -> console.log 'SIGINT' cb = -> console.log "test" setTimeout cb, 5000
When you run this and ctrl–c, the signal is forwarded on to the child process and handled. The parent process closes immediately though and returns to the shell.
$ coffee --nodejs --max_old_space_size=384 block_signal_coffee.coffee ^C SIGINT $ <5ish second pause> test
Then the child process with your code continues running in the background for 5 seconds and eventually outputs
The main problem is the parent
coffee process does not handle any signals in code, so the signals don’t arrive and are not forwarded onto the child. This probably requires a change to coffeescript’s launcher code to fix.
The signal quirk
coffee --nodejs presents outside of Docker could also be bad if it happened under Docker. If the main container process (the parent of the fork) exits before your signal handlers have a chance to complete in the child, the container will close around them. This scenario is unlikely to happen if the above problem is fixed by just forwarding signals onto the child.