Beanstalk app CPU spikes

We are running a docker container on AWS elastic beanstalk, which was running fine for a few weeks but suddenly started to experience very sudden CPU spikes (from ~5% to ~60% in a matter of minutes), who sometimes drop back down quickly, and sometimes stay high for enough time to produce an autoscaling event and spin up a few instances for extra help (Which are terminated some time after that when the CPU spike dies down).

The funny thing is, I wanted to investigate the problem today, so I’ve sshed into every instance (4 in total) and ran top on all of them, trying to locate the CPU consuming process, and was surprised to discover all instances have ~15% CPU busy (system + user combined), while the EBS monitoring page still shows the servers are at 60% CPU.
I’ve measured these figures for the good part of the hour, making sure the CPU high load stays high, while the top command still shows low values.

  • Change .docker directory on Windows
  • Destroy Docker container from Marathon kills Mesos slave
  • Inconsistent runtime kernel parameters in DOCKER container and on host
  • Where does Elixir/erlang fit into the microservices approach? [closed]
  • Dockerfile: create ENV variable that a USER can see?
  • Can't open file when i try to run a shell script inside Docker container
  • I’ve also tried to measure CPU for a while using the advice found here -https://askubuntu.com/questions/22021/how-to-log-cpu-load, and got the same very low CPU stats when querying the server directly.

    My question is – is it possible AWS monitoring system is not showing me accurate data? Is there anyway to verify the data displayed in the monitoring page?

    Any help would be appreciated.

  • reading IP address of a host machine inside a docker container
  • how to sshfs/sftp on boot2docker or docker-toolbox
  • INTEL DEEP LEARNING SDK INSTALLATION FAILING
  • Docker user executing a process cannot be removed
  • Creating first docker container: Can't find host system file on build
  • Supervisor & Docker: How to exit Supervisor if a service doesn't start?
  • Docker will be the best open platform for developers and sysadmins to build, ship, and run distributed applications.