Beanstalk app CPU spikes
We are running a docker container on AWS elastic beanstalk, which was running fine for a few weeks but suddenly started to experience very sudden CPU spikes (from ~5% to ~60% in a matter of minutes), who sometimes drop back down quickly, and sometimes stay high for enough time to produce an autoscaling event and spin up a few instances for extra help (Which are terminated some time after that when the CPU spike dies down).
The funny thing is, I wanted to investigate the problem today, so I’ve sshed into every instance (4 in total) and ran
top on all of them, trying to locate the CPU consuming process, and was surprised to discover all instances have ~15% CPU busy (system + user combined), while the EBS monitoring page still shows the servers are at 60% CPU.
I’ve measured these figures for the good part of the hour, making sure the CPU high load stays high, while the
top command still shows low values.
I’ve also tried to measure CPU for a while using the advice found here -https://askubuntu.com/questions/22021/how-to-log-cpu-load, and got the same very low CPU stats when querying the server directly.
My question is – is it possible AWS monitoring system is not showing me accurate data? Is there anyway to verify the data displayed in the monitoring page?
Any help would be appreciated.