Unusual CPU load

Portal Home > Knowledgebase > Articles Database > Unusual CPU load

Posted by sifro, 02-28-2012, 04:21 AM
Hello, i suddendly started receiving emails from my panel (directadmin) like this one: If i go to "service monitor" i can see lots of apache PIDs, and the websites are very slow. If i restart apache, sites get back to normal, but after around 30 minutes, there are once again a lot of spawned apache children and an unusual high load. Any ideas on how to solve this? I haven't had any abnormal increase in the amount of traffic, so why did it start freaking out? Here are my system specs:
Posted by Farbyte, 02-28-2012, 06:43 AM
Sounds like this could possibly be a HTTP/S DOS attack. If you have lots of connections to the server sat in FIN_WAIT2 state during these high loads then you can pretty much guarantee it is a DOS attack. Run netstat to check during the high loads and block the offending IP/s. You won't normally see any increase in traffic with this type of attack. Keep us updated
Posted by Crashus, 02-28-2012, 10:30 AM
looks like apache consuming too much cpu, but according to your top output there are 21% idle which is strange, and httpd using only 19% per process, which usually jumps up to 100% during ddos attacks. anyhow - can you provide us with at least 'netstat -n|wc -l' output?
Posted by azsolt, 02-28-2012, 10:33 AM
You can check the TCP connections / IP with this code:
Posted by jibinthomas007, 03-08-2012, 05:39 AM
Adding to previous reply, block the IP address with unusual number of httpd connections. ie IP addresses with more than 40 connections.
Posted by ArturasLIX, 03-08-2012, 06:18 AM
I think you can first investigate which website is causing it. In most cases it'll be only one. For example: Such command will show active httpd connections to websites. Also you can check apache's server status page. After that I suggest you to tail that websites apache access log file, and block access with iptables.
Posted by Netonweb, 03-08-2012, 07:26 AM
Is there any high end CPU scripts running? This may be the reason why you're getting a hike in CPU usage. I also recommend you check up with cPanel.NET to see what accounts using what if you can I'm unsure how to do this as I don't have access to my server currently.
Posted by sifro, 03-08-2012, 01:54 PM
Hello, thank you very, very much to everybody. I'm narrowing down the problem, and i guess it's caused by... Googlebot! It opens lots of connections and then apache doesn't close them.... Here are the outputs of the commands you asked me: netstat -n|wc -l netstat -plan|grep :80 |awk '{print $5}' | awk -F : '{print $(NF-1)}' | sort | uniq -c | sort -n Apache server-status shows this: ... and the non machine-friendly version adds hundreds of lines like this one: 22-0 8559 0/58/149 _ 1.57 3 293 0.0 2.36 3.80 66.249.66.44 www.mydomain.com GET /modules.php?op=modload&name=Kalender&file=index&Date=01/31 66.249.66.44 is googlebot's IP. So, basically, googlebot opens connections, and my webserver doesn't closes them.... any ideas to solve this?
Posted by SiteServing, 03-08-2012, 06:34 PM
Might be some buggy high CPU consuming PHP script so you can check this too. Will be easier to find it out if PHP is running as CGI/suPHP.
Posted by sifro, 03-09-2012, 03:46 AM
I don't agree with you... please have a look at this: These values are, in order: So, please correct me if i'm wrong, that 661 means that 661 seconds have passed since the last time Googlebot used that slot to make a request... so basically the server is keeping alive a connection that has been idle for like 11 minutes!! Did i understand correctly? Is it normal? Btw, this command: returns nothing at the moment, even if i have that line in the apache server-status
Posted by AuzzHosting, 03-09-2012, 04:50 AM
http://blog.scoutapp.com/articles/20...-load-averages Read this, it's helpful to see if you should be worrying or not.
Posted by sifro, 03-09-2012, 04:55 AM
thasnk, i know for sure that i need to be worried, because site is very slow during these moments of load around 70%... the problem is: how to solve this?
Posted by AuzzHosting, 03-09-2012, 05:15 AM
Get a new server An overkill one and you will be fine! - joke a side! What are you server spec's? Also about Google bot, try closing all it's connections and see what happens.
Posted by sifro, 03-09-2012, 05:19 AM
It's quite a good one.. i mean, it should handle googlebot requests... Processor Name QEMU Virtual CPU version (cpu64-rhel6) Vendor ID GenuineIntel Processor Speed (MHz) 2394.306 Total Memory 3115464 kB Free Memory 247616 kB Total Swap Memory 5177336 kB Free Swap Memory 5177260 kB
Posted by AuzzHosting, 03-09-2012, 05:30 AM
You using a VPS or dedicated server?
Posted by sifro, 03-09-2012, 05:51 AM
Dedicated.. never had any problems for about 1 year, then it started behaving like this... i haven't noticed any increases in the amount of traffic.
Posted by AuzzHosting, 03-09-2012, 05:56 AM
I would contact your host or hire a server admin.