Tracking a possible security breach.

Portal Home > Knowledgebase > Articles Database > Tracking a possible security breach.

Posted by BrianLayman, 02-27-2011, 01:29 AM
I received suspended notices for my personal account and one of my main hosting partners - the association with him isn't widely spread. Obviously neither of us suspended our own accounts. The suspension was interesting because if it was done from within WHM, the email about it should have said "Account suspended by root (root)" instead it said "Account suspended by ()". Has anyone seen this before or any normal condition that can cause this? Looking in /usr/local/cpanel/logs at access_log and cp_hulk reveal no activity at that minute. error_log displayed a warning at the time the accounts were suspended becase one account didn't have any databases to delete. So I know the exact time it was done, but not how or what. As for why, this sure looks like a shot across the bow, but that notice with no-one specified as the user makes me think this wasn't done with a whm login. I've changed the root pw, shut down ssh and regenerated keys, verified the wheel group, changed wheel accounts passwords, and run rkhunter. Before I consider sending out emails to every customer on that server telling them to reset their passwords, I'd like to know a bit more about what may have happened and hear your thoughts. Does anyone have any thoughts on where should I look next?
Posted by asciiDigital, 02-27-2011, 02:28 AM
What does rkhunter report? Have you checked /var/log/messages?
Posted by BrianLayman, 02-27-2011, 03:23 AM
rkhunter -c reported no problems. rkhunter -c --enable all, did show a few warnings but nothing that leapt out as a real issue - it warned on things like the group file being modified, but I already looked through that file for nastiness. There were no actual rootkits found. It did warn on a program running on a deleted file. I've done a reboot to resolve that. Thanks for the /var/log/messages suggestion. It does show some activity during that time. Specifically the CPanel chkservd service monitor running. The suspend occurred at Feb 26 22:12:24. This is what /var/log/messages shows around that time: So still nothing yet. On a normal suspension the email header includes this: X-Source-Args: /scripts/suspendacct X-Source-Dir: :/whostmgr/docroot These suspension notifications included this: X-Source-Args: /scripts/suspendacct X-Source-Dir: :/ So it looks like the script was run from / All .bash_history files look to be legit, but could have been modified. I did a chmod 700 on the scripts directory to tighten that up. It had root as the owner/group but a +rx on everything for everyone.
Posted by asciiDigital, 02-27-2011, 04:15 AM
I don't think you're going to track this down. Run /scripts/upcp --force and /scripts/fixeverything Since this was an exploit through WHM these scripts could be useful.
Posted by Techbrace, 02-27-2011, 07:10 AM
whast does the following command show? last|head uname -r And did you notice any other changes were made on the server?
Posted by BrianLayman, 02-27-2011, 01:57 PM
Thanks for the tip on running last. I'd not known of that one. When you learn all this on your own, there can definitely be gaps... Anyway, last shows nothing during that time. My kernel version on that server is 2.6.18-028stab070.5 and I've a Sept 17 build date. AFAIK both of those should be ok. I'm starting to agree with asciiDigital. I'm seeing no tracks anywhere. On the positive side, I've reviewed all file changes in the home directory and nothing malicious is in there that I've found.
Posted by Techbrace, 03-01-2011, 11:39 PM
If you think some unauthorized tasks have been done on your server that require root privilges, you have got something to worry about. Do not trust your eyes just so easily if you didn't find anything else suspicious. Unless you know how happened, you can't leave it thinking everything is fine and you're safe!
Posted by Steven, 03-02-2011, 12:28 AM
What version of glibc are you running? rpm -qi glibc |grep Release