Finding the cause of a UDP flood

Portal Home > Knowledgebase > Articles Database > Finding the cause of a UDP flood

Posted by Chris-M, 05-21-2015, 12:20 PM
We had a UDP flood originate from one of our Linux-based Plesk servers (ie our server flooded a remote address with UDP packets). The server was rebooted in rescue mode by the datacenter and we got notified of the problem. We reinstalled the OS and all data, and have since tightened up the firewall to disallow all outgoing UDP packets other than to the local DNS servers to minimize the risk of a repeat of this problem. My question is: what would anyone recommend we check in order to try and track down the source/point-of-entry used by whoever/whatever initiated the attack? We have so far done the following: Checked for suspicious SSH logins (none found)Checked the local DNS server had open recursion disabled (was set to "Allow recursion: Localnets" in Plesk)Checked for suspicious PHP/Perl scripts being executed (none found)Checked system logs for anything obvious (nothing found) Having read what I just said in #2 above, I now wonder if the "Allow recursion: Localnets" setting for DNS would allow a malicious user on another server within our network neighborhood to use our server for a DNS reflection attack. Anyone know if that sounds feasible based on the localnets setting? We have now blocked incoming port 53 in the firewall as all our DNS is external (this was unfortunately not blocked prior to the attack). Any suggestions or ideas welcome. Thanks, Chris
Posted by DeltaAnime, 05-21-2015, 02:13 PM
Do you have a tcpdump of the flood? From there we can help you figure out what service might be getting used. Francisco
Posted by Chris-M, 05-21-2015, 06:32 PM
Unfortunately no tcpdump - the datacenter pulled the plug on the server automatically and then restarted it in rescue mode.
Posted by DeltaAnime, 05-21-2015, 06:34 PM
Sounds like what you've done is the best you can do for now. You could write a script to run through all perl/php files on your server looking for anything doing fsockopen for UDP and such. It's extremely intrusive, though, so I don't recommend at all. Francisco
Posted by Chris-M, 05-21-2015, 06:42 PM
Thanks for your input. I'll run a scan on all the files from the dump that we got of the data after the attack and see if it turns up anything. In terms of the DNS recursion idea, do you think it's possible someone else in our subnet performed a DNS reflection attack given that the recursion setting was set to 'localnets'?
Posted by DeltaAnime, 05-21-2015, 06:43 PM
Check what 'localnets' means in the ACL list, I would've thought it only meant 127.0.0.0/8. Francisco
Posted by Chris-M, 05-21-2015, 07:29 PM
I've done some research and what I can find suggests that "localnets" means all IP addresses in the same range as the server. So if the server is 1.2.3.4, allowing recursion from "localnets" would allow queries from 1.2.3.0-255. I found this information at: http://www.zytrax.com/books/dns/ch7/...atch_list.html. That would seem to confirm my suspicion that a server in the same subnet as ours could have therefore used our server to perform a DNS reflection attack. It seems that "localhost" would be the better setting for recursion to prevent anyone external to the server from abusing it in this way. Last edited by Chris-M; 05-21-2015 at 07:32 PM.
Posted by SneakySysadmin, 05-21-2015, 09:38 PM
Wow. Fail-Datacenter. Epic Fail Datacenter actually. Find a new home for your server. The place you're at is staffed by papermill certified MCSEs1 or something. Aside from DNS recursion attacks the other common culprit for UDP flooding would be ntp amplification attacks. Make sure that if you're running an NTP server only your own clients can connect to it. 1) Must Consult Someone Experienced Last edited by SneakySysadmin; 05-21-2015 at 09:41 PM.