Portal Home > Knowledgebase > Industry Announcements > Web Hosting Main Forums > Providers and Network Outages and Updates > Codero Outage
Codero Outage
| Posted by bloomkey, 03-08-2011, 10:59 AM |
|
Anyone experiencing an outage with servers hosted with Codero. I can't ping them or pull up my server. |
| Posted by Patrick, 03-08-2011, 11:24 AM |
|
Yeah, I admin a friends server there and the whole network appears to be down. |
| Posted by bloomkey, 03-08-2011, 11:56 AM |
|
Thanks Patrick. Anybody got an ETA on when they'll be back up? |
| Posted by SenseiSteve, 03-08-2011, 12:00 PM |
Quote:
|
We expect next update from upstream providers soon. Complete effort is on mitigating the massive DDoS attack. less than 20 seconds ago via TweetDeck - Upstream provider in PHX working to restore a BGP session they dropped and to mitigate a DDoS attack. Phone service is disrupted. 11 minutes ago via TweetDeck - We are experiencing network issues affecting part of our PHX data center. Engineers are working with upstream providers. Updates to follow. 41 minutes ago via TweetDeck
|
From their Twitter account |
| Posted by creative_vision, 03-08-2011, 12:15 PM |
Codero is down again. I have been a customer for 6 years now and I experience this quite a few times.
It appears their entire company is down, there websites are down, their phones are down, their servers are down. It has been over an hour now and i am waiting. And i have no way of calling them to see when they will get this back online.
This makes me VERY upset as i have 250 customers calling me about their emails and websites. |
| Posted by Jacob Wall, 03-08-2011, 12:16 PM |
|
There's already a thread about this in the outages forum... |
| Posted by Patrick, 03-08-2011, 12:23 PM |
Quote:
|
Originally Posted by creative_vision
This makes me VERY upset as i have 250 customers calling me about their emails and websites.
|
Why would you be upset, it's not like they did this on purpose? They are under a huge, huge DDOS attack and a lot of providers would fall just as easy. Unless you're willing to pay some serious $$$ for a DDOS protected network, you have to expect these issues once in a blue moon... they suck, yes, but hardly worth getting upset over. |
| Posted by Codero, 03-08-2011, 12:24 PM |
|
Our upstream provider experienced a massive DDoS attack affecting some of our PHX DC. Follow us on Twitter for updates and ETA's. |
| Posted by creative_vision, 03-08-2011, 12:26 PM |
|
Yes, i see now they are a victim of a massive ddos attack. |
| Posted by no69_2007, 03-08-2011, 01:57 PM |
well DDOS attack can hit every one, mean while i would like to tell you to be true with you clients and tell them the real prob, it would be more better.
Also try to co-operate with them to get thing solved quickly. |
| Posted by Techy, 03-08-2011, 01:59 PM |
Quote:
|
Originally Posted by no69_2007
well DDOS attack can hit every one, mean while i would like to tell you to be true with you clients and tell them the real prob, it would be more better.
Also try to co-operate with them to get thing solved quickly.
|
I agree, I think the more they know the better. Either way it's not good, but at least your honest. |
| Posted by 24x7group, 03-08-2011, 02:00 PM |
Always hate when that happens. Must be a serious attack then 
Best of luck on solving this asap! |
| Posted by Techy, 03-08-2011, 02:00 PM |
|
Yes, there are a couple of threads on WHT now about this. |
| Posted by Techy, 03-08-2011, 02:01 PM |
|
PS - here is the other thread for those of you looking for an update if you don't find it here |
| Posted by lebnene, 03-08-2011, 02:11 PM |
|
No - no ETA's so far, we admin a client's server and she has been on hold for ever this morning. |
| Posted by lebnene, 03-08-2011, 02:13 PM |
This is from their twitter:
"We have an ETA of 20-30 minutes until restoration of virtually all affected servers. New routing being put into place now." (10:12 AM PST)
http://twitter.com/CoderoNOC |
| Posted by John Nagle, 03-08-2011, 02:16 PM |
Codero is so down that their Phoenix data center's IP block isn't being routed at all. Their phone system is down, too. Codero's own main web site seems to be running off a server at Stargate right now, which may be a backup arrangement.
Codero support chat: "An online representative will be with you shortly. You are number 193 in queue. Your wait time will be approximately 801 minute(s). Thank you for waiting." |
| Posted by John Nagle, 03-08-2011, 02:37 PM |
Codero writes on their Twitter: "Network engineers are in final steps to bring back revised routing tables which will restore service to nearly all servers in PHX." Still down, though.
For a minute or so, there was a routing loop within AlterNet, between LAX15 and SJC7. |
| Posted by Spudstr, 03-08-2011, 02:39 PM |
Quote:
|
Originally Posted by John Nagle
For a minute or so, there was a routing loop within AlterNet, between LAX15 and SJC7.
|
routing loops often happen when a bgp end announcement is lost and there is a prefix stuck in a fib somewhere after a few minutes it usually goes away as it propagates through the network. |
| Posted by John Nagle, 03-08-2011, 03:09 PM |
|
Routes to Codero are starting to appear, but they're not fully in place yet. I can get as far as "BANDCON.car2.Phoenix1.Level3.net [4.53.106.78]", so at least traffic is getting to the right city now. Previously, my own ISP couldn't see any route to Codero Phoenix. |
| Posted by John Nagle, 03-08-2011, 03:19 PM |
|
Codero now announcing "Service has been restored to all but a limited number of servers.". Mine isn't one of them. What are others seeing now? |
| Posted by lebnene, 03-08-2011, 03:21 PM |
|
The server we admin is back - speed is fine. |
| Posted by Techy, 03-08-2011, 03:22 PM |
|
Have you guys seen the twitter page for codero with their updates? |
| Posted by Techy, 03-08-2011, 03:25 PM |
|
The link to the twitter page can be found in this thread about codero |
| Posted by John Nagle, 03-08-2011, 04:54 PM |
|
We're back up. Downtime was over 5 hours. |
| Posted by Codero, 03-09-2011, 02:52 PM |
Hey everyone, just wanted to give you an update that our CEO addressed yesterday's attack on our blog. High level overview of what happened and why. We will continue to update as more information is uncovered.
Codero Blog: http://www.codero.com/blog/ |
Add to Favourites 
Print this Article
Also Read
Shared Hosting
Shared Hosting
