Power Outage in 365main (San Francisco)

Portal Home > Knowledgebase > Articles Database > Power Outage in 365main (San Francisco)

Posted by mhalligan, 07-24-2007, 05:18 PM
Does anybody know what's up with 365 main? I heard that there's a power outage in SOMA, and now it appears we can't get to any of our managed customers sites in 365main.. I tried going to technorati.com to find news, but of course, technorati hosts at 365..
Posted by mhalligan, 07-24-2007, 05:31 PM
It looks like Colos 1,3 and 4 are down in 365main.
Posted by gounder, 07-24-2007, 05:33 PM
no wonder Craigslist.com is down. oh well.
Posted by mhalligan, 07-24-2007, 05:38 PM
Kind of glad that all of our production gear got migrated out of 365 in May.
Posted by mhalligan, 07-24-2007, 05:51 PM
Ah, this is a better place for this, sorry didn't realize this forum existed.
Posted by mhalligan, 07-24-2007, 06:01 PM
some information : http://laughingsquid.com/massive-pow...soma-district/ http://radar.oreilly.com/archives/20...in_datace.html
Posted by TonyAdtaq, 07-24-2007, 06:04 PM
http://valleywag.com/tech/breakdowns...out-282021.php Heh
Posted by gounder, 07-24-2007, 06:10 PM
Duh! No Backup Generators?
Posted by mhalligan, 07-24-2007, 06:14 PM
365main actually has an impressive power system. They have two power feed from pg&e feeding into 11 Hi-Tec kinetic flywheel generators.. AboveNet dumped somewhere between $120m and $140m into it before they went belly-up. It seems that for whatever reason when they lost their power today (as with a large part of their neighborhood) 3 of their generators didn't flip onto Diesil .. Probably something to do with the fact that PG&E's power was flickering for a while.
Posted by MyVPSHost-Susy, 07-24-2007, 06:48 PM
Its funny what Digg is reporting about 365Main: http://valleywag.com/tech/breakdowns...out-282021.php
Posted by mhalligan, 07-24-2007, 06:59 PM
Not Digg, but ValleyWag .. ValleyWag is awesome, the guy has essentially made a fortune talking trash about Web 2.0 entrepreneurs..
Posted by Kimmikat, 07-24-2007, 07:17 PM
Many other services like Second Life, Live Journal and others are down to as well as one of the transit sites I maintain.
Posted by feomateo, 07-24-2007, 07:22 PM
Theres no outages here at SF.... I live near that datacenter
Posted by mhalligan, 07-24-2007, 07:34 PM
feomateo: The Chronicle would tend to disagree with you.. http://sfgate.com/cgi-bin/article.cg...AG9NR67253.DTL
Posted by fastdeploy, 07-24-2007, 08:25 PM
I bet 365 main is a busy place tonight and there will be a lot of tired admins before the night is over. This kind of thing inevitably results in hardware failures. This further cements my opinion that data centers don't have "reliable" power, no matter the bold claims and huge investments most of them make. It makes you wonder why they invest so much in generators if they don't work or use them most of the time it seems--couldn't they have switched their gensets on manually while the power was bouncing for a few hours? Maybe they did do this but the tenants were still sorting out their messes. If PG&E itself loses power other businesses don't have a chance. Another reason why geographically dispersed facilities is a good idea.
Posted by rubycloud, 07-24-2007, 08:35 PM
Latest theory: the f*ing EPO switch again... http://valleywag.com/tech/breakdowns...rld-282072.php
Posted by TeleSouthNet, 07-25-2007, 12:37 AM
Ummm I would hope that they 86ed the drunk employee. All I can say is that we considered that building and opted for Fremont. No problems yet in two years, crossing fingers.
Posted by LaserSpec, 07-25-2007, 02:49 AM
The employee thing is just a rumor at this point, I hope no one believes it
Posted by mhalligan, 07-25-2007, 02:58 AM
The real reasoning behind it is probably a coupling of the 6 power outages at the Mission St substation combined with 365's resources being stretched too thin due to their hyperactive purchase of 6 buildings in the last 2 years.
Posted by rubycloud, 07-25-2007, 11:45 AM
More info about the outage yesterday: http://www.datacenterknowledge.com/a...in_outage.html