Reminder, Scheduled maintened Dednow / Pweb 12AM EST 7/11/04

Portal Home > Knowledgebase > Articles Database > Reminder, Scheduled maintened Dednow / Pweb 12AM EST 7/11/04

Posted by PhMatt, 07-10-2004, 10:17 PM
Reminder At 12:00AM EST Pegasus Web Technologies and DedicatedNOW, will be having our scheduled maintenance window, to complete our server moves to our new facility in Clifton, NJ. Please review the following announcement located at: http://clifton.fortressitx.com/index.php?tpl=statement2 We will be updating our temporary forums http://clifton.fortressitx.com/forums/ during this time once we arrive to the new facility, and keep people up to date on how things are progressing. During the downtime, phone support and technical support will not be available until after the servers have all been brought back online, as our administrative team's time will be most valued in getting our clients back online. We will only be placing updates as they occur. We appreciate everyone's time and patience during this transition, and look forward to better servicing you in the immediate future. Sincerely,
Posted by Lorenz, 07-10-2004, 11:54 PM
Move seems to have started. Servers are offline. Best of luck and please move my site first ;-) *praying that all goes well*
Posted by freakysid, 07-11-2004, 12:09 AM
fingers crossed - I was just shutting down my servers and they beat me to the last one! I wish they had a live webcam in Clifton.
Posted by Lorenz, 07-11-2004, 12:11 AM
Yes, a live webcam would be a nice thing - also within the mover-trucks, so we can see how they throw the servers around and how they fall from the truck ;-)
Posted by ImLagging, 07-11-2004, 12:14 AM
Please don't say that. You might jinx us all...
Posted by Lorenz, 07-11-2004, 12:18 AM

Posted by ImLagging, 07-11-2004, 12:23 AM
Bad Lorenz! Bad! Bad! No cookie for you!
Posted by Lorenz, 07-11-2004, 12:26 AM
Serious, this banana made my day LOL
Posted by ImLagging, 07-11-2004, 12:35 AM
With any luck, all of my customers (that are on DN servers) are asleep right now. And yes, I can't help but chuckle every time I see that banana.
Posted by GotGameServers, 07-11-2004, 12:57 AM
I got people emailing me and pm'n me already asking why the servers down. I have 2 servers there. Webserver and another one and both run CS/CZ game servers on them as well. My clients do notice when stuff goes down and i hear about it almost immediately. I guess this means new IP's as well. Meaning dns/ip propagation wait time as well. /me looks forward to my credit for this months downtime... Last edited by XTStrike; 07-11-2004 at 07:00 AM.
Posted by NexDog, 07-11-2004, 12:57 AM
With 20 servers - no chance.
Posted by ImLagging, 07-11-2004, 01:01 AM
I'm not that big yet. Maybe one day though. *knocks on wood*
Posted by Fillos, 07-11-2004, 01:03 AM
I have also recieved no notice of this event I will be up with you through the entire migration I will be responding to my clients support tickets and attempting to placate them to maintain my client base. I am at a loss for words, your stupidity astounds me.
Posted by Paint, 07-11-2004, 01:10 AM
Well our three servers are down. Good luck to Jason over there to get our servers back up safely. All will go well... Ive talked to him alot about the move. It is for the better.
Posted by Shaw Networks, 07-11-2004, 01:22 AM
They could make a good "Lethal Weapon" movie out of this server move Imagine what a semi truck full of servers exploding would look like! Jk, I hope the move goes well for all you guys and they miss all the pot holes along the way.
Posted by X-Max, 07-11-2004, 01:29 AM
they sent email to usual billing email addresses. Don't you receive your payment receipts on regular bases ? If not then i think its your job to keep your email address up-to-date with them.
Posted by SocomNET, 07-11-2004, 01:58 AM
I was never notified of this. Last edited by XTStrike; 07-11-2004 at 07:01 AM.
Posted by freakysid, 07-11-2004, 02:08 AM
You should check into these forums more often. The iminent move has been discussed for weeks.
Posted by twrs, 07-11-2004, 02:16 AM
I guess the move is being performed now. Is there any way to check the progress online? I checked http://clifton.fortressitx.com/forums but found nothing there.
Posted by DaveNZ, 07-11-2004, 02:19 AM
I do understand that there are circumstances beyons the control of Pegasus and that this isn't the way they would have wanted to do it. I should also say that their service has been very good in the past and the connection has been much improved over recent months. However I can't let this go without comment. I have received no notification that I can find - the first I knew was when the server went down. At this stage I'm not jumping to conclusions but it seems very strange that I have no email. Even assuming that our notification email got "lost", 4 days notice of a move this significant is utterly inadequate. Every few minutes I think of something else which will be screwed by this downtime. At the very least this entire month's profit has just gone down the drain. Yes, I'm angry and I'm venting. I'm well backed up and I don't see any long-term problems, but my partner is on the phone at the moment apologising to our live chat celebrity guest and I'm looking at a long list of problems resulting from this. Despite my current mood I do wish the team all the best and hope the new centre works out well. Be warned though - I'm going to be one very unhappy camper if my IP addresses have changed without warning. Very,very unhappy I will be.
Posted by PhMatt, 07-11-2004, 02:20 AM
just posted in our forums: Ok guys, I'm here, network equipments about completed (items we removed from NAC) we got here about an hour and 15 minutes ago, but I needed to setup my desk here before I could reply (connect PC). First wave of trucks should be here in about 5-10 minutes. Crews are working as quickly as possible to take down servers, box them up, put on trucks, transport, and shortly bring servers back up. That's all I have for now, I cannot respond to individual complaints / questions at this time, only provide updates as they are available. Thanks,
Posted by PhMatt, 07-11-2004, 02:23 AM
All notifications were made via the same method we use for our billing invoices, generated to every client at the same time, most received multiple notices. We also addressed the issue with our forums, updated people of the forums: http://clifton.fortressitx.com and posted as an announcement in our regular forums, that we were performing this move. Currently, we're underway, and the first wave of servers to be arriving NOW WOOHOO ! ! ! More posted in our forums / network outages (at WHT) when applicable. Thanks,
Posted by Tom Pyles, 07-11-2004, 02:26 AM
Matthew, Should we stay tuned here for updates. I was under the impression that a back-up site would be in place where we could get updates...I can't seem to find a url for that back-up. Otherwise, good luck with the move. I've been anxiously awaiting the start of new server sales and it looks like it will finally happen.
Posted by Knogle, 07-11-2004, 02:28 AM
Best of luck with the move, Matthew.
Posted by M7I, 07-11-2004, 02:31 AM
Tom Pyles, http://clifton.fortressitx.com/forums is the url for the updates.
Posted by Tom Pyles, 07-11-2004, 02:34 AM
Rob, Thanks for the link!
Posted by DaveNZ, 07-11-2004, 02:53 AM
Okay, I've taken my chill pill and I'm waiting to see what happens. I've got too many problems at the moment to figure out why I didn't get any notification. That can happen later. Just wanted to say thanks for the updates - keep them coming! I like to be fair and I can say you guys are good at providing updates in times like this. It also seems that our IP addresses will be safe for now. I hope I've read that right because that was one of my main concerns.
Posted by Paint, 07-11-2004, 02:54 AM
How is the time schedule going? Everything on time so far Matt?
Posted by SaLanHost, 07-11-2004, 03:02 AM
Is datacenter moving ?
Posted by PhMatt, 07-11-2004, 03:16 AM
Latest update Approximately 4-5 racks are about ready to go from DedicatedNOW, and a few from Pweb. Our Network is up, and Anthony's working on announcing the IP's for BGP routing. 2nd truck should be here shortly, as soon as we're done connecting the switches for the breadracks, and rackmounts, we'll power up, and begin the first round of servers. For those wondering, the FIRST racked customer server put up tonight, was Beachcomber's by chance, but it's still not on yet. Thanks,
Posted by Layershift Andrew, 07-11-2004, 03:18 AM
How many racks are there altogether?
Posted by PhMatt, 07-11-2004, 03:18 AM
Latest update Approximately 4-5 racks are about ready to go from DedicatedNOW, and a few from Pweb. Our Network is up, and Anthony's working on announcing the IP's for BGP routing. 2nd truck should be here shortly, as soon as we're done connecting the switches for the breadracks, and rackmounts, we'll power up, and begin the first round of servers. For those wondering, the FIRST racked customer server put up tonight, was Beachcomber's by chance, but it's still not on yet. Thanks,
Posted by Aussie Bob, 07-11-2004, 03:24 AM
Maybe a hundred, or a bit less or a bit more.
Posted by RossH, 07-11-2004, 03:24 AM
hmmm....beachcomber's by chance.....funny chance since they are your largest reseller with what 400+ servers....
Posted by Aussie Bob, 07-11-2004, 03:27 AM
Thanks Matthew. Keep that information flowing now.
Posted by NexDog, 07-11-2004, 03:30 AM
Sounds like we're behind schedule....
Posted by Layershift Andrew, 07-11-2004, 03:32 AM
Doesn't that increase the chance of it being them?
Posted by RossH, 07-11-2004, 03:34 AM
Exactly my point.....read Dednow's last post...
Posted by Layershift Andrew, 07-11-2004, 03:35 AM
Ah sorry ... thought you were complaining that it was them.
Posted by RossH, 07-11-2004, 03:39 AM
No...I'm just saying it was not by chance that beachcomber went first like DedNow claims...
Posted by darkpunk99, 07-11-2004, 03:41 AM
no, it's still a chance. it's just much more likely that out of the 3000+ servers, a reseller with 400+ will happen to be the first. and he didn't say that all of the rack was beachcomber, just the very first server. beachcomber's main page is still down.
Posted by Aussie Bob, 07-11-2004, 03:43 AM
beachcomber would have servers spread all throughout pweb's/dednow's previous racks. Beachcomber's servers are not all in 1 place. I doubt seriously that they have 400+ servers with pweb/dednow.
Posted by Layershift Andrew, 07-11-2004, 03:44 AM
I think pwebtech.com/dedicatednow.com will be up first, or at the same time.
Posted by RossH, 07-11-2004, 03:48 AM
Well thats what Matt said....
Posted by Aussie Bob, 07-11-2004, 03:50 AM
Where? Anyways, i can't see them going through all their racks and selectively removing beachcomber's servers and setting them up first.
Posted by PierreB, 07-11-2004, 03:53 AM
Well with 40 movers, each movers get 10 servers, easy heh. But for real, I tihnk they move everyone at the same time no priviliges. =]
Posted by Knogle, 07-11-2004, 03:55 AM
Even if they did give BeachComber preferential treatment, i wouldn't be surprised. It's about protecting THEIR business and their clients. As much as they want to treat every client equally, they will prioritise and serve their larger clients who bring in more revenue first. That's the name of the game. You would do that too if you're motivated by profit.
Posted by RossH, 07-11-2004, 03:58 AM
You guys are acting like I think it's a bad thing.....I just thought it was funny that Matt said it was by chance that by chance beachcombers server went first.....
Posted by PhMatt, 07-11-2004, 03:59 AM
Not that badly, consoling down the servers, and packing them, then transporting is a slower process than powering up. Tech's are consoling boxes now, wiring and powering up. Shouldn't be too long before a good majority are up. Thanks,
Posted by Layershift Andrew, 07-11-2004, 03:59 AM
I really can't see beachcomber getting any better service here, it would delay things by a great deal unless they have dedicated racks.
Posted by Knogle, 07-11-2004, 04:01 AM
Good to hear that things are going well, Matthew. Hope to see your new site online shortly.
Posted by PhMatt, 07-11-2004, 04:02 AM
Geez guys, I was just trying to lighten the mood up. Servers were unpacked, we grabbed them, I racked it, looked at the hostname, and knew it was a beachcomber server, nothing more than that. It was a coincidence, nobody's getting any special treatment atm, not even me Thanks,
Posted by PierreB, 07-11-2004, 04:02 AM
Why is everyone anxious about benchcomber's servers? I mean, I'll be worry baout my own servers instead of someone elses. I'm pretty sure the people at Dn/pweb are very busy to bring everyone back online asap. So why not wait and see what happens.
Posted by Aussie Bob, 07-11-2004, 04:11 AM
Keep up the good work! pssst, slips Matt $500 under the table . . .
Posted by AP2k2, 07-11-2004, 04:15 AM
is there any estimate time of when this will be done?
Posted by kimrari, 07-11-2004, 04:16 AM
Keep up the good work. This has to be a massive undertaking.
Posted by JenniH, 07-11-2004, 04:17 AM
We seem to be about 4 hours into the window.... I think (NY = GMT - 5?). So sometime in the next 2 hours (or so) I would guess.... unless there's an update?
Posted by Knogle, 07-11-2004, 04:22 AM
Stay tuned to this thread: http://clifton.fortressitx.com/forums/viewtopic.php?t=2 That's the best you have for now.
Posted by AP2k2, 07-11-2004, 04:51 AM
just wonder if they move like that, will the ip assignment will change?
Posted by RossH, 07-11-2004, 04:56 AM
Yes keep up the good work and good luck....
Posted by JenniH, 07-11-2004, 05:30 AM
I reckon we are 5.5 hours into this, and not a flicker of life anywhere. We are definitely in for an over-run of the projected maximum 6 hours... but how long? The Fortress ITX forum is pretty dead as far as concrete info is concerned. Does anyone have anything that gives us some hope? Any flickers of life on any servers?
Posted by elevation, 07-11-2004, 05:31 AM
Matt, do you have an update? Thanks.
Posted by Aussie Bob, 07-11-2004, 05:35 AM
Come on Pweb, keep the information flowing please!
Posted by Knogle, 07-11-2004, 05:38 AM
Matt doesn't seem to be on the boards, and there's no admin online over at http://clifton.fortressitx.com/forums either. One can't help but feel lost without information, especially when the timeframw draws to an end. Common FORTRESS ITX, keep the information flowing as Rob said.
Posted by JenniH, 07-11-2004, 05:39 AM
Yes... the silence as we approach the 6 hour mark is indeed another cause for concern. I'd hoped for at least 30 minute updates, and at least SOME servers back online by now. Unfortunately, as time goes by without information, it looks more and more likely that there is a serious issue.... I hope I'm wrong. By the way, does anyone else have any sort of contingency plans in place in case this ends up in disaster? We bought hosting on a new server elsewhere and have uploaded our main sites just in case. At 8 hours, unless return looks imminent, we start flicking the nameserver switches. Just call me Ms Pessimist! Last edited by JenniH; 07-11-2004 at 05:47 AM.
Posted by GreatDaneOwner, 07-11-2004, 05:50 AM
And since then we've passed the 6-hour mark...
Posted by NexDog, 07-11-2004, 05:51 AM
Hmmm, getting nervous here...
Posted by Alan - Vox, 07-11-2004, 05:52 AM
Ive got at least 2 of my servers back online.
Posted by JenniH, 07-11-2004, 05:52 AM
It's not good... the silence that is. Not impressive at all. >> Ive got at least 2 of my servers back online << We have three hosts there, and no servers up at all at any of them.
Posted by NexDog, 07-11-2004, 05:55 AM
Good to hear! At least we know the DC works.
Posted by Samuel, 07-11-2004, 05:56 AM
Maybe they are all picthing in =) I personally couldn't sit on a board while work had to be done =)
Posted by NexDog, 07-11-2004, 05:58 AM
We have movement.
Posted by JenniH, 07-11-2004, 06:08 AM
>> I personally couldn't sit on a board while work had to be done << It takes 30 seconds to post a message, and alleviates considerable and widespread concern. Commmunication is essential, but is lacking in this case.
Posted by Samuel, 07-11-2004, 06:13 AM
That 30 seconds ignores that you have to then get back to what you're doing which could equal 5 to 10 minutes away from doing a lot. This must be stresfful for all involved which means (If It was me) i would take the oppourtunity to get a "Break" in =) So I think focusing on the lack of info when really they've been down just a little bit, and is scheduled is how the IT goes.
Posted by Samuel, 07-11-2004, 06:16 AM
Edit, wrong thread hehe, sorry Last edited by Samuel; 07-11-2004 at 06:23 AM.
Posted by Aussie Bob, 07-11-2004, 06:26 AM
Most importantly (too) is communicating with clients. I would have thought they would have had 1 or 2 dedicated staffers that would act as an information conduit, for such a massive undertaking. Sadly, most hosting suppliers do not fully grasp the need to swiftly communicate.
Posted by reggie, 07-11-2004, 06:27 AM
Hi Any word on how the move is going, it's been over 6 hours now, and I have not seen a post in here from anyone who has a server back online yet. Mine are not back yet either. cheers Reggie
Posted by JenniH, 07-11-2004, 06:28 AM
So why do they have a board over there, Samuel, if they aren't going to use it to provide proper info? Sorry, but you're just making excuses, and there are none. If this was so well planned, I would have expected communication to be included. It is, and has been, very very poor. Our 8 hour threshold is approaching. Sorry, but this is far from impressive from our perspective.
Posted by Samuel, 07-11-2004, 06:28 AM
Bob, with this massive of a move I think they are well within a reasonable time frame. If you dont hear from them within 3 or 4 hours from their deadline with this big of a move I would be concerned. Excuses? I'm being reasonable. 8 hour threshhold, what is that? 2 hours past the deadline? Not a big deal, 4 hours and no comment? I would be pissed =)
Posted by tipster, 07-11-2004, 06:30 AM
They are behind schedule. First rack online, second coming up. All this info is in the temp forum.
Posted by Aussie Bob, 07-11-2004, 06:32 AM
Oh yeah, I thought the 6 hour window was a tad optimistic. However, it would be nice for Matt to jump in and give some info every 30 minutes at least. Information vaccums breed fear, panic and anger. We all learned that from the McMaster of such communications incompetance.
Posted by NexDog, 07-11-2004, 06:33 AM
In all fairness, it's 6 hours and 15 minutes exactly since our servers went down.
Posted by JenniH, 07-11-2004, 06:33 AM
More excuses Samuel? Sorry, but if they say 4-6 hours, that is what I expect and is broadly what I plan for. If they over-run, and fail to provide reasonable (any!) information, then I have to assume the worst and act accordingly. That's sensible business practise.
Posted by Samuel, 07-11-2004, 06:35 AM
Jenni, seriously, why do you think I have any reason to have excuses for them? I understand its a stressful time but saying I'm offering excuses for them when I've already stated what I am feeling about it is only hearing yourself. Sorry you're having trouble with your provider, I really do think they are doing a great job. I'm actually watching the result of this for future business =) Bob, yep but they were at the mercy of their provider, and well they were understaffed.
Posted by JenniH, 07-11-2004, 06:38 AM
Samuel... it's just that you just seem very 'understanding' for a firm that has over-run it's timescale, has countless hosts in limbo, and yet is simply NOT communicating with anyone. That is BAD, and there is not getting away from it. 8 hours is just the first phase of our contingency plan NexDog. If we get there, without any word, and are still down, we start to shift. You can't just sit on your hands indefinitely and hope it will turn out fine. Sometimes it doesn't.
Posted by net-trend, 07-11-2004, 06:39 AM
I think it's poorly planned. They shouldn't have given a timeframe of within 4-6 hours for a move like this. Instead they should specify 12 hours at least so none of us will have to worry as yet. I guess they were too optimistic about the move to really take into account what could happen. Luckily the rest of our servers are in The Planet.
Posted by Samuel, 07-11-2004, 06:40 AM
Good luck jenni, I am sure it will work out.
Posted by Samuel, 07-11-2004, 06:42 AM
Maybe they will take the time to comment on all of this. From everyhting I have read lately it looks as though things have been rocky lately unrelated to performance issues, but legal issues which will strain humans quite a bit. I think a bit of patience is warranted as all of you have been with this provider for some time, if they were always bad I think you would have already had a contingency set up. That you didn't is squarlely your responsibility.
Posted by stefaans, 07-11-2004, 06:43 AM
Things are starting to happen with a couple of servers online since about 10 minutes ago. Our first server on BR02 is back online as well
Posted by NexDog, 07-11-2004, 06:48 AM
Jenn, servers are coming back online so hang in there. We have 1 or 2 cPanel servers online but our main rack is yet to materialise. I would say 8 hours is a fair estimate. That would make 90 mins from now so if the first racks are coming online now, the rest should hopefully be done by then.
Posted by GreatDaneOwner, 07-11-2004, 06:48 AM
It would be warranted if I hadn't had way too many other issues with them recently. It was a mistake not to leave earlier. The downtime for me is over 7 hours now.
Posted by Layershift Andrew, 07-11-2004, 06:50 AM
They've not accepted new sales for a long time ... since the sales halt I don't think I've had any major issue with them, and nothing at all in recent months. They've proven to be an excellent provider, and I can still support them on this regardless of them exceeding the maintenance window for this.
Posted by JenniH, 07-11-2004, 06:50 AM
The thing I expect above anything else Samual is honesty. If they thought it might take 12 hours, they should have said so. We could then have planned accordingly (we'd have probably shifted a couple of high profile sites around). To state 6 hours lays a marker down. We planned around that. To over-run, and then SILENCE... well.. don't expect patience. Patience and understanding may come if the provider actually bothers to communicate. These guys aren't. We have no idea what is going on there.... or what sort of outage we could be looking at. That's probably the worst scenario for any serious business. Hence, we set our own contingency markers rather than let it drift into a potential disaster. NexDog: Thanks for that update. Could you keep us posted? Appreciated.
Posted by Samuel, 07-11-2004, 06:54 AM
Equating a mistake with a lack of honesty I think is inaccurate. But I respect your opinion none the less jenni.
Posted by Knogle, 07-11-2004, 06:56 AM
Seems like DNOW's forums are responding to pings now.
Posted by Knogle, 07-11-2004, 07:00 AM
Update: They've stopped responding again.
Posted by Rochen, 07-11-2004, 07:06 AM
Yes, we are starting to see servers come back now too. We have had staff available since the start who have been keeping our customers updated throughout the move. There's not much we can do apart from keep the communication channels open and the information flowing.
Posted by Knogle, 07-11-2004, 07:13 AM
BeachComber.net is back online -- good news!
Posted by The3bl, 07-11-2004, 07:18 AM
About half our servers are online. We sit we wait and wonder.
Posted by MrMcGoo, 07-11-2004, 07:54 AM
I think so....
Posted by DNCOO, 07-11-2004, 08:17 AM
Hi all, I wanted to quickly give an update and get back to work. Unfortunately the move is going a bit slower than originally planned. We have approx. 30+ movers to assist, but due to some restrictions placed upon us, only 8 of our employees, (No Hired Movers) were allowed in the colo to remove servers at any given time. This severly slowed down our moving abilities, but we're coming along. We appreciate the patience and understanding, we'll have all of you guys back up and running as soon as we can! Thanks for hanging in there!
Posted by Matt, 07-11-2004, 08:18 AM
More threads merged....
Posted by JenniH, 07-11-2004, 08:23 AM
And an ETA (an honest one)? We could summise that 30/8 is approx 25%. If, therefore, 6 hours is 25% of the total outage for some servers, we are looking at 24 hours! It could be nonesense, but I'm not taking the risk. I'll start re-pointing some of our most critical sites, which have been down far too long already. >> but due to some restrictions placed upon us, only 8 of our employees, (No Hired Movers) were allowed in the colo to remove servers at any given time << A Question: You must have known that within minutes of the start, and must therefore have been aware that there would be significant delay. Why didn't you tell us 8 hours ago? Last edited by JenniH; 07-11-2004 at 08:29 AM.
Posted by newyorker, 07-11-2004, 08:31 AM
still waiting and it really is frustrating.
Posted by DaveNZ, 07-11-2004, 08:33 AM
Yep, a time frame would be good. What do I tell clients? What's the worst case scenario?
Posted by Burnspot, 07-11-2004, 08:34 AM
No offense, but supposing the restrictions had to do with security issues regarding the movers, a year of planning should have anticipated this potential problem since it is a secure complex. Just guessing...could be way off base.
Posted by DNCOO, 07-11-2004, 08:37 AM
An honest ETA would be a few more hours for proper setup and testing. I really can't give a range, because we have more help arriving every minute, so we are looking to speed things up. This will be my last post for a while as I'm heading back to assist. Thanks,
Posted by GreatDaneOwner, 07-11-2004, 08:37 AM
And you found out only today?
Posted by Knogle, 07-11-2004, 08:41 AM
It's evident that they didn't put in enough effort in planning for this move. For example, they didn't find out about the restrictions on moving, and they didn't properly work out communication procedures. As a result, an inaccurate timeframe was announced, leading to inconvenience for their customers. <<< Edited by request >>> Last edited by SoftWareRevue; 07-11-2004 at 09:04 AM.
Posted by nate, 07-11-2004, 08:42 AM
That is an excellent point! And the 4 days notice was also a plus ! I have 1 machine currently online at this time. All I have to say is: 1) Poor notice as 4 days is hardly enough to prepare for what is going on 9 hours of downtime! 2) Poor communication, as stated by JenniH ... this could have been revealed earlier but comes out only after we are all sitting here wondering what is going on! 3) Poorly coordinated move, as they could have found out the specifics of only having 8 people in the DC prior to 5 minutes before the move One word seems to ring clear: POOR !
Posted by newyorker, 07-11-2004, 08:45 AM
is the dms up? what was the url?
Posted by ViAdcK, 07-11-2004, 08:46 AM
dms.dedicatednow.com
Posted by Knogle, 07-11-2004, 08:47 AM
Nate sums things up very nicely.
Posted by DaveNZ, 07-11-2004, 08:49 AM
Ed Barna (VP) posted at clifton.fortressitx.com - looks like a few more hours. Edit: Damn, sorry bout that post. getting tired.
Posted by ViAdcK, 07-11-2004, 08:51 AM
That's a post from another user quoting what he posted at wht about 15 min ago.
Posted by nate, 07-11-2004, 08:52 AM
Knogle: Funny, I was about to say you summed things up nicely yourself I could not agree more about: "As a result, an inaccurate timeframe was announced, leading to inconvenience for their customers." Just totally POOR .... and where is the supposed customer benefit again ? I think that is so funny how they make it seem like this was for our own good and we are the ones sitting here holding the bag and were completely misinformed !
Posted by Burnspot, 07-11-2004, 08:52 AM
The entire thing is certainly something that doesn't inspire confidence.
Posted by newyorker, 07-11-2004, 08:55 AM
we need an update.....! this is not professional at all.
Posted by nate, 07-11-2004, 08:57 AM
I do not think you are going to get one, as this was stated above: "An honest ETA would be a few more hours for proper setup and testing. I really can't give a range, because we have more help arriving every minute, so we are looking to speed things up. This will be my last post for a while as I'm heading back to assist." .... counting the minutes go buy ....
Posted by dredy, 07-11-2004, 08:58 AM
i can not login into DMS. My login information is correct but i can not login.. anyone has an idea?
Posted by Lorenz, 07-11-2004, 09:06 AM
Just woke up and see what I "promised" my provider (Ultraunix) did happen. People ALWAYS calculate the time that is needed wrong, that is in the human nature So far, who has it servers back online yet? Anyone? I am in the 216.67.227.* block, anyone up there?
Posted by newyorker, 07-11-2004, 09:06 AM
has been 9 hours.
Posted by X-Max, 07-11-2004, 09:11 AM
chill out people, they must be more worried then us because they don't want to listed #@&%^(R& from clients if some server broke off btw, my few servers are back and few are still offline. cheerssss !!!
Posted by JenniH, 07-11-2004, 09:17 AM
>> chill out people << Sorry... but we are running businesses here... businesses that are being damaged by the minute through bad planning. I'll 'chill out' when are sites are back online.
Posted by Knogle, 07-11-2004, 09:24 AM
I doubt it would be 24 hours, because although only 8 employees are allowed in, the 30 movers can do the transportation. Of course, setting up the servers again would be a problem since a substantial number would be at NAC packing up the servers. An educated guess would be 12-15 hours for everything to resume normal operation.
Posted by minotauro, 07-11-2004, 09:27 AM
Anybody access this IPs? 66.246.110.157 216.67.239.27 69.57.173.226 69.57.187.2 69.57.175.146 The servers have more 10 hours offline...
Posted by Lorenz, 07-11-2004, 09:31 AM
Í am in the 216.67.227.* and are also still offline. I wonder who is back online yet?!
Posted by legend1, 07-11-2004, 09:33 AM
Just an FYI. I am on BR09 and I just now this minute came back online. Stable? Only time will tell but it's back
Posted by PhMatt, 07-11-2004, 09:34 AM
Unfortunately, yes, when we tried to gain access to the facility tonight, there was a large hinderance placed upon us with only allowing 8 staff members into the colo facilities at once. This was added by a door check for identification of all equipment and logging on the way out, camera crews constantly following us around, all led to a required revamp of our coordinated efforts, with movers, technical teams, to help take down servers, etc. At that point then, employees slated to go to Clifton were needed to stay behind, to help bring servers down, transport outside of the facilities where then the movers could pack and begin transportation of services. All equipment is now onsite, and all of our staffed personnel, and others are here racking, running cables, bringing up switches, servers, routing, etc. We did fall behind, and were needed most to help get customers back up and running as quickly as possible. Time is essential now for us to get everything up for our clients. Staff members have been working 24+ hours, but everybody is going strong to get things back online for everyone. We had no idea until literally the moment we pulled the network, and began to go into the colo that the restrictions would be placed on us, and ultimately cause a massive delay in our planning. We do apologize for the inconvenience caused, and are working our behinds off to get things in order as soon as we possibly can. Sincerely,
Posted by NexDog, 07-11-2004, 09:36 AM
As long as we know what's happening, Matt, we will get through it. Thanks for the update.
Posted by nate, 07-11-2004, 09:44 AM
Matt, Thanks for the update. It is good to get more information on this situation and a clear understanding of what all was going on.
Posted by Lorenz, 07-11-2004, 09:45 AM
The last caper from NAC! I really feel with you guys after reading the complete court filing and having seen how they ripped you off over the last years. I hope you started a fire in the room when closing the doors at NAC and punched the owners of NAC ;-)
Posted by X-Max, 07-11-2004, 09:48 AM
thanks matt, keep up the good work.
Posted by eclipsewebs, 07-11-2004, 09:51 AM
We are in AG02, any one else there or heard anything?
Posted by Knogle, 07-11-2004, 09:54 AM
Lorenz, you can't blame NAC for following their SOP. They are liable to their customers. They need to mantain security within the facility and it's fair for them to impose restrictions on access to the building if it means improved security. If one of the 30 movers accidentally damaged a server or two of another of NAC's clients, it's NAC who has to answer to the client. If anything, DNOW/PWEBTECH is to be blamed for not investigating before going in IMO.
Posted by Lorenz, 07-11-2004, 09:57 AM
Knogle, did you read the complete court-filing? I think you´ll be surprised how hard NAC made the last years for DedicatedNow (NOT talking abou the IP stuff here).
Posted by Knogle, 07-11-2004, 10:02 AM
Yes, I did. I'm not going to comment on that case because it's an unrelated issue. NAC has it's own clients to protect and it did just that. Unless anyone can prove that NAC intentionally limited access to the facility for DNOW/PWEBTECH solely to hinder their progress, i'm afraid DNOW has to take the brunt of the blame. Did they do their research? Probably not. If they did, NAC would have informed them well in advance about this restriction.
Posted by newyorker, 07-11-2004, 10:07 AM
10+ hours. still down.
Posted by JohnCrowley, 07-11-2004, 10:10 AM
[IMNSHO] That is what Matt meant by "last minute", i.e. NAC DID NOT inform them of these more restrictive security measures until they got there. This is not Standard Operating Procedure. This is a "cry-baby" CEO at NAC trying to get one last dig in on Ded. Now due to jealously and anger over the court case. Nothing more. Hope that vocalizes what Matt and company are feeling/thinking but cannot post for legal reasons. [/IMNSHO] (Note: I have no inside knowledge of actual events, but can extrapolate quite well based on past data.) - John C.
Posted by Knogle, 07-11-2004, 10:11 AM
We'll wait for the post-mortem, if any.
Posted by newyorker, 07-11-2004, 10:18 AM
I think it is finally time to start looking for other dedicated server providers.
Posted by coight, 07-11-2004, 10:25 AM
We are back up after 9 hours
Posted by forumtalk, 07-11-2004, 10:28 AM
still down Cleints ask me " where is 4-6 hours ? "
Posted by Knogle, 07-11-2004, 10:33 AM
Apparently new updates are being posted here: https://forums.dedicatednow.com/showthread.php?t=2657 Can someone echo the updates over to WHT? I don't have an account there.
Posted by coight, 07-11-2004, 10:34 AM
The thread is closed with no new updates
Posted by MrMcGoo, 07-11-2004, 10:37 AM
In a situation like this there will always be those that look for reasons to crtiticize (glass 1/2 empty) and those who judge the people they have chosen to do business with by their history and their actions (glass 1/2 full). Me, I am an optimist. Will I lose customers, quite certaintly....and most are the type of customers who would leave anyway for one reason or another...always looking for that perfect place...always bitterly disappointed... I have been with DedNow since they opened their doors and have never made a better decision. While there have been bumps in the road, no company I have every worked with has shown me greater courtsey or proffesionalism from the top down. So would I have liked this to be all wrapped up in 6 hours ...You bet. But if we take a more long term look (I like to think that is why we are fairly decent sized...a longer view) the uptime over the past year is still excellent...even with this scheduled outage. So why is my glass 1/2 full...Because EVRYONE here bust their behinds everyday to provide the best service we can...and that goodwill allows us to weather these storms. So our servers will come back up, our customers will have there sites back and our business will grow... If you do not believe that Matt, Jay, Ed and everyone else did all they could to make this go smooth...well, until someone can explain to me the reason (value) for that path of action.... I kinda think I am stickin with being a 1/2 fuller type of guy. Back to answering customer e-mails (and getting coffee...must have coffee)....busy busy busy
Posted by nate, 07-11-2004, 10:38 AM
[edited: nothing left worth saying]
Posted by Aussie Bob, 07-11-2004, 10:39 AM
Yes, sure we'll all take a hit, but tomorrow's a new day and life will go on.
Posted by newyorker, 07-11-2004, 10:43 AM
come on!!!!!!!!!! it's almost 11 hours now.
Posted by Nickgold, 07-11-2004, 10:44 AM
BR10D was back up about ten mins ago
Posted by newyorker, 07-11-2004, 10:47 AM
can't login to dms for some reason. i have 4 digit customer id, but it says invalid user id. i don't know what rack i am on. is there a way to find out before logging into the dms.?
Posted by freakysid, 07-11-2004, 10:52 AM
Anyone in the A series of racks up yet? Specifically AI01 and AM04? Thanks! Chins up
Posted by DreamPages, 07-11-2004, 10:55 AM
They just told me as of 9:45am CST Sunday there is no ETA. Probably a few more hours.
Posted by NexDog, 07-11-2004, 10:56 AM
Our AI03 rack came back up 30 minutes ago. Still waiting on 5-6 servers.
Posted by WebHostingNeeds, 07-11-2004, 10:56 AM
Thanks, my sites on BR10F-E is up an running.
Posted by Daytona Host, 07-11-2004, 10:59 AM
Very well stated mark!... Couldn't have said it any better. We're all in the same boat, drinking the same coffee.... Think of this as move any business. If you relocated your office, the moving truck could be delayed, things can get broken, etc. I think most of us our fustrated because we are looking at screens that say: Request timed out. Request timed out. Request timed out. Request timed out. Request timed out. Request timed out. Request timed out. Request timed out. Request timed out. Request timed out. Request timed out. Request timed out. Request timed out. Request timed out. Request timed out. Some of us are better when we're hands-on (involved). In this case, we are simply helpless. All we can do is drink coffee, calm customers, drink more coffee, and PING. (LOL!) oh well, to everyone that is back up - I am very happy for you. I'm glad all went well. To those (like us) that are waiting to see their first server come online - hang in there. We are in good hands. We too have thousands of sites down, but so far our customers are very understanding. They all knew this day would come. Everyone have a good day. I look forward to some sleep after this one!.. :-) Tim
Posted by NexDog, 07-11-2004, 11:00 AM
2 servers in that rack aren't responding too. Worrying....
Posted by snickn, 07-11-2004, 11:01 AM
We've seen 4 of our 15+ servers there come back up, unfortunately they are all spread out.
Posted by JenniH, 07-11-2004, 11:07 AM
Our sites with you are all up now Laurence. No joy for our other Pegasus hosts yet though....
Posted by coight, 07-11-2004, 11:13 AM
Good luck guys!
Posted by inteltechs, 07-11-2004, 11:16 AM
what a day!!! I am getting tired of this whole thing...hopefully, things will get back to normal soon.
Posted by myusername, 07-11-2004, 11:21 AM
yes, very soon, pushing 9.5 hours now and not 1 of 10 servers online.
Posted by DangerMouse, 07-11-2004, 11:22 AM
coming up to 12 hours here... not pleasant.
Posted by coight, 07-11-2004, 11:25 AM
We can't place our new server online with cpanel offline! Grr
Posted by eldawg, 07-11-2004, 11:32 AM
Almost at 12 hours here as well...
Posted by PhMatt, 07-11-2004, 11:42 AM
Currently, Anthony is getting ready to bring all the remaining routes live. I'll probably make no sense, but the VLAN settings without servers / switches on them, was causing a loop through the Cisco's, thus bringing up VLAN's was slow. With the majority of our boxes online at this time, we're turning all routes live. I'm just passing on what this tired old man can comprehend atm. Teams are moving through the racks, double checking for downed servers, missed cat5's, power, etc. Other teams are finalizing the rackmounts, and others working on any servers with problems. We're getting things up as quickly as possible, the last update should reap a much larger % of servers online, especially on VLAN's. By my estimate, we've got in total about 80-85 % live (maybe more). Fresh employees are scheduled in shortly, and the rest of us are working with little - no sleep, bloodied hands, feet, and other ailments, but nobody's quitting until it's done! Thanks,
Posted by MrMcGoo, 07-11-2004, 11:42 AM
Cpanell.net is up...layer1 and layer\2 Ping ?????
Posted by mainarea, 07-11-2004, 11:45 AM
DNS does not resolve from everywhere, last time I checked one of their nameservers was still down. Also, layer2 looked flakey when I tried it. Once it's stable, I'll update what I need... - Matt
Posted by snickn, 07-11-2004, 11:47 AM
Will this bring a lot of the Pwebtech servers up? I've noticed a lot of dnow are coming up..but most pwebtech are still down?
Posted by Lorenz, 07-11-2004, 11:50 AM
I am back online! Yehaawww, only 11 hours down But my congrats you finally got out of the bad arms of NAC! Let´s have a beer ;-)
Posted by billybatson, 07-11-2004, 11:53 AM
Which rack are you on, Lorenz?
Posted by Lorenz, 07-11-2004, 11:55 AM
Good question, range is 216.67.227.*
Posted by eclipsewebs, 07-11-2004, 11:58 AM
I have been able to ping the gateway ip address (64.247.32.1) for over an hour now, but nothing on our server. May just mean that our server is still offline but I am like everyone else, worried. The server has 2 ethernet ports, only 1 is live, so if it is connected to the wrong port we appear down. Our server is located at AG02 if anyone from PWEB sees this. Thanks,
Posted by NexDog, 07-11-2004, 12:01 PM
Excellent. We still have 6 servers down though.
Posted by nate, 07-11-2004, 12:03 PM
I am not actually sure about the 80%-85% estimate of people's machines bing up, but I know that 3 of my boxes are still currently down. My machines are speread all over the place, but I am now at over 12 hours downtime.... Poor !
Posted by JenniH, 07-11-2004, 12:04 PM
Matt got my hopes up with his last post... recovery sounded iminant. But we still have sites dead as a dodo here.
Posted by NexDog, 07-11-2004, 12:04 PM
Our Pweb servers are up - but they missed two and they're all in the same rack.
Posted by DreamPages, 07-11-2004, 12:04 PM
Over 12 hours here.
Posted by The3bl, 07-11-2004, 12:05 PM
Still 7 servers down here and the customers are getting very restless.
Posted by freakysid, 07-11-2004, 12:06 PM
I have two servers in AI01 and two in AM01 and they are all down.
Posted by snickn, 07-11-2004, 12:06 PM
9 unreachable here...
Posted by billybatson, 07-11-2004, 12:08 PM
AM01 still down for me, too.
Posted by M7I, 07-11-2004, 12:08 PM
One server still down in BR03, other one in BR10 has came up.
Posted by forumtalk, 07-11-2004, 12:08 PM
we are online now
Posted by AP2k2, 07-11-2004, 12:09 PM
66.246 still down. almost 12 hours now.
Posted by M7I, 07-11-2004, 12:10 PM
66.246.37.xxx here. Still down.
Posted by freakysid, 07-11-2004, 12:11 PM
Yes - we have all been down for the same time. I've just noticed an extra hop showing up in the traceroutes: But still unreachable.
Posted by M7I, 07-11-2004, 12:12 PM
I saw that hop show up lastnight at random times when tracing the dn sites.
Posted by ElrondTwg, 07-11-2004, 12:12 PM
We were up for a couple hours and now it's back down.
Posted by kiwami, 07-11-2004, 12:13 PM
YESSs!!!! wow.. no idea how worried my poor little "lets start a webhosting company.. oh no the server is down what shall i do" *** was! I hope everybody'll be back online soon ! <>
Posted by kyregts, 07-11-2004, 12:18 PM
Hey all, What a way to wake up eh? Phones ringing off the hook this morning with folks unable to get to sites or email. Ah well... Can someone tell me if Spenix.com is effected by this move? I didn't get an email from them and I hadn't checked their forums for a while so I may well be outta the loop. OOPS.
Posted by freakysid, 07-11-2004, 12:20 PM
yes Spenix.com is affected.
Posted by snickn, 07-11-2004, 12:20 PM
Spenix was at DNow I believe, so yes they would be affected.
Posted by NexDog, 07-11-2004, 12:21 PM
need.......sleep......so.....badly..........
Posted by kyregts, 07-11-2004, 12:23 PM
Welp...at least we know what's happening. Thanks all.
Posted by mainarea, 07-11-2004, 12:24 PM
Speeds are dead slow for the servers that are online... from multiple servers, I can't pull more than ~100K/s from the CPanel sites on the network. Looks like I'll need to postpone my urgent upgrades even more. It's already 12:30 and things still aren't fully back online... that's more than twice the window suggested by them at first. Hope everything comes online very shortly. - Matt
Posted by billybatson, 07-11-2004, 12:25 PM
Yes, I know that hundreds of web hosters worldwide have had a sleepless night!
Posted by NexDog, 07-11-2004, 12:26 PM
Sunday afternoon/evening here.
Posted by Lorenz, 07-11-2004, 12:27 PM
FXP to my GNAX-Backup server goes as usually with 900kb/s. All fine here.
Posted by coight, 07-11-2004, 12:28 PM
Monday morning here
Posted by billybatson, 07-11-2004, 12:29 PM
Likewise - you in Australia or New Zealand?
Posted by coight, 07-11-2004, 12:32 PM
Perth
Posted by Lorenz, 07-11-2004, 12:33 PM
6:32 in the evening here, Germany ;-)
Posted by freakysid, 07-11-2004, 12:35 PM
I feel sorry for the pwebtech/DN team - they are obviously having big problems getting us who are still down onto the network.
Posted by ElrondTwg, 07-11-2004, 12:36 PM
I feel sorry for us all. I didn't think this many of my local customers checked their email at 6AM on Sunday mornings :-). I've never received this many calls on a Sunday before...
Posted by Ebart, 07-11-2004, 12:37 PM
Hmm. I could stay up but im better off to get 4 hours sleep then might check up again. Hopefully then all is well, and yes, had I known it would be this bad, I would have moved our own site to another datacenter during the transfer. Night (from Adelaide, Australia)
Posted by JenniH, 07-11-2004, 12:38 PM
I've promised myself that next time a provider decides to change buildings, we simply leave (luckily we have plenty of spare capacity across various centers). There's no way I'm letting a third party put me through this again. I've REALLY had enough of this farce now.
Posted by kyregts, 07-11-2004, 12:41 PM
Amen Elrondtwg! I don't have alot of customers and most of them are non-profits...geesh you would think they all hit the mailservers at the sametime on sunday mornings! I sorta wish we could see what was going on so as to get a handle on progress but I know that's asking too much. Frustrating to just sit here.
Posted by Lorenz, 07-11-2004, 12:43 PM
kyregts, if you browse trough the thread, you see that the DedicatedNow people posted updates VERY often.
Posted by kyregts, 07-11-2004, 12:44 PM
I think I read somewhere in here that they are moving 40 racks of machines? That's alot of hardware to packup, move, unpack, setup, startup, test, and push live. 24hrs of darkness is reasonable I would think. Doesn't help you guys with commercial clients but still...it's an awful lot of work to do.
Posted by ElrondTwg, 07-11-2004, 12:44 PM
It really is just frustrating because I can't do anything about it, as has been said before. An endless screen of repeating "Request timed out." is very disheartening.
Posted by DreamPages, 07-11-2004, 12:44 PM
Any one come up in the last hour? Or is everyone who was down an hour ago still down like we are?
Posted by Koutaru, 07-11-2004, 12:46 PM
I'm still not up. I just wish their ETA (4-6 hours) was more reasonable since right now, it's double.
Posted by ElrondTwg, 07-11-2004, 12:46 PM
I'm sure there are other ways to go about doing this that would minimize downtime, although they would take more time and money on the part of the DN team.
Posted by coight, 07-11-2004, 12:50 PM
If it isn't up soon plenty of providers will need to wait for their dns to re-propagate (another 24 hours)
Posted by Daytona Host, 07-11-2004, 12:51 PM
No, nothing changed here. The last hour has been no-activity -- BUT!.. Maybe that means we will get a huge burst of activations in the next few minutes. * Glass Half Full * *Glass Half Full* Got to keep remembering that... LOL! :-) Good luck to all, Tim - Internethosting.CC
Posted by DreamPages, 07-11-2004, 12:53 PM
This was posted at 10:42 (over 1 hour ago): and Sound likes more than 80% of the people are still down, not up. Shouldn't take over an hour to bring up 15% of their servers. We're now in our 13th hour of being down. Losing money like crazy but anyone there relly care?
Posted by Lorenz, 07-11-2004, 12:54 PM
Well, my server is up since 2 hours already again and all stable. I guess when they said 85% are up now, this was pretty correct.
Posted by snickn, 07-11-2004, 12:55 PM
I have 9 servers spread down over many different racks....85% might have been a bit optimistic
Posted by JenniH, 07-11-2004, 12:57 PM
I certainly don't buy the 85%.
Posted by newyorker, 07-11-2004, 12:58 PM
we have only one server down atm.
Posted by snickn, 07-11-2004, 01:01 PM
One of our servers just came back....
Posted by ImLagging, 07-11-2004, 01:01 PM
1 of my servers came up recently. Now I just have to wait for the other one. Oh, and the glass is neither half full, nor half empty. The glass is too big for the amount of liquid.
Posted by kyregts, 07-11-2004, 01:01 PM
progress is a wonderful thing...
Posted by GreatDaneOwner, 07-11-2004, 01:06 PM
Damage management is in full swing on DN's part. It was a great idea to merge the threads here where hardly anybody's looking when searching.
Posted by kyregts, 07-11-2004, 01:06 PM
Ah well...They seem to be making progress. I'm gonna run off, make some calls, and try to distract myself for a while. Hope everyone get's the servers back SOON!
Posted by Daytona Host, 07-11-2004, 01:17 PM
I'm going to walk away for a couple of minutes. Maybe this is like boiling water. :-) Going to walk away, and maybe when I come back in 30 minutes all will be green.. LOL! Tim
Posted by Koutaru, 07-11-2004, 01:19 PM
:/ I'm still hoping for updates as it's been almost an hour and a half since the supposed 15% to go
Posted by PhMatt, 07-11-2004, 01:22 PM
Ok, teams are still going up and down the aisles, looking for anything out of the ordinary, other teams consoling servers, fixing boxes, etc. We're getting ready to bring the ticket system back online, fresh tech's are arriving, who will begin to start weeding through tickets, and working on downed servers that are racked, powered, and networked. Pweb servers are still being rackmounted, with their own teams who have been working on them since arrival. We're still just working as diligently as possible to get things online and working. Thanks,
Posted by nate, 07-11-2004, 01:22 PM
to be honest ... it goes something like this: 1) premove: optimistic 2) 0-6 hours down: optimistic 3) 6-8 hours down: expected as much but still optimistic 4) 8-10 hours down: starting to worry and less optimistic 5) 10-12 hours down: even less optimistic 6) 13+ hours: not optimistic at all & getting pissed off ! It is to bad all of this optimism that everyone has cannot buy clients that are lost. Enough is enough .... Dednow should admint to having royally screwed this up and start figuring out what they are going to do to: 1) get these machines up ASAP 2) make a formal apology 3) compensate all of the hosts who have been so ill effected 4) get their stuff to gether so that nothing like this EVER happens again ! ... just my 2, 3, and 4 cents ...
Posted by HostVector, 07-11-2004, 01:23 PM
I am still down with 2 servers for 12 hours now - nobody is answering the tickets. DN - please respond to tickets and give us ETA on remaining servers!!!!
Posted by nate, 07-11-2004, 01:25 PM
This makes it sounds like everyone should be up and running. However, I have 3 boxes that say otherwise. Are you actually expecting us to believe that my 3 machines were just overlooked or not plugged in or something .... lets get real here ! Refer to my previous post !
Posted by JenniH, 07-11-2004, 01:31 PM
Read DN's message of 07:23 AM: >>All notifications were made via the same method we use for our billing invoices, generated to every client at the same time, most received multiple notices. We also addressed the issue with our forums, updated people of the forums: http://clifton.fortressitx.com and posted as an announcement in our regular forums, that we were performing this move. Currently, we're underway, and the first wave of servers to be arriving NOW WOOHOO ! ! !<< No hint there of a problem at the NAC end. Now read DN's message of 08:18 AM: >>Approximately 4-5 racks are about ready to go from DedicatedNOW, and a few from Pweb. Our Network is up, and Anthony's working on announcing the IP's for BGP routing. 2nd truck should be here shortly, as soon as we're done connecting the switches for the breadracks, and rackmounts, we'll power up, and begin the first round of servers. For those wondering, the FIRST racked customer server put up tonight, was Beachcomber's by chance, but it's still not on yet<< STILL no hint of a problem but they MUST have been well aware of the NAC issue by now. Why didn't they tell us? At 08:59 AM: >> Sounds like we're behind schedule.... << >> Not that badly, consoling down the servers, and packing them, then transporting is a slower process than powering up. Tech's are consoling boxes now, wiring and powering up. Shouldn't be too long before a good majority are up. << Still not mention of the delay at NAC. First hint from DNCOO at 01:17 PM: >> Unfortunately the move is going a bit slower than originally planned. We have approx. 30+ movers to assist, but due to some restrictions placed upon us, only 8 of our employees, (No Hired Movers) were allowed in the colo to remove servers at any given time. This severly slowed down our moving abilities, but we're coming along.<< Then at 01:37 PM: >> An honest ETA would be a few more hours for proper setup and testing << But then (when it is clear to all there is a serious problem) it all had to come out at 02:34 PM, via DN: >> Unfortunately, yes, when we tried to gain access to the facility tonight, there was a large hinderance placed upon us with only allowing 8 staff members into the colo facilities at once. This was added by a door check for identification of all equipment and logging on the way out, camera crews constantly following us around, all led to a required revamp of our coordinated efforts, with movers, technical teams, to help take down servers, etc. At that point then, employees slated to go to Clifton were needed to stay behind, to help bring servers down, transport outside of the facilities where then the movers could pack and begin transportation of services. All equipment is now onsite, and all of our staffed personnel, and others are here racking, running cables, bringing up switches, servers, routing, etc. We did fall behind, and were needed most to help get customers back up and running as quickly as possible. Time is essential now for us to get everything up for our clients. Staff members have been working 24+ hours, but everybody is going strong to get things back online for everyone. We had no idea until literally the moment we pulled the network, and began to go into the colo that the restrictions would be placed on us, and ultimately cause a massive delay in our planning. We do apologize for the inconvenience caused, and are working our behinds off to get things in order as soon as we possibly can. << Now THAT sequence doesn't look too good to me, from any perspective, especially communication. However, more hours pass and at 04:42 PM we get an encouraging message: >> Currently, Anthony is getting ready to bring all the remaining routes live. I'll probably make no sense, but the VLAN settings without servers / switches on them, was causing a loop through the Cisco's, thus bringing up VLAN's was slow. With the majority of our boxes online at this time, we're turning all routes live. I'm just passing on what this tired old man can comprehend atm. Teams are moving through the racks, double checking for downed servers, missed cat5's, power, etc. Other teams are finalizing the rackmounts, and others working on any servers with problems. We're getting things up as quickly as possible, the last update should reap a much larger % of servers online, especially on VLAN's. By my estimate, we've got in total about 80-85 % live (maybe more). Fresh employees are scheduled in shortly, and the rest of us are working with little - no sleep, bloodied hands, feet, and other ailments, but nobody's quitting until it's done! << Then at 06:22 PM: >> Ok, teams are still going up and down the aisles, looking for anything out of the ordinary, other teams consoling servers, fixing boxes, etc. We're getting ready to bring the ticket system back online, fresh tech's are arriving, who will begin to start weeding through tickets, and working on downed servers that are racked, powered, and networked. Pweb servers are still being rackmounted, with their own teams who have been working on them since arrival. We're still just working as diligently as possible to get things online and working << Getting excited about this last one? Well sorry... it sounds rather like a variant of the message at 04:42 doesn't it. Not impressive is it? Well, not from my point of view anyway. The first sequence demonstrates how the major NAC packing issue was not communicated... until it HAD to be. And who's to say that there isn't an equally (or more) severe issue in play now? What confidence can we have given the above? I'm trying to be fair here, but it's getting harder and harder to just sit back and accept the situation. Last edited by JenniH; 07-11-2004 at 01:35 PM.
Posted by Nickgold, 07-11-2004, 01:33 PM
can someone tell dedicated now to get the nlayer routes they are working really slowly
Posted by Perfecthost, 07-11-2004, 01:33 PM
I have been quiet about this, so far; however, it has been over 13 hours of downtime. I have ticked-off customers. I hate doing damage control! -Lamar
Posted by snickn, 07-11-2004, 01:33 PM
What percentage of the pweb servers are done?
Posted by GreatDaneOwner, 07-11-2004, 01:42 PM
There will be no compensation, as they will figure it was somebody else's fault. And honestly, if you got, let's say, one month free, would it compensate for the damage to the image of your business, let alone lost revenue? Look at the line "no sleep, bloodied hands, feet, and other ailments, but nobody's quitting until it's done!" - they are actually convinced they have pulled off something heroic! The fact that they fed you misleading information which mislead your customers too? Too bad. 14 hours now.
Posted by billybatson, 07-11-2004, 01:43 PM
Which racks are the PW servers on?
Posted by Laci, 07-11-2004, 01:48 PM
I agree GDO , they should keep their comments about there condition out of their posts and focus on what this is doing to the rest of us.
Posted by ImLagging, 07-11-2004, 01:49 PM
Well, my 1 server that was online is now offline again. Anyone else seeing their servers go poof?
Posted by halldorr, 07-11-2004, 01:52 PM
I have to agree witht he people that are angry here. We had expected 6-8 hours of downtime and could live with that in the time slot that was mentioned but we're now looking at 14 hours of downtime. The thing is we have one server online and all of the others are down so what is going on over there? Customers are pissed and to compound matters one of the servers we have on DN was populated with clients who suffered through a crap EV1 server and now they are experiencing downtime yet again. This went from a good situation, to a bad situation which is only getting worse.
Posted by servermaze, 07-11-2004, 01:52 PM
My server hasn't been online since they started
Posted by JenniH, 07-11-2004, 01:57 PM
I just hope a few people can see through this charade. 3,000 servers in 4-6 hours? Was that EVER realistic? This looks like a total shambles to me. Reading through their posts gives me zero confidence this is going to resolve quickly. Even less confidence that we will avoid serious 'teething' issues for weeks to come. And no, given the 14 hour outage, this isn't something they can lay wholly at the door of NAC.
Posted by DreamPages, 07-11-2004, 02:01 PM
Well we're now in our 14th hour of downtime. This is the biggest FUBAR I've seen in a long time. All wiring shouldhave been done in advanced so they only had to plug in the hardware. It's also been over 2 hours since they said they were bringing up the rest of the servers. This is BS. Last edited by DreamPages; 07-11-2004 at 02:06 PM.
Posted by twidnet, 07-11-2004, 02:03 PM
I'd like to know where this figure of 3000 servers is coming from. There is no way this is accurate wen you consider how short a time DN has been around, how long they haven't been taking orders, and including some margin of attrition. I agree with Jenni, and I'm surprise anyone is cutting them any slack at all. This should have been planned much better if it really has been in the works for a year and laying it all off on NAC no matter what sort of disagreements they have is ridiculous. I've been through moves before that involve physical relocation of equipment. This is the worst process I've ever witnessed and I don't see anything that tells me they actually tested the new network before throwing clients in there. DN's communication stinks.
Posted by snickn, 07-11-2004, 02:04 PM
The court papers say 1500 servers...I agree, where's 3000 coming from?
Posted by servermaze, 07-11-2004, 02:05 PM
I agree with you. This BS
Posted by DreamPages, 07-11-2004, 02:07 PM
Court papers?
Posted by Aussie Bob, 07-11-2004, 02:11 PM
http://www.webhostingtalk.com/showth...hreadid=290814
Posted by ImLagging, 07-11-2004, 02:12 PM
NAC and Pegasus/DN are involved in a legal dispute right now.
Posted by JenniH, 07-11-2004, 02:15 PM
I heard 3,000... but exactly the same logic applies to 1,500. The scale fo the thing just doesn't fit the timescale. It's appalling. They have no excuse whatsoever.
Posted by DreamPages, 07-11-2004, 02:18 PM
So we're down 14+ hours because of this? This move is sounding more and more like hogwash. The excuses they keep giving and their status updates don't even jive. Over 2 hours ago they said they had 80%-85% of the servers update and brining up the rest. Even if they had 3000 servers, they should have the remaining "15%" up. In any case, they should have did some planning and taken precautions to ensure they would have all their customers down for any more than 6 hours. And it absolutely ridiculous to try and move 3000 servers at the same time. It should have been broken down over several weekends.
Posted by Koutaru, 07-11-2004, 02:19 PM
Very true but at the end, we cannot do anything but wait as impatient as I'm getting I know when it comes up, it'll come up and whatever I'm posting now probably will not speed up the process. Still, I agree this was poorly managed and the time frame was ridiculous. I hope I'm strongly encouraged by things that will happen after this fact or else I'll have no choice but to move again.
Posted by DreamPages, 07-11-2004, 02:22 PM
We'll all probably have to move. I've lost faith they'll be up today. I did this type of stuff at EDS (Electronic Data Systems) for several years. You don't move this many servers at once. You NEVER had a customer down more than 4 hours, 6 in extreme cases. PERIOD! NO EXCEPTION! If you needed hardware down for more than 4 hours overnight, you had to build their sites on another server in the meantime. I know they can't do this with 1,500 or 3,000 (whatever it really is) but they should have never attempted more than 50-75 at a time. I've already lost a $100/month client.
Posted by twidnet, 07-11-2004, 02:23 PM
No, we're down 14+ hours because DN didn't plan well. Period. The problem with moving them in pieces from what I understand from another host (who has great communication and explains things well) is due to DN's use of the NAC ips (technical) and because they probably want to hightail it out of NAC as quickly as possible (economic). Their problem here is the same as when Nick told everyone DN had told him the move was going to happen in May. Too little notice, too little planning, and this time, too little in the way of updates. They should have known up front that NAC wouldn't allow that many people in the NOC, because they should have done a dry run of what was going to happen WITH NAC BEFORE the scheduled move. Maybe they aren't talking or something, but it's inexcusable and clients are paying the price.
Posted by nate, 07-11-2004, 02:24 PM
I have to say that initially it did not sound bad if they had to move 1500 servers and had 30+ people to do it. That is about 50 servers each to take down, package, move, re-rack, and power per person. Now, obviously since they were only allwed to make use of 8 people at a time (staff only) that would seriously cause issues. In their defence, it was a last minute thing that NAC sprang on them. However, we could have been updated at that time so we would know to expect further delays. Also, as I sit here, I currently have 3 machines still offline for 14.5 hours. This is very frustrating for me and all of the shared hosting clients that are on these machines ... However, the people that are working on this problem have likely been awake for 30-40 hours. I do not doubt everyone is working hard to try and fix this issue. They say that 85% of the machines are online, so the rest of us have to be coming online soon right ? What really counts is that this get cleared up as soon as possible and for all of us to stay informed of what is going on.... We could use an update.
Posted by DreamPages, 07-11-2004, 02:24 PM
I guess if enough people move, they might feel the pain.
Posted by DreamPages, 07-11-2004, 02:26 PM
Yea, the lack of updates WITH REAL INFORMATION about what's going on is ridculous. The few "updates" have been filled with lies.
Posted by MirageSF, 07-11-2004, 02:30 PM
Does this mean we all have new IPs and the old ones wont work ? If so im in the **** !
Posted by twidnet, 07-11-2004, 02:31 PM
This just kills me. The people doing the move have been up for x hours. Who cares? They're the ones who chose to do the move, and it's their job. I've pulled long shifts before. They're the ones who should have planned things better, and I agree they should have posted that bit about the lack of access for that many people sooner. But they should have known about that lack of access problem way before this morning. They have neglected one of the major rules of business. Never leave your clients guessing as to what's going on. Never, never, never.
Posted by nate, 07-11-2004, 02:31 PM
Well, I would not say they are lies. But they are trying to do what all of us are doing .... and that is keep the clients happy. I know they are working on things, but it would be good to have someone give us an update like every 15-30 minutes. Lets just see what happens because at 14.5 hours, it is basically a whole day to our hosting clients ... it does not look good either way, and maybe it will be sorted soon. right ?
Posted by Laci, 07-11-2004, 02:32 PM
not from my understanding
Posted by DreamPages, 07-11-2004, 02:32 PM
2 1/2 hours ago they said 80%-85% of the servers were up and they were bring the rest up. That was an obvious lie or they would be up by now.
Posted by universal2001, 07-11-2004, 02:33 PM
Hmm, doesn't NAC have to release any type of IP or do any sort of switch transfer. Perhaps, NAC is not putting this as a priority on their list and they are taking the time. Simply talking, if you went to court to snatch IPs which belonged to NOCA, do you think they would make it a priority doing the switch.. Maybe I'm wrong and there is nothing required to be done on NACs side but things don't look good atm.
Posted by JenniH, 07-11-2004, 02:33 PM
>> We could use an update.. << The problem is that the updates tend to be worthless. Remember the one HOURS ago giving the impression that they were just applying the finishing touches? And the earlier updates which didn't mention the serious NAC problem at all? Are you really prepared to believe the next 'update'.... whenever it comes?
Posted by nate, 07-11-2004, 02:34 PM
Well, I was just stating a fact. I was not asking for anything at all, and we are all in the same boat right ? I mean, it is not like they are all sitting around eating burgers or something while the rest of us freak out .... they have a job to do and they know it. Lets see what happens.
Posted by snickn, 07-11-2004, 02:34 PM
The "honest" ETA we received at 9:30 stated a few hours.
Posted by whmcsguru, 07-11-2004, 02:35 PM
I've got to say, this has been a nightmare NOT just for pweb customers but for those responsible for their servers as well. Come on people, get your acts together and do the job you were supposed to do. Firstly, stupid question (maybe), but I'm going to ask it anyways: WHY were all servers moved at once? Seems to me that logic would state you move them in waves. One, one weekend, another, another weekend, a third, the next, and so on. By the time you get all this done, you have the move down to a science. THINK people! This could have prevented downtime of this capacity and kept a reasonably decent reputation to your company. Secondly, WHY is this taking so long? Your customers (and those of us unfortunate enough to be up all night holding their hands because you have no clue what you're doing) deserve that! NOT lies, like you've kept on going about, but the TRUTH. WHEN WILL SERVERS BE BACK UP? The million dollar question that everyone wants to know, ESPECIALLY those of us waiting on Cpanel! Come on people, get a clue! So, how long does it really take to move a server and network? About as long as it takes to drive from point A to point B, login, update 1 file and reboot the server. It's THAT simple. In fact, if the gateway is the same, and the routers are, it's even more simple. Ya'll should have had this online hours ago, what's the excuse this hour? jeez, talk about a fiasco. I agree, anyone with a pweb server should yank it, just to teach them a lesson in business practices. Patience is one thing, well over 12 hours of downtime is a completely different issue altogether. Can we say unprofessional?
Posted by twidnet, 07-11-2004, 02:39 PM
That doesn't explain why some servers on DN's own IPs still aren't up. Why is it that people find it so hard to hold DN responsible for their actions? Coming up on 15 hours now.
Posted by brzell, 07-11-2004, 02:41 PM
Has anybody with servers at location AM01 online yet? I can ping the gateway IP but not the server.
Posted by twidnet, 07-11-2004, 02:44 PM
Yeah, we are in the same boat, and the responsibility lies with DN fully. I know you were just stating a fact, but between that and Matt's bloodied hands crap, it pisses me off. I don't really care that they planned so poorly they have to stay up until it's done. I do care that they planned so poorly that it's impacting real, live clients and I care that they still haven't learned anything about communication since the last "move" schedule in May. Not even four days notice, an incredibly optimistic move timeline, no updates when serious issues come up (like the number of people allowed in NAC), more "real soon now" posts. We've seen what happens. They're not impressing me here, and honestly I don't think they will ever learn the communication lesson. People will forgive things when you're honest with them.
Posted by billybatson, 07-11-2004, 02:45 PM
We're waiting for a server on AM01 as well. freakySid speculated that the servers there haven't been racked yet. Anyone confirm?
Posted by The3bl, 07-11-2004, 02:50 PM
Badly planned is all there is to it. Still 7 servers dead in the water same 7 that was dead when 85% of servers were on line, same 7 that were dead when the ETA was a couple of more hours. I guess I should be happy that most of the servers came online after about 7 hours. But these last seven may not be back for days at this rate.
Posted by jonmck1234, 07-11-2004, 02:54 PM
Any more updates from ANYONE? Our servers have been down for 15 hours+!!!! PLEASE HELP!!
Posted by coight, 07-11-2004, 02:54 PM
We have been up Not fairing to badly, it was 9 hours and 10 minutes exactly so I'm sure all your boxes will come online very soon.
Posted by servermaze, 07-11-2004, 02:54 PM
Funny, no response from them Do they even care
Posted by nate, 07-11-2004, 02:55 PM
Agreed. We would all like updates, and there is a real problem for which dednow is responsible for. (and working on) I definately have my fingers crossed
Posted by servermaze, 07-11-2004, 02:56 PM
15+ hours also
Posted by freakysid, 07-11-2004, 02:57 PM
I have one server 69.72.158.122 in AI01 which has just come back online - sort of. I can reach it from only some locations at the moment.
Posted by GreatDaneOwner, 07-11-2004, 02:58 PM
So you would still believe their responses? I've already made the necessary changes and waiting for propagation.
Posted by Nickgold, 07-11-2004, 02:59 PM
Pings are going up even more
Posted by universal2001, 07-11-2004, 03:00 PM
Technically speaking, 1500 servers. Say 1200 is used for shared servers, each of these shared servers have 400 accounts. Due to this downtime, it could be said that you may received 3 cancellations as a direct result. Now let's do some maths. 1200 servers with 3 cancellations on each box x $8.95/mo = $32,220 (loss income - based on 1 month). Now lets say that client would of stayed for 6 months. $32,220 x 6 = $193,320 USD. Of course, this is just one way to look at the picture.
Posted by ViAdcK, 07-11-2004, 03:01 PM
We still have one server down in the BR12 rack. Things are really getting out of hand..
Posted by Perfecthost, 07-11-2004, 03:01 PM
I think my machinery must have fallen off the back of the truck. -Lamar
Posted by DreamPages, 07-11-2004, 03:02 PM
I hold DN totally responsible.
Posted by universal2001, 07-11-2004, 03:04 PM
In all seriousness, dedicatednow has had great uptime in the last 6 months. I'm sure they are working their butts off to get those servers online. Let's just give them some more time.
Posted by twidnet, 07-11-2004, 03:07 PM
Absolutely not. Hosts that treat their clients with contempt by not providing adequate information are not deserving of my business. Does anyone know what DN might have said about the connections at this new NOC? Everything I've traced from goes through uunet or nlayer.
Posted by freakysid, 07-11-2004, 03:07 PM
Another server wombat.globalnamehost.com in AM01 (I think) was up just now too.
Posted by dandanfirema, 07-11-2004, 03:08 PM
Just had one in BR13 come online. Still waiting on AM05 AM07 BR13 (different one) BR07 AN04 AN02 BR07 (2nd one) AN03
Posted by 2uantuM, 07-11-2004, 03:09 PM
People are getting over emotional over nothing. If you thought that the down time will be longer than that, say so in the letters you send out to your clients. If I was in this case, which I have been before, I would say that it would be possible to have 12 or more hours of downtime. If they meat their deadlines, great, things went faster than what you told your customers.
Posted by JenniH, 07-11-2004, 03:10 PM
>> Let's just give them some more time << Are you serious? How much time do you want? We've given them 15 HOURS already! We've put up with misinformation and zero information all day long. And our servers/sites are STILL down.
Posted by Perfecthost, 07-11-2004, 03:10 PM
So, how do you know where your's are? I must be the only ignorant one. -Lamar
Posted by nate, 07-11-2004, 03:11 PM
I just had a machine come up in: BR07A-D Remaining 2 are in: AN04B-E & AN02D-C Lets keep it up !!
Posted by Steven, 07-11-2004, 03:14 PM
Well they did say
Posted by Laci, 07-11-2004, 03:14 PM
you arent going through this right now are you? , it was short notice and they were ill prepared .
Posted by JenniH, 07-11-2004, 03:14 PM
>> People are getting over emotional over nothing. << May I say, with an attitude like that, how happy I am that I'm not one of your YOUR customers. Some people...
Posted by twidnet, 07-11-2004, 03:15 PM
So we should have just given people some random number of additional hours on top of what DN told everyone, when DN are the ones who should know, through planning, abut how long this will take? Would that be because they exaggerated their ability to get it done in the time they said or because we should know they shouldn't be believed or both? I can't believe more people are not outraged about this and keep patting DN on the back like they've done something special here.
Posted by The3bl, 07-11-2004, 03:19 PM
Yeah right so it is all our fault, their customers for repeating what they told us? Makes perfect sense to me. I should have told my clients you will be down for 2 days even though our vendor said 6 hours. I will have to remember that.
Posted by twidnet, 07-11-2004, 03:22 PM
> We have chosen MCI and Above.net for our backbone providers, and are adding Level3 and Global Crossing in the near future. Interesting. I don't see the first two in any traces from anywhere. Not that I care all that much at this point, because like others I'm making other arrangements.
Posted by Mark_YH, 07-11-2004, 03:23 PM
A bit off topic but I was curious if there was some reason why DN wouldn't have chosen to move say 250 servers at a time and taken a month to move everyone over, rather than trying to move everything and everyone all at once? I know it doesn't make a bit of difference now but if there was some reasoning for it I would like to understand it... I do hope they get this move completed quickly for the sake of everyone that has been affected....
Posted by mainarea, 07-11-2004, 03:28 PM
CPanel's sites still aren't up 100% after the move, this is pissing me off... it's better than my server being down, but it still sucks. - Matt
Posted by The3bl, 07-11-2004, 03:29 PM
Routers are not cheap.. To move servers in small batches would have cost addtional money for new routers. That and I am sure if you read the law suit you will see there was some serious issues with NAC that forced this move all at once.
Posted by Koutaru, 07-11-2004, 03:31 PM
I would still like an update, nonetheless, no matter how falsified some people claim it is. Why are some servers still down and how many exactly?
Posted by M7I, 07-11-2004, 03:38 PM
Just had my last server come up. It shows that is has been up over 5 hours, but has not been accessilbe. I guess they are making it around and correcting little network issues that are keeping servers down. I sure hope it was not just disconnected! Good luck to all of you that still have servers down, I hope they get everyone up very soon.
Posted by DNCOO, 07-11-2004, 03:39 PM
I just wanted to chime in and clarify a few details as I see people are tired, upset, angry, and most importantly stressed out.. As mentioned in previous threads, we were involved in legal proceedings with our provider. I am not going to go into those details, but it took a judge and a temporary restraining order to allow us to move out. This wasn't something that could be stragically planned, we had to wait for a window to open andtake the opportunity and run without looking back. I know many of you might not understand our situation, but it was not a plesant ordeal. It affected you OUR clients and YOUR clients the same.. same stress, same downtime. We didn't sit around and decide to move our datacenter when we thought it would be convenient for us, we had to use the power of jurisdiction to protect us and allow us to move all our servers out in a very VERY short time frame. If we had the luxury of scheduling a move, we would have chosen a rack at a time over several weeks, months, etc..but that was simply not our situation. When people say that we should of known the restrictions, the restrictions were put on us VERBALLY at a moments notice and that is a simple and honest truth. I do greatly apologize for what everyone is experiencing, but it will all be resolved soon, we can all get some much needed sleep, and we will be able to put this issue behind us. We are doing our best and we do acknowledge everyone's right to be frustrated and angry. We will be much happier when this situation is over. Back to work!
Posted by porcupine, 07-11-2004, 03:40 PM
Agreed, for the boatloads of cash many of us Pay to CPanel, I'm disappointed to see that their promises last year of getting "all their eggs" out of one basket dont seem to have happened, because everything I can see in relation to CPanel is completely down. We have IP's going dark next week as we'd renumbered out of them months ago. Some lazy arsed customers still hadn't migrated their main IP's, so are running around like ducks with their heads cut off, and now we cant move the license files as necessary, ARGH! How irritating, I'm very disappointed.
Posted by Perfecthost, 07-11-2004, 03:40 PM
Support ticket to DedNow:
Posted by freakysid, 07-11-2004, 03:41 PM
So I have two servers up in the past half hour - two more to go. Come on guys you can find them in AM01 and AI01
Posted by mrzippy, 07-11-2004, 03:41 PM
You would have been wise to post that last one several (many) hours ago. You have a lot of pissed of customers simply because you didn't bother to explain the situation clearly.
Posted by JenniH, 07-11-2004, 03:43 PM
ETA? Issues causing the most recent delays (given the earlier optimism)? Reason we wern't told about the 8 man NAC delay earlier than we were? Etc. We need facts. We need to know when our servers will be up.
Posted by Koutaru, 07-11-2004, 03:45 PM
I agree that the the post should have come sooner but I appreciate it anyway Ed. However, I still would an update, however, on the situation as of now. Thanks and keep us aware of what's going on
Posted by Bocaweb, 07-11-2004, 03:46 PM
I see many servers back up but I am seeing lots of packet loss, anyone else seeing the packet loss?
Posted by freakysid, 07-11-2004, 03:47 PM
I imagine it will be a matter of days before they iron out the network.
Posted by twidnet, 07-11-2004, 03:49 PM
Give me a break. DN could have planned this move for ANY TIME. You didn't have to get an order to allow you to move. You had to get an order to allow you to use NAC's ips because you were still using those ips and did not finish changing ips by the time you wanted to end your contract or didn't get enough IPs to begin with or some other random thing you will almost certainly blame on someone else. I may not be up to speed on all the technical ins and outs, but I know a half-assed job when I see it, and I can read legal items. If you've been plannig this for a year as claimed, you should fire whoever did the planning. They did a crap job of it.
Posted by coight, 07-11-2004, 03:54 PM
LOL Tech: Tech: Tech: Another Tech:
Posted by Laci, 07-11-2004, 03:55 PM
Ed, I understand your position , and I understand what you went threw with NAC , but please try to look at it from our view point , we were given 4 days notice which because of our set up was ok we were able to get the word out others were not so lucky, but here we are 16 hours or so past ETA , our client are climbing up our ..... body parts and all we are getting from you is have sympathy for us , that wont fly to our clients. The point is better real (even if its ugly) communication.
Posted by newyorker, 07-11-2004, 03:56 PM
Have a server still down on: AN02 rack
Posted by ViAdcK, 07-11-2004, 03:59 PM
This is looking to become 24+ hours downtime..
Posted by Eiv, 07-11-2004, 04:00 PM
Non of my servers are up yet. I am getting very upset.
Posted by whmcsguru, 07-11-2004, 04:01 PM
And that short time frame was why? Because you launched an all out attack against NAC, your current provider. There's an old saying about burning bridges. Don't do it! You , effectively did by launching an all out attack on NAC legally. As well, regarding this: It's been shown that you have lied to your customers regarding this, you chose to originally move these 2 months back,with no notification whatsoever. Thankfully that fell through. You'd think you would have learned then, right? You had 2 months to work things out, you refused to do so. I'm not saying NAC isn't at fault here, but you're just as much at fault as they are for your customer's downtime. Don't try the "poor me" attitude here, it doesn't work well. SURE it was. You just had to plan correctly. Again, the legal procedings are brought on by your own stubbornness and inability to cooperate with your current provider. Who pays for this? Your customers. Sadly, not going to happen. WHT is a community of individuals, and this crap will show up every time someone does a 'search', as well as the last time you tried to pull this without notifying your customers. You don't just unplug all servers and run. You move them over groups at a time, common sense dictates that. Now, what's going to happen when the electric company raises the rates, or the current bandwidth provider raises the rates? Your customers will have to go through the same crap again. Unprofessional, that's all I've got to say. I feel sorry for the individuals with servers with you guys. At one point i was looking for one there, but, given the state of the company, nd this whole fiasco, and how it was handled by all parties (nac included), there's 2 more providers I can cross of the list of professionality.
Posted by newyorker, 07-11-2004, 04:09 PM
Can anyone PM me a list of current and well-known NAC resellers?
Posted by coight, 07-11-2004, 04:15 PM
15minuteservers.com - It's run by NAC
Posted by TotalChoice - Bill, 07-11-2004, 04:21 PM
One of our boxes is still down, been down since 12AM EST last night. NOW TO MAKE MATTERS WORSE, the IP's on another of our boxes just started routing somewhere else. So we now have two downed servers. This is awful.
Posted by snickn, 07-11-2004, 04:24 PM
Justr hope you don't need a FSCK on any of your servers Then they will have to request the root pass from you..etc. We only have 3 down now...better then 15.
Posted by thedavid, 07-11-2004, 04:26 PM
Well, here we are - 16 hours after the one nac/pwebtech server we have went down. After 4 days of account moves, attempts to contact customers, and all that - we're left with that server providing only DNS for resellers we couldn't get in contact with. We set the TTL's extra high, to give a nice margin-of-error for those customers - hoping that it would at least smooth things out. Did I take into consideration that it'd be 16 hours later, and still no server? No. Sites are now down. Those that are now seeing them down are contacting, and we're of course giving them the IP addresses we've procured elsewhere. By the end of this, all of the clients are going to be off of that server. So what's the incentive to stay with pegasus/dedicatednow/fortress/whatever now? All the work of moving everyone is done, on our part. All that's left is for them to hopefully get the server back online for a little while so that the customers can get *some* uptime while their DNS changes. I'm dissapointed.
Posted by MirageSF, 07-11-2004, 04:30 PM
Waiting on servers on rack AN03, any ideas ?
Posted by DreamPages, 07-11-2004, 04:30 PM
Mine's been down 16 1/2 hours. How much longer?
Posted by X-Max, 07-11-2004, 04:32 PM
WHATEVER, One thing is for sure, i will never ever do business with NAC or any provider based in NAC. Last edited by X-Max; 07-11-2004 at 04:41 PM.
Posted by cnm72, 07-11-2004, 04:37 PM
I am just coming online after being down for 16+ with this situation... there were no issues on my end due to this and I am thankful... as a future reseller I have learned a few things from this situation by monitoring everything that has taken place: a) always have an email list backup as a way to communicate with your clients even if there is a forum system used for normal communications b) once you receive this sort of info from you host, try to research it to find out if there is any history on the situation c) when telling clients about this open the "window of downtime" a bit more just to be safe d) have a back up site with another host for emergency communication purposes I don't blame anyone for this... ordinarily my hosting company is superb in support/communications and rolled with this situation as best they could... in my 7 months of being with them this is the first major situation we have been through and it was/is being handled swiftly and professionally as I have come to expect from them... there is no such thing as perfection and I am sure that MANY lessons were learned by the main players in this migration... good luck to all... Last edited by cnm72; 07-11-2004 at 04:44 PM.
Posted by Laci, 07-11-2004, 04:41 PM
We are back up now , my thoughts go out to those who arent.
Posted by snickn, 07-11-2004, 04:43 PM
We have 2 servers out of 15 left....
Posted by servermaze, 07-11-2004, 04:44 PM
I agree with you 100%
Posted by twidnet, 07-11-2004, 04:44 PM
I feel for the guy posting over at the DN forums whose COLO equipment, which requires special handling for shutdown and power up, was taken out of NAC to this new place. He wasn't told about the move. Can't get anyone from DN to respond. Doesn't have access to the new place. Etc.
Posted by DreamPages, 07-11-2004, 04:48 PM
Down 17 hours now. This is ridiculous.
Posted by The3bl, 07-11-2004, 04:48 PM
I feel for us all....
Posted by servermaze, 07-11-2004, 04:50 PM
My server is up now YEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEES
Posted by Lorenz, 07-11-2004, 04:51 PM
Mine is since 6 hours *g* ;-)
Posted by ElrondTwg, 07-11-2004, 04:52 PM
A lot of my friends in the hosting community are telling me their servers have started to come up in the past few hours. Anyone know how I can contact DN to let them know that mine is down? None of their sites resolve for me so I can't find any place to enter trouble tickets.
Posted by jonmck1234, 07-11-2004, 04:52 PM
one up.....four to go
Posted by twidnet, 07-11-2004, 04:55 PM
Good point. It's a total fiasco.
Posted by steves36, 07-11-2004, 04:59 PM
24 hours!
Posted by shimmy, 07-11-2004, 04:59 PM
My server was never up (BR12) , but now I can't even reach dedicatednow.com or clifton.fortressitx.com
Posted by GreatDaneOwner, 07-11-2004, 05:00 PM
I'm surprised that while most posters here criticize DN, the response on their own forums (both new and old are down for me as I'm writing this, though) is mostly positive. I mean, what would have to happen to you till you see that you've been ... well, mistreated?
Posted by gimbo, 07-11-2004, 05:00 PM
I have a reseller account hosted with eurofficial.com 17 hours , and they are down yet
Posted by thedavid, 07-11-2004, 05:00 PM
At this point - I'm operating on the idea that the server will *never* come back up. Maybe they dropped it, or its hard drive failed on bootup, or whatever else. Mom always said 'prepare for the worst, hope for the best'... (sigh)
Posted by Jay H, 07-11-2004, 05:04 PM
Unfortunately, this situation has went from 'understandable' to 'unbearable'. I'll be switching providers after the few servers I have there actually come back up. The total lack of communication (i.e., your company site offline during the move, whose brilliant idea was this?) and the fact that we are working on 17 hours of downtime now is completely NOT understandable from a provider who promises uptime. Four days notice for a downtime that was more than likely not going to only last 4-6 hours is COMPLETELY not understandable. I'm sure DedNow knows that many of us have hundreds and hundreds (if not thousands) of clients that are DIRECTLY impacted by this. Why not use their head and give further notice? If you would have done that, at least I could have moved my servers to a different datacenter in time and avoided this disaster you've created. DedNow better be thinking to themselves right now, "Oh boy, we screwed up.". Also, "How are we going to regain trust in our client base?". And last but not least, "Lets schedule some more billing employees to handle the cancellations". You should learn from your mistakes and correct them instead of continually making the same mistakes.
Posted by Jay H, 07-11-2004, 05:05 PM
Don't worry, many of us have servers STILL offline. They would have to be clumsy as hell to drop all of ours.
Posted by thedavid, 07-11-2004, 05:11 PM
Yeah, I know... But having worked for a large telecom (working with the several datacenters they had) I do know this - servers do *not* like to be powered off and carted across town. Power-up in those instances are hardest on hardware. Wonder if they had any contingency plans in effect for hardware that fails on bootup? Ugh...
Posted by whmcsguru, 07-11-2004, 05:13 PM
This just in A truck carrying an undisclosed number of servers has been involved in a near fatal accident. The contents of the truck were spilled into the street and surrounding water. Sorry, my humor for the day
Posted by thedavid, 07-11-2004, 05:14 PM
ROFL. Thanks for the laugh.
Posted by Koutaru, 07-11-2004, 05:17 PM
The last update from Matt was HOURS ago, and I'm not talking just 2 or 3. Where's our update?!
Posted by Jay H, 07-11-2004, 05:20 PM
They posted the update to their website. Sorry, my sarcasm is growing as the day goes on ....
Posted by Eiv, 07-11-2004, 05:21 PM
IS DN/Pwebtech going to compensate us for the ridiculous downtime. I am really pissed off with this.
Posted by Jay H, 07-11-2004, 05:22 PM
^^ Would be nice ^^ Although, at this point I really couldn't care less. I'm going to place the servers I have with them over at EV1.
Posted by classaction, 07-11-2004, 05:27 PM
I think that it is time to step up the game. I am loosing to much of my business and I am sure others are do to poor planning on Pwebtech/Dedicated now. A class action lawsuit is in order in my opinion. I am not going to loose revenue because a company's poor actions lost my business.
Posted by thedavid, 07-11-2004, 05:27 PM
Everyone's over at theplanet now, here.. Purchased/provisioned the server as soon as I got done reading the notice of the physical move.
Posted by mrzippy, 07-11-2004, 05:28 PM
Unless the combined revenue loss is proven to be in the thousands, then I doubt any lawyer is going to take on your case.
Posted by The3bl, 07-11-2004, 05:29 PM
Lets have a sing along. . 99 servers on the rack take one down drive it around 98 servers on the rack... Take one down drive it around 97 servers on the rack..........
Posted by classaction, 07-11-2004, 05:29 PM
I am sure if you add up the hundreds/thousands of clients of Pwebtech/DedicatedNow it will.
Posted by Eiv, 07-11-2004, 05:30 PM
Oh well. At this point I need all my server up asap. I still have around 10 servers that is down right now. But I do think they need to compensate us for this. This is really beyond bearable.
Posted by ImLagging, 07-11-2004, 05:31 PM
I'll only sing if all servers are back online by the time we reach 0 servers on the rack. :p
Posted by classaction, 07-11-2004, 05:31 PM
I am slated to loose $10,000 on the year because of this move. And that is if nobody else cancels with me.
Posted by eldawg, 07-11-2004, 05:32 PM
Gee...nice first post, buddy.
Posted by mrzippy, 07-11-2004, 05:32 PM
Then go for it.
Posted by ElrondTwg, 07-11-2004, 05:33 PM
LOL, I only have one server at DN and can easily prove over $1,000 in damages for myself. We won't be billing a single DN client of ours this month and we've lost several people who have been with us for over a year. I'm probably an extreme case since our average client pays 19.95/mo, but cut that in half and multiply by 1500? You're going close to a million there...
Posted by classaction, 07-11-2004, 05:34 PM
I am tired of this. I think everyone else is too! I worked by but off for this company and now it's on a crash dive because of 17.5 hours of downtime!
Posted by classaction, 07-11-2004, 05:35 PM
If others are willing to talk with me regarding this issue, it can be arranged! I should say, 17.5 hours and counting...
Posted by mrzippy, 07-11-2004, 05:35 PM
Can you prove negligence? ie: Not just saying that they "should have planned better", but actually prove that they could and should have done something different? Can you prove they did not meet the fudiary duties, etc.? Seems to me that they did give notice of the move, but they will simply argue that some unavoidable problems happened, etc... you'll have a very difficult time, since it doesn't appear they purposely tried to hose you. The fact that they did a bad job of planning isn't enough to win a class action suit. Although you might win in civil court.
Posted by coight, 07-11-2004, 05:38 PM
That's one bad thing about leasing you have zero control over what the datacentre does.
Posted by classaction, 07-11-2004, 05:39 PM
I am not looking for some criminal case... I am looking civil, that is all I can do. Hell, NAC could very well be the pure cause of this entire issue over the 6 hours. Pwebtech stated six hours, they have to live up to their six hour guarantee. Besides, how much money are our clients loosing by their sites not being online? Everything trickles down to the smallest client.
Posted by brzell, 07-11-2004, 05:39 PM
Its fun sitting here watching your business go up in smoke !!
Posted by mrzippy, 07-11-2004, 05:39 PM
That's not how damages would be calculated. Otherwise, you could easily say you had 500 clients all paying $100/month.. They will simply calculate your cost payable to the provider. But basing your claim on YOUR individual lost revenue is going to be very difficult in a class action.
Posted by ElrondTwg, 07-11-2004, 05:40 PM
Yea, no ****ing s**t... 2 years of credibility with my customers gone in a weekend where it's sunny and beautiful outside and I can't bear to leave my screen...
Posted by mrzippy, 07-11-2004, 05:43 PM
It was not a guarantee unless you have it in writing with a signature from an officer of their company. Let's turn it around for a second.. Let's say YOU have 1500 customers on one server. (for the sake of argument...) Now let's say you announce an outage for a harddrive replacement that is scheduled to last 1 hour. Now, 9 hours later and the outage is still on. Are you customers going to win a class action lawsuit against you?
Posted by capecod, 07-11-2004, 05:45 PM
I have more than twenty servers. But some of them are stilll down. If this was due to an accident, I would not be up set so much now. But, DN did this on purpose. They did not give us notice long enough. I asked them to stop the move, but they did not listen to me.
Posted by classaction, 07-11-2004, 05:47 PM
They very well could sue me, you never know. They stated four to six hours, I can understand if it runs a little over, but we are now 13 minutes from 12 hours over their stated time frame. Besides this is America, where anyone can sue anyone for anything. Not that I have that attitude, but simply put because of this transition to their new facility I have lost a substantial amount of my business.
Posted by mainarea, 07-11-2004, 05:48 PM
Your problem is with DedicatedNow, not NAC. Plain & simple. All servers were out of NAC about 12 hours ago. - Matt
Posted by classaction, 07-11-2004, 05:49 PM
In my opinion, their servers should have been the last to go online. The clients should always come first.
Posted by mainarea, 07-11-2004, 05:50 PM
You're not going to get anywhere suing dedicatednow. Their SLA does not cover this kind of stuff, and they had no SLA on their estimated downtime. A lawyer will laugh at you when you bring them your story, I guarantee that. THeir servers are online so that they can communicate with you... - Matt
Posted by snickn, 07-11-2004, 05:50 PM
Having a method for the clients to communicate with the compan yis more important. There will be servers taht don't come up due to manual fscks being needed to be done etc...the ticket system and billing database needs to be up some Pegasus can contact the clients.
Posted by mrzippy, 07-11-2004, 05:51 PM
Then go for it. Monday morning go see your lawyer and let everyone here know what are the costs, and how to join in the lawsuit. It would be interesting at the very least.
Posted by qps, 07-11-2004, 05:51 PM
UCI/PWEB/DNOW/etc: Having been in a similar situation before, I empathize, and I hope that in the coming days, weeks, and months ahead you will take comfort in the fact that you will hopefully never go through a situation like this again now that you have escaped the anti-competitive and predatory behavior of your provider. Best regards, Jeffrey Kehe
Posted by DreamPages, 07-11-2004, 05:54 PM
We're going into our 18th hour of being down. Spent over $100 in online ads. Usually get close to $1,000 in orders through our own Ticket company ecommerce site. Had a client quit on me already. Been great weekend.
Posted by classaction, 07-11-2004, 05:54 PM
I have a lawyer on retainer and have already spoke with him regarding this situation. He stated that the firm will look into this tomorrow morning and let me know. I will be sure to pass the same information he gives me on to all of you to consider. If the firm says there is a chance, I will have a phone line setup for this by Tuesday afternoon at the latest. I am not playing around here. This is my livelihood; this is how I support my family.
Posted by Koutaru, 07-11-2004, 05:57 PM
Now I have to leave, still with no update or any CLUE to what's going on to my server. So much for communication.
Posted by billybatson, 07-11-2004, 06:03 PM
Can anyone give us a run down on what racks are up and what racks are still down? AM01 is down, as far as I know, with the exception on freakysid's machine(s).
Posted by brzell, 07-11-2004, 06:03 PM
Has anybody up yet on location AM01? This is really starting to get scary ! They will not answer support tickets or phone calls!!
Posted by brzell, 07-11-2004, 06:06 PM
We dont even know if they have lost or left some servers back at NAC ! Come on Pegasus, dont do this to us!
Posted by billybatson, 07-11-2004, 06:06 PM
We're still down on AM01, but freakysid thought one of his machine, which is up, was on that rack.
Posted by twidnet, 07-11-2004, 06:09 PM
Hope you'll congratulate the people escaping from an unorganized, poorly communicating provider (DN), too.
Posted by inteltechs, 07-11-2004, 06:15 PM
our servers are on that rack... it's still down.
Posted by ElrondTwg, 07-11-2004, 06:21 PM
For those of you who are familiar with the poor state of technology and situation of some foreign countries, perhaps you will appreciate this... A good friend of mine runs a software development firm in Bangladesh, a country most known for its poverty, roaming blackouts, annual flooding, and underwear. Having no access to high-speed Internet their office is connected 24/7 to the Internet through multiple dialup modem lines... After this day's disaster... he can now boast higher uptime in 2004 than I can... me in my American datacenter and he in his third-world flood-ridden, blackout-torn (they live off of diesel generators over there) village office. Oh the irony... I can hear him laughing right through the Instant Messenger program...
Posted by Daytona Host, 07-11-2004, 06:21 PM
Is this my missing servers???? http://clubs.juniata.edu/compsoc/drop/truck.jpg Just trying to keep my spirits up.
Posted by universal2001, 07-11-2004, 06:28 PM
Yeah that would be the Redhat servers, FreeBSD ones are o.k.a.y.
Posted by coight, 07-11-2004, 06:39 PM
No I believe this is your missing server http://gallery.brouhaha.com/ruckus_fire/img_3657 http://gallery.brouhaha.com/ruckus_fire/img_3731
Posted by Eiv, 07-11-2004, 06:45 PM
Stop posting those pictures. It scares hell out of me. Does anyone have any update? I am getting very crazy right now.
Posted by Perfecthost, 07-11-2004, 06:46 PM
Ridiculous.
Posted by whmcsguru, 07-11-2004, 06:46 PM
Something told me this was coming with the move. I agree the problem is with pwebtech, but it is ALSO with NAC if they're refusing to release the IP's being the major PITA's that they can be. However, if pwebtech had played nicely then nobody would be in this situation, pretty simple really.. Gotta wonder though, how many people left after the last time they tried pulling out on their customers over the weekend. And if they didn't why not? Certainly you can't say you didn't have forewarning this was going to be happening, especially after the lawsuit et all.
Posted by puzzlekid, 07-11-2004, 06:47 PM
Thanks God that WHT isn't on their datacenter now! I'm hosting on Spenix.com, and the strange thing that is happening to me is that one of my accout is suspended and all others still works. Their website is down, but mine did not get down! Just one!
Posted by Laci, 07-11-2004, 06:49 PM
last time? what happened?
Posted by Joshua, 07-11-2004, 06:50 PM
They originally had servers in both BurstNET and PWebTech, and then migrated them all to PWebTech. Something tells me that there was some poor disaster planning done by the makers one of the most popular (possibly the most popular) hosting control panel out there today. We have 1 server of our own at Burst, and 1 server of our own in ThePlanet for redundancy. If one server goes down, we'll still be able to communicate with clients and switch our sites over to the new server. We do daily backups to a second harddrive and backups every 2-3 days to a server located at EV1. If we knew that extended downtime was going to be happening, we'd switch over to another server. If something went wrong with one of our servers, we'd still have our backups offsite, ready to transfer elsewhere. Sure, CPanel may have a lot of data to transfer, but I don't see any huge hassle of them creating a mirror server elsewhere. With the number of licenses they have, I'd expect them to have servers located in at least 2 different datacenters across the country. It's not like they can't afford another server... or maybe they've come down with the WHT "el cheapo n00b user" syndrome -Josh
Posted by mainarea, 07-11-2004, 06:53 PM
Looks like they still have NAC bandwidth: http://www.fixedorbit.com/AS/25/AS25653.htm and http://eng.nac.net/announce.shtml (look for 25653, that's pwebtech). - Matt
Posted by whmcsguru, 07-11-2004, 06:53 PM
this thread will show more. Basically, they tried pulling everyone out (with zero notice) on Memorial day of this year, same thing. It became public, they cancelled, for whatever reason. They denied it at the time, but the court papers show that they actually DID try to do this (read through the thread, the link to the docs is somewhere). Of course, suing your current datacenter won't help you get the ip's from 'em either. I'd say it'll be a few days guys, most likely NAC's screwing around with pwebtech and someone's got their head too far up their tail to come to a reasonable agreement.
Posted by GreatDaneOwner, 07-11-2004, 06:56 PM
A couple of weeks ago they were planning to make the move, and told only a handful of selected clients beforehand. These hosts informed their customers, naturally, and all hell broke loose. DN were not planning to inform anyone else until it was only a day or two to go [edit: my memory was too kind to them - zero notice was planned, actually; just checked]. Then when they saw the outrage they backed off. Last edited by GreatDaneOwner; 07-11-2004 at 07:02 PM.
Posted by Laci, 07-11-2004, 07:00 PM
looks like I have a few things to consider GDO .... this is aweful
Posted by Joshua, 07-11-2004, 07:01 PM
Moral of the story: Don't pull a "UnitedColo". Give plenty notice . I think that the servers that still aren't up have surpassed the time it took to unrack all of UnitedColo's servers, and fly them cross-country to Florida
Posted by twidnet, 07-11-2004, 07:04 PM
And the proof of this is where? Why would NAC violate a court order? If I was them, I'd be happy to get DN out the door at this point and watch them fall on their face, which they have done. What remains to be seen is how happy people are going to be when DN doesn't get everyone off the NAC ips by the deadline they gave in the legal papers. Because we all know it's going to happen. Looking at this fiasco of a move tells us how well they plan things.
Posted by billybatson, 07-11-2004, 07:06 PM
How long has it been since we had an official update?
Posted by X-Max, 07-11-2004, 07:07 PM
it is mainly due to NAC, which is causing problem. why don't they understand that they should cooperate with company hosting with them since years and which was probably their biggest client !!! What is guarantee if i host with company based in NAC, NAC will not repeat this practice with them, if both went in dispute ? In whole situation the only negative point i see is that its my mistake that i am hosting with provider in NAC otherwise i must not have been in such kind of situation. My all DN servers are back except 2 which was hosted with NAC Premium bandwidth / pwebtech and these both use ips with NAC ownership and i think this is what causing the main problem. Edit: My all DN servers are back except 2 which was hosted in NAC Premium bandwidth. Yes at present i am not able to make out who is wrong because i am really MAD from explaining my clients, sitting on my single a$$, since last 18 hours that its not my fault if their web sites are offline and data center is still moving servers and i don't have any updates or contact with anyone. Last edited by X-Max; 07-11-2004 at 07:11 PM.
Posted by GreatDaneOwner, 07-11-2004, 07:07 PM
I have already considered those few things... as I said earlier, I'm just waiting for propagation to complete. This is for my VPS with them. As for Bluewho (this is what you were hinting at, weren't you ) - I trust them 100%. If in their best judgement they say they're staying, I'm staying with them. GDO
Posted by kickmybutt, 07-11-2004, 07:09 PM
LOL.... too dang long! And at this point, it's beyond ridiculous and more like idiotic! DN screwed the pooch on this one!
Posted by AP2k2, 07-11-2004, 07:10 PM
they should move 25% at a time instead of moving ALL. Geezh..
Posted by Laci, 07-11-2004, 07:11 PM
I'm not leaving anyone , merely considering my DC options.
Posted by thedavid, 07-11-2004, 07:13 PM
I've already sent in our cancellation request. This long with no updates is not something I can forgive easily.
Posted by twidnet, 07-11-2004, 07:17 PM
Get a grip. It's NAC's fault that DN said 4-6 hours and it's now 19? Please. DN screwed up. Too much in a hurry to bolt NAC and not enough planning will do that. And you only know what DN has said about the problem they have with NAC. No one that I know of has heard the NAC side of the story, and you don't hear these kinds of problems with places like Voxtreme, who have servers at NAC. I really don't care about the he said/they said crap anyway. I'd like to know why on earth DN didn't have better planning in place and why they gave a woefully inadequate timeline to their clients. Who, by the way, are the ones who pay DN's bills for them.
Posted by brzell, 07-11-2004, 07:19 PM
We went thru the whole IP migration process last week and completed it. I guess it really does not matter because we have been down now for over 19 hours
Posted by ElrondTwg, 07-11-2004, 07:22 PM
With no update...............
Posted by Jay H, 07-11-2004, 07:23 PM
Exactly. This has gone from understandable, to unbearable to outrageous. The complete and utter lack of updates is horrid. I have 1 server with them that is still down, 19 hours of downtime. (going to be more, I bet). I have a server provisioned and on standby for when this one comes back up, I can't wait to start the data transfer and get it the hell away from DedNow.
Posted by brzell, 07-11-2004, 07:23 PM
What a way to run a business.....just take the phone off the hook and ignore e-mails. I AM PISSED !!!!!!!!!
Posted by AriNorth, 07-11-2004, 07:27 PM
Laci, I wish to stay with my current provider as well (the same as yours) but I am also considering my DC options.
Posted by X-Max, 07-11-2004, 07:30 PM
twidnet as i told you, at present from the information available all i can make out is that yes, its NAC fault, they are acting like $hit with pegsus. The point is that, if they can do like this with pegsus they can do the same with Voxtreme, site five or whom ever hosting with them. If NAC is not at fault then they should clear their stand point here in public. But if they can't post anything due to trial still running then its not my fault that my thinking is going against them. i know, 6+ extended down time is not due to NAC but the basically they forced pegsus to move out from NAC like crazy. Yes its clear that pegsus failed on their promises and the last thing i want to see is my server online so that i know that its not broken in move and it will bring my business online.
Posted by GreatDaneOwner, 07-11-2004, 07:30 PM
Just curious: how much do you think this whole thing will hurt DedNow? Have they grown big enough to be able to stay afloat after losing this many customers and with such a big dent in their image and credibility? Or was the name change a clever plan to control the possible damage caused by this?
Posted by GreatDaneOwner, 07-11-2004, 07:33 PM
That makes three of us... And probably more.
Posted by M7I, 07-11-2004, 07:40 PM
We only had two servers at Pwebtech/DN and they are both back up. One was at BR03 and the other BR10. I think stating the 4-6 hour window was unreachable at the beginning and I am not suprised it was missed, but come on....Where is the communication. I do not believe there has been a DN update for over 4 hours. I appreciate the hard work and long hours that the staff is putting in, but just a small update will make a lot of people happy.
Posted by billybatson, 07-11-2004, 07:40 PM
When was the last time someone had a server come online?
Posted by DreamPages, 07-11-2004, 07:41 PM
Mine have been down over 19 1/2 hours.
Posted by twidnet, 07-11-2004, 07:42 PM
X-Max, your reasoning makes no sense. Did NAC tell DN to get out? No. Did they tell them they were going to shut everything down if they didn't leave? No. DN MADE THE DECISION. They didn't like whatever NAC was charging, and that's fine. It's a free country. Solution? Move. But not like this. Trying to sneak a move back in May? Then, learning no lessons AT ALL, doing it again now? You're fitting right in with people blaming NAC for everything when the fact of the matter is: DN DID NOT ADEQUATELY PLAN THIS. Period. You can try to lay the blame at NAC's door, and DN can try to blame everyone but themselves for everything, too, but at the end of the day, DN 1. claims to have planned this for a year, but never bothered to get completely out of NAC's ips, knowing they would eventually have to do just that 2. tries to pull a move on two days notice in May, without even notifying their customers AT ALL 3. gives a notice of a grand total of about four days before this move 4. doesn't even notify some people at all (at least they say they never received notices, lke the COLO guy) 5. gave a tremendously optimistic timeline for the completion, and they had to have known it would take a lot longer than 4-6 hours with (generously) 1000 servers or so 6. when the moves started and they KNEW within minutes that NAC wouldn't let a bunch of people in the NOC, failed to update 7. failed to update in a timely manner throughout the process, even when it became clear that they were going to overrun the timeline 8. keep playing the woe is us game (bloodied hands, up for hours, blah blah blah) I have no sympathy for DN (and I'll tell you that I won't do business with NAC, either, until they give their side of the story and even only then if I find their explanations suitable). Know what? That's my right. Yours too. Don't want to do business with anyone in NAC? Don't. But don't say it's just because of what DN claims they did, when the only thing we have is DN's story on the problem. In your blind loyalty to DN you're missing out on some very, very real problems they have.
Posted by mrzippy, 07-11-2004, 07:43 PM
When is the final date for all NAC ips taken by pwebtech that need to be renumbered? In other words, when will NAC be legally allowed to null-route the IPs taken by pwebtech?
Posted by X-Max, 07-11-2004, 07:43 PM
HasGreatDane, DN is very reputed and there are many loyal customers of them. I am sure they will loose some clients but, if they are able to run there DC without any down time and proper support as it was before i am sure they will gain much much more business then what they may loose now. There support is best at the price and their staff is very helpful. if its me i will still host with them if uptime is more then 99.96% in their new DC.
Posted by whmcsguru, 07-11-2004, 07:46 PM
DN hasn't changed names, nor has PWebtech. They've been the same for as long as they've been around. DN is just a budget provider kind of like sm and tp. As far as how much it'll hurt 'em? Honestly, it won't hurt 'em a bit. Now, if they keep up the same cheap practices that they've had going all along during this ordeal, THAT will hurt them. By cheap practices, I mean: -- not informing customers of what's going on -- moving servers without telling customers (or planning to) -- attacking their current DC when they KNOW damn well they need the ip's for another few months at least. -- Planning poorly for this kind of problem (all they had to do was call ahead and find out they can't have 30 guys in the DC). -- Lying to individuals who are their customers. THAT will kill any business, I don't care which it is. The responsibility and fault lies not only with DN, but with NAC as well. They were ordered to provide service for a limited time, but NOT to provide that service with the same IP addresses, and, quite honestly, I wouldn't put it past NAC to refuse to reallocate those ip addresses. I mean, they just lost one of their biggest customers, what you expect 'em to make it EASY on them to leave?
Posted by X-Max, 07-11-2004, 07:51 PM
twidnet you are right, i agree with you that it was very lame decision on DN parts to move out like this but i read somewhere they was going to turn them off if DN wasn't to move their servers out from NAC. whatever, you are right the problem do exists on DN part also and i will be happy to know NAC stands point before i take my informed decision. I will study this whole fiasco in detail when my servers are online.
Posted by twidnet, 07-11-2004, 07:52 PM
Incorrect. NAC was basically ordered not to change anything related to what ips DN was using, and not to interfere with the announcements, based on the order. The date given in DN's legal filing says they want use of the ips until September 1. After this debacle, I would have no faith at all in their ability to get the remaining people off those ips by then. Maybe they will. If I have to renumber anyway, I might as well do it somewhere where the provider actually talks to their clients.
Posted by puzzlekid, 07-11-2004, 07:53 PM
Kinda strange thing here with Spenix.com Their website come up now and Matt responded me that my account is suspended for a malicious IRC program. I thought he is more profi than that. He could just send me an email instead of suspending my account. My main account!
Posted by ViAdcK, 07-11-2004, 07:55 PM
After 22 hours, all our servers are up again. What a relief... My thoughts are with all of you still affected.
Posted by GreatDaneOwner, 07-11-2004, 07:55 PM
The uptime for my VPS for the current month is 93.63% (!!). And a huge part of the downtime was caused earlier, not by this move. My VPS went down several times and they never cared. They told me that it was not monitored and I had to tell them when it was down. And this was not the first serious problem with them. So I'm wondering what makes them that good. I will never know, though; I've already signed up with my new VPS provider.
Posted by Eiv, 07-11-2004, 08:08 PM
Man.. I need to cry. I still have a few server offline. Is there anyone still down?
Posted by coight, 07-11-2004, 08:09 PM
So is everyone up now?
Posted by shimmy, 07-11-2004, 08:11 PM
Nope I'm down
Posted by thedavid, 07-11-2004, 08:12 PM
Yup, just passed the 20 hour marker here. I think, at this point, I don't really care if it ever comes back up. DNS updates are already in the works for those clients I mentioned earlier in the thread, and everything else is being served elsewhere. Hour by hour, dedicatednow becomes less important. Something tells me that there's something drastically wrong at this point - not that they didn't have enough people to move the servers, not that the servers are still being moved, but there's something archetecturally wrong (power, network) that's keeping them offline. A damaged router, perhaps? Who knows? Nobody, that's who. I've been able to contact nobody, been able to get no updates, etc... Just doesn't seem 'professional' to me.
Posted by billybatson, 07-11-2004, 08:13 PM
Down on AM01.
Posted by eclipsewebs, 07-11-2004, 08:13 PM
We are still not up either. Sure would be nice if we could find out what is supposed to be up.
Posted by puzzlekid, 07-11-2004, 08:14 PM
What alternatives are there for VPS?
Posted by DreamPages, 07-11-2004, 08:17 PM
That's funny. Yea, right. They screwed up so badly they aren't admiting anything.
Posted by ImLagging, 07-11-2004, 08:21 PM
Both of my servers are still down.
Posted by Daytona Host, 07-11-2004, 08:24 PM
Still have 2 servers down here.
Posted by kyregts, 07-11-2004, 08:27 PM
My Spenix stuff is up and running fine. puzzlekid - Matt and his folks at Spenix don't play when it comes to security issues and IRC of any flavor is just that. He did the same thing to me when a forum I was running had a security problem. Fix the problem(ie, can the IRC) and you'll get reenabled.
Posted by brzell, 07-11-2004, 08:27 PM
Still down on AM01 over 22 hrs .....No updates....No e-mails....Busy Signal on Phones....
Posted by puzzlekid, 07-11-2004, 08:34 PM
yeah, fix the problem with ssh accound disabled. I have to hack into it and fix things there And there are no other "folks" at spenix, just Matt! And come on, wtf security problem would a forum be for the server? Are you one of Matt nicks?
Posted by Samuel, 07-11-2004, 08:36 PM
Erm, a forum has the ability to be very insecure... if you don't realize that then consider this, a one line bit of code can take down an entire server very quickly if coded badly or maliciously. But this thread has to do with DNOw moves
Posted by kyregts, 07-11-2004, 08:40 PM
puzzlekid- I was using YaBB and was woefully behind on some security fixes that were quite....dramatic in scope. My stupidity. hehehe...I reckon it would be pretty difficult to fix your stuff when you can't get in eh? Sorry about that...it's been a long LONG night...ohyeah...night AND day. Now...I'm off to bed I think before I embarrass myself again.
Posted by thedavid, 07-11-2004, 08:42 PM
Exactly - can we keep it to dedicatednow server move downtime, everyone? Thanks.
Posted by NexDog, 07-11-2004, 08:46 PM
Our last 2 servers just came online. Praise be to god, allah and all that is holy.
Posted by nate, 07-11-2004, 08:48 PM
I still have one that is down in: AN02D-C I wonder what is taking 20+ hours ? Very strange indeed.
Posted by inteltechs, 07-11-2004, 08:48 PM
are those dedicated or colo servers?
Posted by billybatson, 07-11-2004, 08:48 PM
Where were they, NexDog?
Posted by MonaCK, 07-11-2004, 08:48 PM
I'm still waiting for mine. Why do I have a feeling that my box fell off of the back of one of the moving trucks?
Posted by universal2001, 07-11-2004, 08:51 PM
All servers up for us.
Posted by freakysid, 07-11-2004, 08:52 PM
For those in AM01 - I did have one server come up 5 hours ago. Still waiting for the other one and for one in AI01
Posted by Daytona Host, 07-11-2004, 08:55 PM
All mine are up except '1' ... Come on, 1 more and I'll be happy.. :-) We're almost there... Tim
Posted by amerasearch, 07-11-2004, 08:56 PM
This will just about destroy us. We're small anyway. Thanks guys!
Posted by nate, 07-11-2004, 08:59 PM
That is how I felt about 4 hours ago ... but nothing positive is happening on this last box I have I sent in a ticket but I doubt that will help. I tried calling, but no go on that as well. I really hope this is sorted soon ... for me and for all of you.
Posted by NexDog, 07-11-2004, 09:00 PM
One is leased from DN and the other is leased from Pwebtech. The Pegasus one was in AM03, not sure about the other.
Posted by PilotHost, 07-11-2004, 09:12 PM
Does anyone from AN08 have a server online? Thanks, Troy
Posted by nate, 07-11-2004, 09:17 PM
Or how about a: AN02 ??
Posted by BeDifferentSolutions, 07-11-2004, 09:29 PM
Why are people so filled with doom and gloom? It is Sunday and even a 24 hour outage will not "destroy" a standard, commodity-based hosting business. People will find out from this just how resilient customers really are. Years ago we had a server down for over 24 hours with over 200 sites. We lost 1 customer. Just 1. Apologize. Refund their money for the month (actually apply a credit so you are not out real dollars). Tell them all the positive to come from this move. Maybe if you advertised you were some RackSpace service or offered 100% uptime, then you should lose business for being stupid.
Posted by DreamPages, 07-11-2004, 09:33 PM
They could at least give us the real story. No one know what the real problem is. We've only been given crap about not getting into the new data center and something 8-9 hours ago about the servers coming up. Alot of us have no idea if we will have anything up tonight, by morning, or even in the next 24 hours.
Posted by TotalChoice - Bill, 07-11-2004, 09:38 PM
We still have servers down. When will this be resolved!!!!!!
Posted by amerasearch, 07-11-2004, 09:38 PM
Maybe it's the fact that they said 4-6 hours, and people even gave the benefit of the doubt and said 8-10 hours, then they come back AFTER THE FACT and say we had this problem or that. They should have been upfront to begin with. That's why people are P--------. Maybe it's the lack of updates. "oh we got a couple more to go". I can bet that's way off base.
Posted by BeDifferentSolutions, 07-11-2004, 09:39 PM
I have complete understanding that you guys should be pissed. In fact I would be furious too. I am just pointing out the fact that this issue will not "kill busiensses". It sucks and I feel for everyone. But don't get so upset and negative to think your business is history. It isn't. Customers will stay. Last edited by BeDifferentSolutions; 07-11-2004 at 09:44 PM.
Posted by twidnet, 07-11-2004, 09:42 PM
Not to mention the fact that in some places, it's been Monday for quite some time. You know, a regular business day? The whole world doens't revolve entirely around the east coast of the USA.
Posted by amerasearch, 07-11-2004, 09:43 PM
Some of us might not be so large as others to be able to withstand it maybe?.........
Posted by amerasearch, 07-11-2004, 09:45 PM
And while I am on the East Coast USA, the majority of my clientele are abroad, and so have to agree wholly with this statement
Posted by Perfecthost, 07-11-2004, 09:49 PM
I guess I am feeling pretty embarrassed about this. We have NEVER had anything like this happen before. If feels kinda like being on a sports team with a non-losing streak for over 5 years, then getting skunked and finding out that hundreds of people bet on you to win one more time. However, this is no game. We are in damage control mode now, trying to calm customers with pretty much no information to pass along. In a few hours, some of our busier sites on the other side of the world will wake up and realize something is desperately wrong. -Lamar
Posted by amerasearch, 07-11-2004, 09:54 PM
How true that is. Many of mine already have learned of this....... I have absolutely no faith that our sites will be up anytime soon, and I've spent entirely too much time going back and forth on these forums only to learn nothing. I'm outta here, and hope the phone is done ringing for the night.......
Posted by Perfecthost, 07-11-2004, 09:58 PM
However, I will agree with Billy (or whatever his real name is), although we are going to lose customers over this, they will be only a small percentage. If you are a very small company and worried about losing all your clients, I advise to contact each one right now with a personal note. Customers get antsy if they think you do not know about the downtime or just do not care. Do not be negative in the email. That will make them negative also. Finally, contact them every 30 to 60 minutes to let them know you are still there and have not left. We are trying to answer questions just as fast as they come in. Damage Control, baby! -Lamar
Posted by amerasearch, 07-11-2004, 10:03 PM
Been there, done that, only nothing new to offer, as we are being offered nothing
Posted by HostFolks, 07-11-2004, 10:09 PM
Feel for you guys, hours on end of downtime is not good. It may be worse for your health than actually loosing money by it. We had the whole deal with Burstnet's day-or-so downtime with one reseller server and a few resold servers on their network and it was 'pull yor hair out time' eventhough, the vast majority of our actual hosting clients and our sites were up on a different network. Stick with it and pull something postive out of it once it's all done and dusted.
Posted by Eiv, 07-11-2004, 10:12 PM
I would appreicated if at least they give us an update. My clients keep asking me for whats going on and ETA for the service to be backup.
Posted by halldorr, 07-11-2004, 10:12 PM
Bah, went out today hoping to come back to good news. All but one server is up now which is good but we've got 22 hours of downtime on a server containing very sensitive clients
Posted by mctDarren, 07-11-2004, 10:17 PM
Wow. I feel like I just ran a marathon weeding through this thread. Truly feeling for those that are still down. I have to reinforce what PerfectHost says. Last year I had a reseller account go down for over 24 hours. Granted, I only had 15 customers at the time, but lost none because of (a) credit for that month's service and (b) constant communication with them via email. I truly hope no one loses it all because of this...
Posted by App-Jason, 07-11-2004, 10:37 PM
Not to disagree, UmBillyCord- but this sort of thing DOES kill businesses. As soon as my sites come back up, I'm backing them up for my customers and then helping them find new hosts. This will be the second outage in 4 days that I've had nothing to do with and these people are going to be furious at 8 AM EST tomorrow. I've had enough.
Posted by NexDog, 07-11-2004, 10:51 PM
So your closing up your business?
Posted by ImLagging, 07-11-2004, 10:51 PM
One of my servers came back up again. The same one that was up for about an hour about 6 hours ago. Now I'm just waiting for the second server and I'm happy (relatively speaking)
Posted by MonaCK, 07-11-2004, 10:53 PM
I'm going on 23 hours of downtime here.. Ah well. What can you do? Pray that it'll all be up in the morning, I suppose...
Posted by App-Jason, 07-11-2004, 10:55 PM
At this point Laurence, I really believe so.
Posted by Koutaru, 07-11-2004, 11:00 PM
Wow and 6 hours later I come back and guess what? It's still not up. Surprise surprise, good thing I didn't waste my time.
Posted by akuo, 07-11-2004, 11:02 PM
I guess it all comes down to individual circumstance. We've been involved in a couple of instances similar to this (admittedly not as bad), and have always been amazed at the resilience of our customers. Our fears are probably amplified by the fact that we, as hosts, live, eat and breath hosting. If you're a customer-oriented host, a server going down for half a day represents a tragedy of epic proportions. In all honesty though, the average web hosting customer isn't generally half as concerned by such things. Good communication at a time like this can even strengthen your customer's loyalty - hard to do when you've got no information to pass on of course Best of luck to you all!
Posted by The3bl, 07-11-2004, 11:04 PM
That is not good. I hate to see something like this run anyone out. We will be busy moving lots of clients and dedicated customers who have already expressed desires to locate somewhere else, many have lost all faith in Pweb and do not want to continute to place their business in their hands. So looks like we will be getting another rack at DV2 soon.
Posted by thaphantom, 07-11-2004, 11:10 PM
yea I figured that Pegasus would have this many issues, my condolences for all those of you still with them... So anyone still down?
Posted by TCH-Mike, 07-11-2004, 11:10 PM
Still down here
Posted by NexDog, 07-11-2004, 11:11 PM
Yes, it's a shame that this experience could make someone want to give it all up. I had a hell of a day yesterday trying to keep my cool when angry clients would call us non-communicative and non-responsive. We have a mailing list and this was announcement but 50 or so people say they didn't get the message and then would submit tickets with an email address on a server that was down so they couldn't see our response. People were using the emergency pagers on our forum without actually reading any of the forum and there were various threads on the issue going on. All very exasperating. And to top it off, to wake up this morning and have my sysadmin tell me that 2 servers weren't up, one of which is the current reseller server where most clients have been with us less than a month was very disappointing. Luckily Jason responded to me immediately, found the servers and brought them online.
Posted by Koutaru, 07-11-2004, 11:12 PM
Still need AM01 all the way back up. :/
Posted by MonaCK, 07-11-2004, 11:12 PM
Still down... *whistles* *sets up parchisi board*
Posted by The3bl, 07-11-2004, 11:18 PM
Geez we have been going throughthat ourselves but we must not be as VIP as you becasue Jason Matt no one is answering our emails or the phone. VERY very frustrated and upset at the moment and I have gotten 1 hour of sleep in the last 36. .
Posted by kickmybutt, 07-11-2004, 11:24 PM
Man, I just can not believe that they have not popped in just to say SOMETHING!! even if it was to say "You suckers!!!" But, I bet you $100 that Matt comes back and says "Sorry I did not update everyone sooner, I wanted to keep my focus on getting everyone's servers back online". It only takes 1 minute to type something out. This little stunt is costing many people dearly and DN doesn't even show up for HOURS to let anyone know whats going on. Nice Job!
Posted by coight, 07-11-2004, 11:24 PM
Laurence we had the same problem, people posted in our reseller forum asking why it was down we even have guidelines on posting server down messages indicating to try several steps before posting a message and that includes checking our news/server status messages and still their were three threads! Around 20 tickets also! We post them in mailing lists on an official outage page and also on our forums and people are still ticked off!. Most hosts don't even bother informing their clients let alone post it in four different places
Posted by NexDog, 07-11-2004, 11:39 PM
Robert, I'm still fielding nasty tickets. One client just said that using a mailing list was "pathetic". The funny thing is this guy or his client submitted about 20 tickets and all were answered immediately. Some days I just want to rip people's throats out.
Posted by TotalChoice - Bill, 07-11-2004, 11:41 PM
WE ARE UP! 23 hours and 40 minutes of down time. Thanks DN!
Posted by Koutaru, 07-11-2004, 11:42 PM
I can imagine how DN feels right now :p probably the same that we all do with all the complaints we've been recieving... and probably a little scared and antsy to respond to this 34 page thread. Heh.
Posted by DreamPages, 07-11-2004, 11:45 PM
Wish we were up. 23 hours, 45 minutes
Posted by Perfecthost, 07-11-2004, 11:46 PM
Me too.
Posted by myusername, 07-11-2004, 11:48 PM
I will take the hundred dollar bet. Now, to change the topic from bashing to positve thinking.... How many of you are getting more orders than usual on a Sunday night? My guess is more than a few of you. Yes, you are lossing clients, as am I, but guess what? There is a hint of a chance that the clients that did decide to leave the 50-100 or who knows how many hosts are shopping for a new one, on the same network.... We shall see how it pans out.
Posted by Laci, 07-11-2004, 11:49 PM
I read this and I'm shocked ( although Ive been around a long time ) I cant imagine how they rationalize this , and the lack of support. Hang in there to those who are still down
Posted by myusername, 07-11-2004, 11:52 PM
WOW! I am glad your average sale is $8.95/mo I am also glad that you have 1500 overloaded servers! Must be nice to be large enough to not give a hoot about losing 4500 customers for no reason other than poor planning. Many of the hosts here would be happy to have 4500 domains, let alone not give a whistle about losing that many. Must be nice to be able to loose that many customers do to bad planning on someone outside of your direct controll's err.
Posted by Aussie Bob, 07-11-2004, 11:53 PM
UmBilly is right. It might be dark now, but light comes in the morning.
Posted by The3bl, 07-11-2004, 11:58 PM
Uh Bob morning has come and gone it is now the afternoon..
Posted by Koutaru, 07-12-2004, 12:00 AM
It has officially been 24 hours of downtime
Posted by Jay H, 07-12-2004, 12:01 AM
Well, looks like I'm approaching the 24 hour mark. Thanks DN!
Posted by myusername, 07-12-2004, 12:03 AM
I wonder what the 30-40 movers faces looked like when the only people with security clearance came out and said, "Oops, sorry, they are only allowing 8 of us to unplug the servers. I guess 8 of you can go to the new datacenter and help plug things in, and then since we will be sooo slow in doing things, we only need one truck to run the servers to the new DC, so the other 31 of you can just go home." "Sorry about that, I guess we forgot about that minor detail after all the thought and planning that went into the new datacenter move."
Posted by DreamPages, 07-12-2004, 12:05 AM
Maybe thought went into it. Obviously the planning part was left out.
Posted by freakysid, 07-12-2004, 12:10 AM
The other two of my severs that were down are now up. One that was up is now down - but that could be because I have a faulty driver of config of my NIC on that server? So almost there. But good thing is all my shared hosting servers are up!
Posted by halldorr, 07-12-2004, 12:23 AM
Well all but one of our servers are back but it's the worst possible one fr us to have down Over 24 hours now, clients are furious and I thought EV1 was bad
Posted by Daytona Host, 07-12-2004, 12:27 AM
We're in the same boat. Our largest reseller server is still down. No word from the NOC. Nothing. I don't know what to do at this point. I don't think I've ever been so embarrassed in a professional situation before.
Posted by halldorr, 07-12-2004, 12:31 AM
The exact same feeling we've got. While our clients are typically quite good about these things, they can only be patient for so long.
Posted by amerasearch, 07-12-2004, 12:31 AM
Stopped to check in and completely not surprised. This is a fiasco of major proportions. Yeah, ours is still out.
Posted by Eiv, 07-12-2004, 12:31 AM
Are they responding to ticket or phone? I really need to get my server up asap. Anyone know how to get hold of someone?
Posted by MonaCK, 07-12-2004, 12:39 AM
24 hours, 38 minutes for a 4 - 6 hour server move. *ponders* Okay. A watched pot never boils.. I suppose I'll hope that they manage to get things back online by 9am ET tomorrow morning when I head into work... Meanwhile... well.. right now, i'm a huge supporter of "misery loving company". If other people are still down, at least I know I'm not alone...
Posted by rois, 07-12-2004, 12:45 AM
Its hopeless if their own 'signup' page happen to be on one of the server that is still down And if the client happens to be shopping around in the 50-100 host that are also waiting for their server to go up, then no luck for the client too... so a lose-lose situation imho.
Posted by Daytona Host, 07-12-2004, 12:47 AM
You can rest assured you are not alone.
Posted by Koutaru, 07-12-2004, 01:01 AM
It's.. 25 hours now Go us last few unfortunate souls :p
Posted by Jay H, 07-12-2004, 01:04 AM
This is damn pathetic. 25 hours and counting. Tisk tisk tisk
Posted by The3bl, 07-12-2004, 01:08 AM
My theory is they ran out of cables and some one is trying to find a Frys open at midnight.
Posted by Jay H, 07-12-2004, 01:11 AM
^^ LOL ^^
Posted by jayglate, 07-12-2004, 01:13 AM
We are truly sorry for the delay for anyone who still has servers down. Alof ot servers for some reason arn't coming back up properlly, despite being shutdown safely. Our staff is working double overtime with alot us have already been going for over 36 to 48 hours straight without any rest, to rectify any and all outstanding issues. Currenlty our staff ing working on all tickets in the system. Please DO NOT submit duplicate tickets as that will only slow down the dispatching of the techs to fix the issue, and might actually result in your server being rebooted, after it was already brought back up. So if you need to submit again please do so under the original ticket #. We thank you all for your patience in this matter, And we look forward to resolving any and all outstanding issues as fast as possible. Thank you.
Posted by jayglate, 07-12-2004, 01:15 AM
I think we actually have several hundred cables more than what we actually need. To add to that, every server is already plugged in to ethernet and power. As i mentioned in my previous post, a number of servers havn't came backup properly. So each and every server needs to have a crash cart hooked up to it.
Posted by Perfecthost, 07-12-2004, 01:26 AM
I feel like I just sank to the bottom and fell through the cracks. -Lamar
Posted by NexDog, 07-12-2004, 01:28 AM
I'm sure many here will be glad to see you're still around, Jay. If people have outstanding tickets and they are being worked on, a small "Your server has been located and is being worked on" would go a very long way. Everyone understands that this is a huge undertaking but everyone needs good communication. It's not too late to turn this thing around.
Posted by Daytona Host, 07-12-2004, 01:29 AM
I agree. Communication is the biggest fix-all.
Posted by NexDog, 07-12-2004, 01:29 AM
If you have a ticket open, you can be sure it's being worked on.
Posted by inteltechs, 07-12-2004, 01:32 AM
481643 is my ticket... it has not been answered for the last 6 hours.
Posted by halldorr, 07-12-2004, 01:33 AM
me too, except I've now filled out a trouble ticket...I hadn't wanted to clog their helpdesk anymore...
Posted by coight, 07-12-2004, 01:33 AM
We had 3 from one customer in a 15 minute period and then 5 minutes later a post on our forum as to why we werent responding to his ticket
Posted by nate, 07-12-2004, 01:34 AM
Well, I submitted a ticket about 3 hours ago via their little web form but never received any autoresponse or confirmation, etc... So I then resubmitted again and nothing... I guess I donot know, after 25.5 hours of downtime whether they are even aware of my server being offline. kinda worries me !
Posted by NexDog, 07-12-2004, 01:41 AM
Which webform is that? You should email support@dedicatednow.com or unixtech@pwebtech.com and get an email receipt of your ticket.
Posted by inteltechs, 07-12-2004, 01:41 AM
....
Posted by nate, 07-12-2004, 01:43 AM
This is the one: http://www.dedicatednow.com/?pid=contact_us Submitted 2 times about 4 hours ago and nothing ... trying email now. Thanks for the help.
Posted by The3bl, 07-12-2004, 01:45 AM
I have sent a few emails to those addresses and got no auto response over the last 12 hours. I finally used the ticket form at DN's web site and got a auto response so at least they should know we are alive and waiting.
Posted by nate, 07-12-2004, 01:51 AM
Funny, sad, and true all at the same time I finally got a ticket number so maybe when I wake up, it will magically be working. *fingers crossed*
Posted by Koutaru, 07-12-2004, 02:03 AM
26 hours and counting. I think I'm going to take a break soon and hope for the best even though support is crawling. (sigh)
Posted by arubenstein, 07-12-2004, 02:07 AM
University Communications, Inc., d/b/a Pegasus Web Technologies and DedicatedNOW, (UCI) has made some very strong allegations against Net Access Corporation (NAC). We feel it is necessary to clarify many of these issues. There are always two sides to every story, and we would hope that the reputation we have built by almost ten years in business as an Internet Provider would make people pause and at least question the allegations in this matter. Never in our history of dealing with tens of thousands of customers has this sort of thing happened to us (or has anyone accused us of this sort of behavior). There are many facts that are in dispute, and until there is a full trial of the facts with testimony from witnesses (including the owners of NAC and UCI), I would ask that you remain open minded about this situation. Please read the full text, complete with several documents, at the following URL: http://www.nac.net/statements/2004-07-11.asp
Posted by Jay H, 07-12-2004, 02:09 AM
Too little, too late. You guys have disappointed me beyond belief today with your lack of updates and the matter of servers being down for 24 + hours. This is beyond understandable. I don't feel for you, I don't care you've been up 36-48 hours. I would have loved more notice so I could have made this move on my time. Because of your actions, a lot of us have been forced to be up 36-48 hours just answering tickets from our furious clients. I'm baffled at this point.
Posted by thaphantom, 07-12-2004, 02:17 AM
HAHA That is great! GO NAC contiue to show the smack down to those pegasus losers!
Posted by mrzippy, 07-12-2004, 02:17 AM
Interesting reading. Very informative, and it's nice to see "the other side" of the coin - finally. Looks like it's no longer "he said she said". NAC has finally spoken back. It'll be fascinating to see how the courts decide. Sure makes pwebtech/dn look silly, though. Although I wonder what flavor of salt to use when reading anything anymore from any provider at all.
Posted by mrzippy, 07-12-2004, 02:20 AM
"According to our records, UCI completed the removal of equipment from NAC at 7:36 AM, on July 11, 2004." Might be helpful to post the timezone, just to remove any doubt and provide absolute clarification.
Posted by thaphantom, 07-12-2004, 02:28 AM
most likely local time zone which I *believe* is EST
Posted by arubenstein, 07-12-2004, 02:34 AM
sorry. It was EST, GMT -0500. I updated the document. Thanks for pointing that out.
Posted by thaphantom, 07-12-2004, 02:37 AM
Alex, Thanks for the read, it is great! I knew Pegasus was lying since the beginning, well maybe not lieing, but deff fabricating the stats. BTW I loved the part in your response to the TRO how u but how UCI was in trouble financially, with a 63k OVERDRAFT to their bank (http://www.nac.net/statements/statem...t%20Access.pdf page 14). You really should have shut them off soone
Posted by RossH, 07-12-2004, 02:38 AM
Thanks alex for finally shedding some light on the situation and not taking the blame for all of UCI/DNOW/Pegasus screw ups.
Posted by iFlash, 07-12-2004, 02:49 AM
This makes zero sense. If you have the servers labelled and you know who has servers that are down (and how could you not know since everyone is helpfully letting you know in this forum) why on earth would you not just find those servers, hook up a crash cart and bring them up? You do have your stuff labelled, don't you? What is the problem with getting these people's servers back on line? Get these people back up for crying out loud!
Posted by coight, 07-12-2004, 02:53 AM
One thing about one of those nac documents states that you have received over 60,000+ abuse complaints since they were your customer then further down it's 45k?
Posted by thaphantom, 07-12-2004, 02:56 AM
coight, diff timelines read it again.
Posted by mrzippy, 07-12-2004, 03:03 AM
My favorite document is the affidavit from David Kovach, former senior unix admin for the plaintiff. Here's the guy who apparently built the server farm basically saying that university/pwebtech/dn screwed themselves with regards to the IP renumbering issue. Seems they had YEARS to plan and do more planning. http://www.nac.net/statements/statem...d%20Kovach.pdf lol. Number 19 is my favorite point. Last edited by mrzippy; 07-12-2004 at 03:07 AM.
Posted by coight, 07-12-2004, 03:06 AM
Yep sorry, I haven't slept for 17 hours
Posted by Daytona Host, 07-12-2004, 03:18 AM
We're up. It's been a very long road and I sincerely want to wish everyone else good luck. I feel like I know so many of you all now and I would like to say thank you for your jokes, information, and just being there to 'vent' with. We all run the same type of business and in some ways we all are competitors - but I wish to see NO ONE loose their hosting business because of problems like this. I hope each of you all are able to hold on to your businesses and we all joke about this day a few weeks from now. 7/11 should become an international holiday. On this day, all hosting providers get a day off. :-) [Dreaming] Have a wonderful night, Tim
Posted by RossH, 07-12-2004, 03:24 AM
Read again.... In the first part it said in the 5 years UCI has been a customer nac has handeled over 60,000 complaints. Then farther down it says in the past 2 years there have been 45000+ complaints.
Posted by inteltechs, 07-12-2004, 04:37 AM
are they still working on it? My servers are still down...
Posted by Jay H, 07-12-2004, 04:46 AM
Same here. 1 server still down, would like it up now.
Posted by The3bl, 07-12-2004, 05:01 AM
4 pop back on line about 30 minutes ago, 1 still dead as..... I want it up.
Posted by inteltechs, 07-12-2004, 05:03 AM
do they need root password?
Posted by The3bl, 07-12-2004, 05:07 AM
Doubt it if they open one of the MANY emails and support tickets to them they have it.
Posted by Jay H, 07-12-2004, 05:35 AM
Yawn. *Wonders how much longer my server will be down in this screwup *
Posted by 0wned, 07-12-2004, 05:41 AM
I'd really like to know why earlier when tracing the IP of our downed machine it would go through alter (new DC) and time out, but now the trace to the same IP leads back to NAC although the trace now completes the machine still isn't reachable?? Any ideas?
Posted by inteltechs, 07-12-2004, 05:47 AM
what a nighmare!!! our servers are still down... our customers have complainted left and right. Give me a good place to hide now ....
Posted by mdrussell, 07-12-2004, 06:06 AM
Our last one came up around 9am BST today... 28 hours later.
Posted by Wako, 07-12-2004, 06:17 AM
How is the cpanel's licensing server? Is it still down?
Posted by whmcsguru, 07-12-2004, 06:21 AM
Cpanel and all relative servers have been up for a few hours now (over 12).
Posted by John D, 07-12-2004, 06:34 AM
Anyone know what happened and why the servers have been down so long? I thought the 8 hours downtime that was planned was quite a lot
Posted by Jay H, 07-12-2004, 06:35 AM
After 44 hours of no sleep, bloodied hands and all out exhaustion (yes, I still have my sarcasm), I can finally hit the sack as my last downed server at DedNow has finally came back online .... 30 hours and 30 minutes later.
Posted by GreatDaneOwner, 07-12-2004, 06:41 AM
It was 4-6 hours in their announcement. They wouldn't tell what happened, or to be precise, what they tell is clearly not truthful.
Posted by RossH, 07-12-2004, 06:42 AM
Congrats Jay and good luck with that well deserved sleep.
Posted by The3bl, 07-12-2004, 07:07 AM
I cannot beleive this.... Still a server down and my poor customers are taking a beating. You would think with as much money as they take out of my credit card every month they could at least respond to a support ticket to say they are working on it, to go to h*** or something. The silence is deafening
Posted by inteltechs, 07-12-2004, 07:13 AM
you are still lucky I have colocated 2 servers with DN and both of them are still down...
Posted by eclipsewebs, 07-12-2004, 07:19 AM
You got that right, we have a colo server with them and if it isn't coming up right I would like to know why. Gotten only the automated response to my support ticket at 9 pm est last night. I sent it to tech@pwebtech.com since we colo with them. Anyone got a different pwebtech email address that would be better?
Posted by inteltechs, 07-12-2004, 07:23 AM
support@dedicatednow.com or unixtech@pwebtech.com
Posted by mdrussell, 07-12-2004, 07:28 AM
Anyone got a phone # for the new facility?
Posted by The3bl, 07-12-2004, 07:34 AM
No but if you get it let me know. If I had an address I would pay a friend to go there and find out what is going on.
Posted by RossH, 07-12-2004, 07:37 AM
Allright techark paying someone to drive down there and break some legs is a little extreme isn't it j/k
Posted by Knogle, 07-12-2004, 07:37 AM
Wow, what in the world is DNOW up to? Around 30 hours into the operation and some servers are still down? They must be having some fun.
Posted by mdrussell, 07-12-2004, 07:41 AM
I know where the DC is, I saw it in construction.... but this is frustrating.
Posted by Perfecthost, 07-12-2004, 07:41 AM
We had about 28 hours of downtime. I am totally embarrassed and ashamed of it. I'm too tired to be mad right now. Wait until I've had a little more sleep. -Lamar
Posted by Perfecthost, 07-12-2004, 07:43 AM
Oh, and thank God we lost zero customers.......so far. -Lamar
Posted by inteltechs, 07-12-2004, 07:43 AM
looks like only colo servers are down at this moment. Our servers are still down.
Posted by 0wned, 07-12-2004, 07:47 AM
We still have a server down and it isn't colo.
Posted by Curious Too, 07-12-2004, 07:52 AM
I still have two servers down that are not colo servers. I moved websites off of those servers back in May when I first heard about this server move, so it's not hurting business.
Posted by Aussie Bob, 07-12-2004, 08:01 AM
Your cancellation requests will come in (if they do) 4 to 5 days after the downtime, thus allowing for propagation etc.
Posted by Perfecthost, 07-12-2004, 08:04 AM
You are just a fount of good and joyous news! -Lamar
Posted by FHDave, 07-12-2004, 08:07 AM
How far are you to the DC? Since it's colo, can you just give them a visit and work on your server instead of letting them do it, if that's at all possible?
Posted by freeva, 07-12-2004, 08:08 AM
I seriously cant believe this. I still have 2 server that are down. This is a nightmare.
Posted by Knogle, 07-12-2004, 08:11 AM
Dave brings up a good point -- you should be able to visit the DC if it's geographically not too distant.
Posted by The3bl, 07-12-2004, 08:14 AM
Yep I agree it is a nightmare and we will get the cancelations in about 4 days after the servers are back online. This is just disgusting, I feel very sorry for my clients catching it from their clients and I have nothing to tell them.
Posted by FHDave, 07-12-2004, 08:14 AM
Only if you are a colo customer, on which case, you should be able to visit the dc 24/7. If you are a dedicated customer, then you should (may) not have access to your server.
Posted by whmcsguru, 07-12-2004, 08:22 AM
32 hours and STILL counting? All I've got to say is DAMN. If the crap w/ NAC didn't discourage me from hosting there, this most certainly would. Maybe someone needs to teach these guys a lesson in professionality.
Posted by X-Max, 07-12-2004, 08:31 AM
my customers are extremely pissed off, i have already given 50% of them with one moth free hosting costing us more then $1500. I wish i should have given time to move my premium server to other location which is STILL DOWN.
Posted by X-Max, 07-12-2004, 08:32 AM
btw, i have already moved all account to another server and now waiting for propagation.
Posted by nate, 07-12-2004, 08:50 AM
.... wow, i got 6 hours of sleep hoping I would wkae up to 100% of my servers online, but guess what ? yep, you guessed it, 33 hours and counting ! I am not sure what to say. ... still no reply to my tickets .... .... still no answering phones .... what do I do ?
Posted by ImLagging, 07-12-2004, 08:58 AM
Not trying to gloat, but my last server came online at about 3:30am EST, which is right when I decided to give up for the night and go to bed. I feel sorry for anyone that's still affected. Not having an answer for your customers as far as what's going on or even an ETA just sucks. I feel that there's nothing worse. Hopefully this will be resolved soon for everyone.
Posted by Knogle, 07-12-2004, 08:59 AM
nate, get a new dedicated server provider, and prepare to move. you're evidently unhappy with the service, and unless you're willing to go through this ordeal again should something similar happen in the near future, you're better off moving.
Posted by Laci, 07-12-2004, 09:07 AM
Morning all, Are there still ppl down?
Posted by mdrussell, 07-12-2004, 09:11 AM
I sent in a reboot request 3 hours ago for a box that went down -- still nothing.
Posted by GreatDaneOwner, 07-12-2004, 09:17 AM
Based on the DN forums: lots. Mainly colo customers.
Posted by nate, 07-12-2004, 09:17 AM
I allready have servers in: DV2, thePlanet, Burst, Sagonet and finally here. It is not a question of just simply moving, as I know better then to put "all of my eggs in one basket" ... It is just that I need to know what is happening after 33 hours. Moving is the simple answer, but I have been with dednow since the day they opened their doors and there is something to be said about their support and quality of service up till now. I will give them the benefit of the doubt, but at the same time, I need this machine brought back up. I am sure you can understand. Last edited by nate; 07-12-2004 at 09:21 AM.
Posted by JenniH, 07-12-2004, 09:27 AM
I look forward to hearing their excuse for this mess... except there is no excuse. They can only really blame the NAC for the first few hours, as they posted 40 pages back when they had all the boxes on their premises. It is especially galling given the pre-move boasting about how long it had been in the planning. Quite apart from the shambles itself, there's the aspect of communication. This has been dire... no... that's putting it too kindly. Customers have been left high and dry with virtually no information: and most of the time the information forthcoming has been totally misleading (remember the message almost 24 hours ago giving the impression that they were dotting the i's and crossing the t's?). Information in a difficult situation is critical. It equips you to deal with a problem and manage it. Customers have been left devoid of service, and devoid of the means to manage it. Like it or not, this episode will tarnish their name forever. And they can hardly claim that it isn't deserved.
Posted by DangerMouse, 07-12-2004, 09:30 AM
They cant even blame NAC according to this... http://www.nac.net/statements/2004-07-11.asp
Posted by GreatDaneOwner, 07-12-2004, 09:32 AM
The post above by JenniH sums up the whole thing the best, if you ask me. If you don't want to read the whole thread, read just that one, and you'll get the main idea.
Posted by inteltechs, 07-12-2004, 09:35 AM
our servers are finally up.......thanks to whoever who brought them back.
Posted by NexDog, 07-12-2004, 09:38 AM
Coooorrrrrr, what a crock of sh!t! You didn't present the other side so I guess there was no other side. Check out the cheap shot. How professional! Ha! I can remember at least 2 instances last year where service was interrupted due to NAC incompetence (Fire and Power outage). Everyone knows that and that was never the point. Yea, I'm sure those people were just there to keep an eye on the Pegasus crew. Already done that. Glad to see the back of NAC and be with Pegasus Proper.
Posted by GreatDaneOwner, 07-12-2004, 09:43 AM
Are you sure that after all that has happened "Pegasus Proper" is the right phrase? Maybe "Pegasus P*** Us All Off" or "Pegasus Playpen", to make it milder.
Posted by mdrussell, 07-12-2004, 09:43 AM
Whilst I will reserve stating my opinion under the given circumstances, the fire actually was a fault of the elevators (iirc, it occurred in one of the elevators...), maintained by the building, so it was not NAC's fault...
Posted by NexDog, 07-12-2004, 09:45 AM
I thought NAC maintained the building.
Posted by NexDog, 07-12-2004, 09:48 AM
Fortunately most of our servers were online after 12 hours, another 4 came on after about 14 and the last 2 came online at the 18 hour mark. If I still had servers down, I'd be pissed off of course but they did alright by us so I'll stand by them.
Posted by MonaCK, 07-12-2004, 09:49 AM
33 hours, 49 minutes. *whistles*. *thanks God that she is not curing cancer with her server*
Posted by mdrussell, 07-12-2004, 09:52 AM
Nope, NAC and Pegasus lease (well Pegasus did) from the Octagon 10 Office Center...
Posted by GreatDaneOwner, 07-12-2004, 09:53 AM
I was relatively lucky given the circumstances, so it's not that. What bothered me deeply was their false information and non-communication. And I had had very bad service before the move too. But as I said, the main thing was that I trusted them, was loyal to them even when I had doubts and then they lied to me. I cannot forgive this.
Posted by akuo, 07-12-2004, 09:54 AM
12-18 hours for a promised 4-6 hour move is alright by you?
Posted by NexDog, 07-12-2004, 10:03 AM
I've always received the best service. So good in fact that I could hardly believe my luck in finding such a provider willing to work on such a personal level. I don't think I've ever had to wait more than 15 minutes for a TT response and they've done drive replacements 3 times quicker than EV1 and twice as quick as Rackspace. I can understand how they felt about posting during the nightmare. When we have a server go down, I dread going into the forum and posting updates. It really feels like you're "in the trenches". They have posted some updates and I agree that just showing a face helps things alot but the end result would be the same. I mean, it's not like they're all at the new data center admiring the view out the 3rd floor. I'm sure they are literally frantic and running around like blue-assed flies on acid trying to get all the servers online. Even if I had a server down, I wouldn't let 2 years of exemplary service go down the toilet. I'd be mad but deep down, I'd understand. I feel very sorry for those with just one or two servers and having them both down. That could really hurt business. In our case, we could absorb the blow if 100 people walked out the door. Everyone here must have been a customer of DN for quite a while and experience the superior service and support up til now. Just weather this storm and it will be worth it, I'm sure of it.
Posted by DangerMouse, 07-12-2004, 10:04 AM
Nexdog, You seem to have missed the one, most important point, of that whole statement... And that is that NAC wrote to PWeb on June 3rd informing them that their moving staff would need to be detailed, along with the equipment they were taking out... this information had still not been presented by July 10th, a few hours before the move was to commence.
Posted by NexDog, 07-12-2004, 10:05 AM
Yep, alright by me.
Posted by The3bl, 07-12-2004, 10:06 AM
Beats 34 hours.
Posted by mainarea, 07-12-2004, 10:10 AM
Are people still down now? - Matt
Posted by MonaCK, 07-12-2004, 10:10 AM
I would have been thrilled if it were only 12 - 18 hours. You almost expect that... but yeah -- like Techark - 34 + hours of downtime... really not very acceptable.
Posted by MonaCK, 07-12-2004, 10:12 AM
Yup.
Posted by nate, 07-12-2004, 10:13 AM
I still have a machine down at this time. no replied to emails, no picking up phones, no updates in forums ... anyone have any suggestions on this ?
Posted by The3bl, 07-12-2004, 10:17 AM
Hire the mob to go over and make them an offer they cannot refuse? Get my server on line or .... I wish I did have a suggestion but I am at my wits end. I have emailed. phoned tried IM eveything but a carrier pigion.
Posted by akuo, 07-12-2004, 10:18 AM
Very true. I was going to ammend my original post along those lines, but this is a fast moving thread Even if I were in NexDog's shoes with a 'mere' 18 hours max downtime, I'd be more than a little concerned that others served by the same company have been down 30 hours+. I'd be concerned that there have been major communication holes throughout the process. This whole thing just leaves a real bad taste in my mouth. I guess it just depends how much of a second chance you're willing to give people.
Posted by dside443, 07-12-2004, 10:22 AM
If you have a trouble ticket open, I'm sure they will get to it as soon as they can. Moving so many servers to another datacenter is not a small task and many unexpected problems have popped up and will likely continue to pop up for a while. It is important that they dedicate all of their resources in solving such problems rather than spending a lot of time comforting the customers. I understand that those of you with servers down are under a lot of pressure, but so is FortressITX. In this kind of situation, I think you'd rather have them working on urgent problems than hanging out by the phones or forums.
Posted by GreatDaneOwner, 07-12-2004, 10:35 AM
I think if you take the view that your servers are up, things are running OK, you miss the point: it could be your server that has been down for how long? 34 hours? You're just lucky, that's all, but there's no guarantee that the next problem will not hit you hard in the face.
Posted by The3bl, 07-12-2004, 10:37 AM
Sure I would but it does not take 5 minutes to have someone post a message every couple of hours to say we are all still here and working on it. If you are going to move 1500 servers at least hire a temp to sit around and say we have your ticket and you are number 999 in line.
Posted by RossH, 07-12-2004, 10:38 AM
Correct me if I'm wrong Laurnce but wasn't NAC one of the very few companies that actually had power during the East Coast/NY power outage? I remember EZZI and the rest were down, something has to be said for that.
Posted by Knogle, 07-12-2004, 10:49 AM
NAC Incompetence? That's probably why NAC decided to get their act right and sort out the entire moving process. Unfortunately, DNOW/PWEBTECH didn't respond, eventually leading to this messup. As one-sided as this may seem, NAC's statement and documents lead me to believe that PWEBTECH mishandled this entire process, right from the planning (failure to correspond with NAC, failure to notify customers in advanced, etc) to the execution (lack of respons to tickets submitted, lack of communication to customers, overshooting of projected timeframe by over 500% and counting, etc). I guess this is when PWEBTECH has to depend on consumer loyalty to keep the business running.
Posted by dside443, 07-12-2004, 10:52 AM
NAC's main datacenter did not go down because the electricity did not go out in that area (Parsippany, NJ). Their presences in New York and Newark did not go down because, IIRC, Telehouse and Equinix respectively were on backup generator power. EZZI runs its own datacenter AFAIK and its power did not have a working backup generator. Many, many companies in the East Coast had power during the blackout. The locations where NAC was alive on backup power were not owned by NAC but leased, so the example that you use is not a good one to describe NAC's reliability.
Posted by RossH, 07-12-2004, 10:56 AM
And thats why I said correct me if I'm wrong....
Posted by dside443, 07-12-2004, 10:59 AM
And that's what I did.
Posted by nate, 07-12-2004, 11:17 AM
i have to admit that all of this bantor is great and all ... but there are still those of us with machines down and have not heard anything at all. why can we not be updated or at least get a repsonse to our tickets. is my last machine totally busted or what ? i am getting very nervous.
Posted by RossH, 07-12-2004, 11:24 AM
Nate.....posting over and over isn't gonna help.... I'm truley sorry about your downtime, no customer deserves this much disrespect. I guess the true answer is your server will be up when they get around to it.
Posted by nate, 07-12-2004, 11:28 AM
I guess i just keep telling myself that maybe they will come here and post an update and note that i need help. nothing else seems to be working, so i was just trying to raise a red flag. what would you do ?
Posted by kickmybutt, 07-12-2004, 11:33 AM
Someone over at their FORTRESS forums post this.... too funny http://home-base.workisboring.com/news
Posted by RossH, 07-12-2004, 11:36 AM
Cry.......thats exactly what i'd do....
Posted by mctDarren, 07-12-2004, 11:42 AM
I don't care who you are, now that's funny...
Posted by nate, 07-12-2004, 11:52 AM
agreed !!
Posted by BiGWill, 07-12-2004, 11:53 AM
LOL that one's really funny! a small laugh in all this mess - hope everyone will be up very, very soon!
Posted by dswallow, 07-12-2004, 12:02 PM
Just to add to the info... My colo server with PWebTech that was in rack AM05 and had been updated with new IP's prior to the move is still down/inaccessible.
Posted by nate, 07-12-2004, 12:18 PM
my last machine is now online I am so excited I do not know what to do ! I think my new lucky number is 36 YEE HAW !! And to the rest of you that are still down, I sincerely hope that your machines come online soon ! I stuck with this thread through it's entirety, but i think I am done with it now ... Good luck everyone !
Posted by twidnet, 07-12-2004, 12:26 PM
DN loves people like you, blindly accepting their failures. Duh, DN already presented their side. Don't be an idiot. And DN has shown the epitome of professional behavior, right. The rest of us have noted repeatedly DN's constant shifting of blame, mostly to NAC. You're being dense. They are talking about the normal, everyday service they provide, not extraordinary events. Yeah, that would be why DN claims they only found out the morning of the move that they couldn't have 30 people wandering through the NOC when that is clearly a lie. That would be why DN blames NAC for hindering DN's ability to get this move done more quickly. Sure. Yeah, it's THEIR NOC. THEIR problem if DN damages someone else's equipment. I guess you'd just let DN do whatever they damn well pleased without any concern for the thousands of other clients NAC has to answer to. Like I said, bad providers love people like you, because people like you will excuse anything. Why don't you use your supposed exellent connections up there and find out why 36 HOURS LATER there are still a lot of people down and there are still NO UPDATES from DN about this fiasco. There's nothing "proper" about this at all.
Posted by amc-james, 07-12-2004, 12:34 PM
wow. smackdown=laid. gj twidnet
Posted by eclipsewebs, 07-12-2004, 12:53 PM
Well our server finally is back online. It was offline for exactly 36 hours 5 minutes and 15 seconds. I know this because the log shows at 12:04:06 the network cable was pulled off of the machine. It was then powered off, not shutdown. It next appears online at 12:09:21. At this point I am happy to be back online, but I still do not know the fall out from our clients yet.
Posted by JenniH, 07-12-2004, 01:04 PM
This NAC issue is simply a red herring as far as the overall outage is concerned. After about 8 hours Matthew McCormick of Pegasus stated "All equipment is now onsite " (see page 9 of this thread). So even in the most favorable scenario, the other 28+ hours are down exclusively to them.
Posted by whmcsguru, 07-12-2004, 01:08 PM
re : NAC v UCI Honestly, after seeing everything from UCI here in the past few months, and judging by the attitude shown in all things related here, I'd go with NAC on this one. However, the jury's still out, as there has been zero official ruling on the matter. NAC hasn't pulled anything like this on it's customers, although it COULD (could) have been them playing with the ips, it's most likely not them at all. NAC didn't force UCI out, UCI chose to leave on bad terms. NAC was extremely gracious in allowing UCI to keep service going even when well over $100k was owed. They should have gone to the courts sooner and forced the hand. It's clear that, as stated by the documents UCI was planning on running out and not paying the bills. NAC , in their history (though I could be wrong) hasn't EVER pulled something as monumentally stupid as what UCI has pulled in the past 6 months, from attempting to move customers without notification to theft of customer property (that's what it is people, you move colo boxen that aren't yours without explicit permission and TIMELY notification, and don't provide the box owners with a location and access to their materials, then it's theft). Nor, I'm quite sure WOULD NAC ever do something that monumentally stupid. UCI knew damn well that they'd need their own ip range some 2 years back (at minimum) if they were going to move out of the NAC house as it were. So, the excuse of "NAC is harming my business by refusing to allow me to carry ip's" is nothing but a crock. They should have started on this IMMEDIATELY once they knew the dc was under way. When it comes down to it, I'm honestly not sure I'd ever do business with either company, but I am 1000% sure I'd never do business with UCI. This whole fiasco and how they (mis)managed it has the making of one of the biggest FUBAR's that anyone has ever seen. Ignorance is indeed bliss. To ignore screwups THIS monumental means that you really just don't care. I feel for the customers of those involved in this, especially those of the ones blindly saying "I'll stay with UCI" after a screwup this big. Doesn't really say alot about your own reliability now.
Posted by DreamPages, 07-12-2004, 01:21 PM
Well ours is back up but an entire customer domain is missing. So much for "only moving" the server from one place to another. You guy might hear from customers missing domains.
Posted by DNCOO, 07-12-2004, 01:27 PM
All, Our staff has been working around the clock to get all servers back online. This has been a most difficult task, a move of this magnatude would not take place without some issues. We have had a lot of servers that were experiencing hardware related issues after the move. Everyone here is working at full speed to resolve all your tickets, fix any and all servers that are still down to hardware related failures and restore services as quickly as we can. For those of you who have experienced an extremely long period of downtime, it's more than likely that your is/was a server is in a hardware repair queue and is in line to be troubleshooted and repaired by a technician. When you have machines that run for months and months, some cases years with little interruption, when you shut them down for a period of time and you bring them back up, you have a good chance of experiencing hardware troubles. I know there is no bigger inconvenience than downtime, but rest assured that none of your requests are being ignored. We will continue our efforts and ensure all services are returned to full operation. Thank you,
Posted by upside, 07-12-2004, 01:33 PM
Ed, what is a working phone number for Pegasus?
Posted by whmcsguru, 07-12-2004, 01:34 PM
If the machines are shut down properly, and handled properly, this is untrue. I've moved countless pc's from one house to the next, powered them up and they ran like a charm. Servers are no different. Of course, again, this could have all been prevented with PROPER prior planning and moving things one wave (say a wave of 1-200) at a time, getting them setup THEN going back for more.
Posted by DreamPages, 07-12-2004, 01:35 PM
Actually a single move of this magnitude is ridiculous. It should have been broken down into smaller groups.
Posted by MonaCK, 07-12-2004, 01:45 PM
Have i mentioned lately? 37 hours, 45 minutes. Here's hoping that my hard drives didn't go.
Posted by thaphantom, 07-12-2004, 01:51 PM
/me cant wait for the TRO to be lifted and NAC to take back the IP addresses. I suggest anyone still with them to move as soon as possible, come Fri, you may have no IP addresses. DedNOW/Pegasus can say all they want that it wont happen, however a Juge can and most likely will remove the TRO, therefor causing Pegasus to lose all IP's
Posted by twidnet, 07-12-2004, 01:57 PM
Cry me a river. And things could have been done to mitigate those issues. You get an F in that area. Are you claiming that ALL servers that are still down are down due to hardware issues? That's what it looks like to me. And if that IS the case, why are the people with servers still down STILL in the dark about what's going on? You absolutely have to know at this point exactly what machines are down. You absolutely have no excuse for NOT knowing this, because people have been posting to the forums (http://clifton.fortressitx.com/forums/, remember, where you guys said you would post updates and you haven't) and sending you tickets and you should KNOW what servers are down and WHY. What about the colo people who have not heard anything? Are you saying all their equipment is having hardware issues too? Don't you think it would be wise to have some temp manning a helpdesk or a phone or something and have them relay whatever the techs say to the people who have been waiting for 38 HOURS at this point? That might play in Peoria, but we're not all that stupid. That's the lamest excuse yet in all of this. You've gone from blaming NAC to blaming hardware. What will be next, I wonder. Those requests ARE being ignored. The one thing you guys have shown mastery of in this is your ability to ignore your clients. Hoo-rah.
Posted by thedavid, 07-12-2004, 01:59 PM
This has not been my experience at all. Here's a particular story from when I worked for a large unnamed telecom that companies like DN use for transit - I have a bunch more stories, but this illustrates it best. We had acquired a company several years back that had several hundred unix and NT servers. Kinda similar to what you'd see with DN, except they had their own custom programmed control panel - hardware was kinda typical of what can be purchased through DN. These were shared hosting servers. Anyway, the datacenter had a fire in an unrelated section of the building. Firefighters showed up, and forced the power to be turned off until they could surmise the situation. Upon re-power-up, over 20 of those few hundred servers that had been running 24x7 for that long had hardware failures (mostly hard drives, SCSI ones at that). Power-up and also physical transport is *hard* on a server, especially if it's been doing 24x7x365 duties. Lots of hard drives were replaced that day, and the backup system got a good test. Your moving your 1 pc, that does not require a 24x7x365 operation cycle does not mimic the environment that a production server is in.
Posted by thaphantom, 07-12-2004, 02:01 PM
HOWEVER if the systems are shutdown properly, via shutdown command or ctrl alt del via console it should not cause any issues. If u have a power failure, or unplugged that is a diff case
Posted by thedavid, 07-12-2004, 02:03 PM
No, these were *hardware* failures I mentioned in my post. Nothing to do with corrutpion, fsck, all that (there were those too, I'm only mentioning the ones that required hardware replacement). It was 'hard drive won't spin' or 'hard drive is emitting a loud screeching noise, system won't boot' issues.
Posted by thaphantom, 07-12-2004, 02:10 PM
correct, however when u just turn off power to a system instead of shutting it down the heads that read/write to the D have a much higher chance of bouncing against the platters, thus causing HD failures. If u shut it off correctly then it can still happen, however the chances of it happening are alot more slim.
Posted by whmcsguru, 07-12-2004, 02:17 PM
even the 3 finger salute (ctrl-alt-del) isn't the best way to shut a server down, although it's certainly better than just yanking the power. I've never had an issue with servers that were shut down properly. I HAVE had an issue with moving servers or pc's that were NOT however. If the thing isn't shut down properly, of course, it's got to go through the typical checks upon coming back up, take longer, and yes, it might just damage hardware. That's a known issue and problem. The lesson? DON'T shut servers down inappropriately. Lesson 2, especially good in this case? When moving servers from point a to point b, don't do it all at once. Do it bit by bit, then you can be sure that the servers are not problematic when starting up.
Posted by net-trend, 07-12-2004, 02:19 PM
I don';t understand your post. First you said it was not true what wolfstream said, then later prove that what he said was true? Powering up disk drives is the most stressful thing that can happen. True. However, shutting down a server/pc improperly will mean that the hard drive heads are mos likely to be unparked. Current hard drive tech. allows for several Gs of force to be applied to it but only when the heads are parked properly.
Posted by whmcsguru, 07-12-2004, 02:23 PM
Just a random thought. Do you REALLY think they consoled some 1500-3000 servers (numbers really unknown) and rebooted them properly? Well, if you do, think again. I'm sure they did that to a few key servers (those of their bigger clients), but not to everyone.
Posted by thaphantom, 07-12-2004, 02:27 PM
oh I agree, ctrl-alt-del isnt the best, but it beats the hell outta unplugging em
Posted by bhaputi, 07-12-2004, 02:27 PM
I do not think the point being made has to do with what they DID, rather what they SHOULD have done. It does not take that long to drag a monitor on a cart around, hit ctl+alt+del, then pull the plug. It certainly takes les time to do that than it does to fix the result of NOT doing it.....
Posted by thedavid, 07-12-2004, 02:28 PM
What I'm saying is this: "Powering up disk drives is the most stressful thing that can happen. True." And, given the amount of servers there, plus the predominant technology being eide at dedicatednow (which is *not* known for its long MTBF) - there are going to be some hardware failures. Period. Even with proper shutdowns, or chants to the server gods. Saying that it's not possible to happen because you can lan-game (or whatever) with your PC is not really accurate.
Posted by thedavid, 07-12-2004, 02:29 PM
Did I say that? <-- looks up Nope.
Posted by dswallow, 07-12-2004, 02:33 PM
Have you bought a hard drive lately? Like in the last 15 years? Modern hard drives automatically park the read/write heads when power is removed since the voice coil retracts the arm. There might be data corruption issues if a system isn't shut down properly, if data were being written at the time power was removed, but it should be rather rare such problems would result in a computer not being able to boot up successfully.
Posted by thaphantom, 07-12-2004, 02:37 PM
that is true, however on an unplug they can still bounce on the way back.
Posted by iFlash, 07-12-2004, 02:44 PM
At this point, people should not be wondering WHY their server is still down. Or having to guess if it is down due to a hardware issue. Pegasus should know EXACTLY which servers are down, why and who they belong to. Saying if your server is still down it is most likely due to a hardware issue is pathetic. What if their server is down simply because it needs a manual FSCK and nobody has bothered to do that yet? But I guess it would take a professional company that had planned this out properly to be able to accomplish something as simple as identifying which servers are down and why and then notify those customers about the status of their servers.
Posted by Layershift Andrew, 07-12-2004, 02:59 PM
I left one of my development servers up intentionally (shutdown the rest 5 mins before scheduled to go offline) to see if they did actually shutdown. And they did. Jul 11 07:37:22 saturn shutdown: shutting down for system reboot Jul 11 07:37:22 saturn init: Switching to runlevel: 6 Jul 11 07:37:23 saturn shutdown: shutting down for system reboot Jul 11 07:37:24 saturn atd: atd shutdown succeeded Jul 11 07:37:24 saturn rc: Stopping keytable: succeeded Jul 11 07:37:24 saturn mysqld: Stopping MySQL: succeeded Jul 11 07:37:29 saturn httpd: httpd shutdown succeeded Jul 11 07:37:30 saturn psa: Stopping Plesk: succeeded Jul 11 07:37:41 saturn kernel: Kernel logging (proc) stopped. Jul 11 07:37:41 saturn kernel: Kernel log daemon terminating. Jul 11 07:37:42 saturn syslog: klogd shutdown succeeded Jul 11 07:37:42 saturn exiting on signal 15 Jul 11 16:14:55 saturn syslogd 1.4.1: restart. ... wasn't powered down until the shutdown completed. Last edited by Layershift Andrew; 07-12-2004 at 03:02 PM.
Posted by thaphantom, 07-12-2004, 03:02 PM
Cranky, Thats great for you, however I have heard a number of other people werent so lucky :-/
Posted by The3bl, 07-12-2004, 03:06 PM
All I know is after 48 hours of very little sleep, getting my butt chewed on a regular basis and having nothng to tell those chewing on my behind, I feel like I am in the twiight zone. I do not really care anymore how you are supposed to shut down a hard drive or the finer points of head bounces. I do not even care at this point who's fault all this is. What I do care is that they get the 2 servers that are down back online so that my customers can stop suffering. I will do a post mortem on this whole thing after my clients stop suffering. HELLO is there anybody out there? Some one form Pweb go look at my servers. <<>>
Posted by MonaCK, 07-12-2004, 03:14 PM
39 hours, 15 minutes.
Posted by The3bl, 07-12-2004, 03:26 PM
One server is back online.... One dedicated customers machine left to go. Now if I can get it up I can get some sleep finally.
Posted by thaphantom, 07-12-2004, 03:28 PM
thats what he said to her.... oh wait sorry
Posted by maxhest, 07-12-2004, 03:41 PM
Hmmm, Reading all 45 pages of this is like pulling teeth.. I hope this gets resolved soon - man I wouldn't want to be Fortress/PWTech/DedNow.. Good luck guys, hope you get everything working asap. -Max
Posted by The3bl, 07-12-2004, 03:42 PM
Uhh maybe I need to rephrase that..
Posted by BeDifferentSolutions, 07-12-2004, 04:02 PM
I think this statement says it all. This is the first thing I thought of when I was reading this. Second thing is do you think all 3000 servers were 'handled with care'? Or where there so many, they were thrown into vans and driven to the next stop? I certainly am just speculating. But logic dictates...... Last edited by BeDifferentSolutions; 07-12-2004 at 04:07 PM.
Posted by thaphantom, 07-12-2004, 04:06 PM
yep I agree completely... so sad for everyone using them...
Posted by boldfish, 07-12-2004, 04:32 PM
Well I got to the end of the thread, and the end of my tether. We're still down. I'm just a little fish in a huge pond and i'm reselling space on a server that is shared with other resellers and it looks like it may never come back up. I can't get any information from the guy I pay because he can't get any information from FITX So, what do you do? wait and see or start looking for hosting for my 20 or so extremely disgruntled clients. if anyone cares please get my server back up (64.21.169.194) maybe if we all clap very hard tinkerbell will wave a magic wand?
Posted by thaphantom, 07-12-2004, 04:36 PM
everyone may wanna look at this: http://www.webhostingtalk.com/showth...hreadid=295675
Posted by twidnet, 07-12-2004, 04:48 PM
I see that WHT has How-To areas. I would like to nominate a new area. The How NOT To Forum The first post should be "How NOT to move clients to a new NOC". The examples for that post can be drawn from everything DN has done in this move, beginning with their aborted attempt to sneak everyone out in May.
Posted by boldfish, 07-12-2004, 05:14 PM
Hey clapping must have worked... my server is back. now where should I take my twenty customers so it doesn't happen again I wonder.
Posted by MonaCK, 07-12-2004, 05:21 PM
*grabs magic wand* You think it'llw ork for me? I still think my box is the last one in the corner...
Posted by boldfish, 07-12-2004, 06:49 PM
That's gotta be a load of bollocks. we were told about the move on Saturday evening. no time to prepare or back up servers. gotta be an inside job. stuff it. I'm moving before they shaft me again.
Posted by NexDog, 07-12-2004, 06:57 PM
Aw, come on, tell me how you really feel. You left out my mother - how rude.
Posted by NexDog, 07-12-2004, 06:59 PM
Heh, now you just showed your age. Don't forget that banana on your bib.
Posted by NexDog, 07-12-2004, 07:04 PM
Man, you have got some serious problems. Instead of appreciating what someone has taken the time to write, you rip it down and out your own twisted theme on it. Shame on you.
Posted by mrzippy, 07-12-2004, 07:10 PM
Is this the final excuse then? How nice that they "allowed" you to publicly state, on their behalf, why it wasn't all their fault after all! Please pass along our profound thanks to the "upper management". They are blaming THEIR OWN contracted IT firm? Uh huh. So it wasn't NAC's "fault" after all for limiting access to the colo area? (As previously stated in posts from pwebtech.) It was the mysterious IT firm's fault? And nobody from pwebtech thought it appropriate to maybe pass on this information themselves? They prefer it be released by one of their customers? Blah. Who schedules a move using contracted IT support staff that is so unrealiable that NONE of them show up for work? Maybe they didn't get paid? With the various payment issues now publicly posted and verified by various courts, I wonder if this contracted IT company perhaps had a contract that said "no payment no work".
Posted by twidnet, 07-12-2004, 07:13 PM
Know what my problem is? That DN COULD NOT BE BOTHERED TO POST THIS THEMSELVES. Know what my other problem is? Over 43 HOURS of downtime. Know what your problem is? You're too willing to take a bullet for DN. Their own personal human shield of suckers who blindly accept the crap they spew out.
Posted by iFlash, 07-12-2004, 07:14 PM
I think what Marty and NexDog are missing is that there is NO excuse for not letting people know what is going on. There is no excuse for DNOW not properly planning for this. Proper planning would include having someone dedicated to posting updates and ensuring that there are enough people to answer tickets. It is inexcusable that people still have no idea of if or when their servers will be up. It is inexcusable that at this point people don't even know if their servers have a hardware problem or not. There is no reason that this had to be the disaster that it is. The blame and in this case, there should be blame placed on some, falls squarely on DNOW. They are working feverishly because they are desperate and trying to salvage what is left of their company. Not because they are so committed to their customers. Don't buy that for a minute. If they were so committed they would have planned this out better.
Posted by thaphantom, 07-12-2004, 07:15 PM
Wait so when they relized all the issues, WHY didnt they postpone it? Oh wait, thats right they burned their bridges with NAC, so who suffers... the end uses. Great one DedNow... 2 big thumbs up
Posted by Laci, 07-12-2004, 07:29 PM
This whole experience reminded me of when the stock market crashed and the great depression started , people were..... quiet , mad , crying , going out of business , begging , pleading , bargaining etc . How could they see the effect of their actions and not bother to give a decent response . I know I know too busy getting the servers up , but it leaves me to wonder how under staffed is DedNow that no one can take a moment to give an honest explanation , it leaves one to wonder ...do I want to be in a data center that is so under staffed they don't spare the man power to post.................
Posted by iFlash, 07-12-2004, 07:39 PM
And what is the name of the IT firm that DNOW contracted? Surely they wouldn't mind if one or two of us verified this, would they?
Posted by MonaCK, 07-12-2004, 07:42 PM
Meanwhile, 43 hours, 42 minutes, and just now they're trying to get my servers back up. Trying is the operative word. I have no faith in anything anymore, and my patience is completely gone.
Posted by akuo, 07-12-2004, 08:21 PM
What a shame that excuse has a massive hole in it. Presuming they DID only had a couple of additional people there to move the servers, why the hell did they a) still go ahead with the full move and b) not tell anybody to expect delays EARLY in the piece.
Posted by whmcsguru, 07-12-2004, 08:21 PM
So, let's recap here, some 40+ hours later shall we? A> DN still has servers down, yet has no friggin clue what ones? Professional much? B> DN has tried blaming everyone from NAC (yeah right) to their IT outsource group (mmm hmmmmmm) for their rather astronomical screwup. C> DN has refused to answer support tickets and get servers up in a timely fashion (40+ hours?) D> DN completely and totally mismanaged the entire move, including taking servers FROM NAC that were colo'd there without informing individuals where their new home would be, following specific instructions to the new individuals, or basically showing an ounce of respect for their customers? E> DN has repeatedly failed to update customers publicly with what is going on here? F> DN Has attempted in the past to move servers without any notification at all? I just got one question for anyone even considering remaining with this unprofessional organization. WHY??? Come friday you may not have your ip addresses, and what will DN's excuse be? "Oh, we're sorry, we didn't know". Well, common sense would have dictated that they needed new ones, and the documentation (legal mind you) shows that they DID know. Like I said a couple of pages back, ignorance is bliss. I've got this bridge over a river in the middle of the Sahara I'll sell you for a pretty penny, options going quick!!!
Posted by Joshua, 07-12-2004, 08:39 PM
I am still stunned about how long the downtime has been for certain users. United Colocation flew servers from CA to Tampa, FL, and SagoNet was able to hook up all 700 servers and have them all online within 12 hours of the beginning of the move. Maybe they should've consulted Sagonet before the move. I know that Matt McCormick has a contact down at Sago (a brother-in-law of his, actually), so I'm surprised no tips were passed .
Posted by thaphantom, 07-12-2004, 08:49 PM
12hrs? now that is damn impressive
Posted by porcupine, 07-12-2004, 09:18 PM
The better question DN customers should be asking themselves is this: I haven't read all the articles in the legal papers, but I did run across the one that named the owner of NAC to securities in the form of the DN hardware (doubtlessly servers and routers/switches). That being said, if DN doesen't pay their bills (didn't read if they had or not), to which it sounds like they are huge bills, will this not give the officer of NAC the ability to have their equipment reposessed? That could be an utter nightmare to say the least..... Granted it'd make it damn easy for NAC to take their customers from right under their noses, literally.
Posted by thaphantom, 07-12-2004, 09:19 PM
they already paid the bill.
Posted by twidnet, 07-12-2004, 09:21 PM
They paid the 93K bill listed in the documents NAC posted?
Posted by thaphantom, 07-12-2004, 09:24 PM
yea, otherwise NAC wasnt going to release the equipment that was released sat night/sun morning
Posted by porcupine, 07-12-2004, 09:32 PM
Thats just presuming they did. There are always ways around it. Judging by their bank sheets, it seems unlikely they pulled it out of their butts to say the least.
Posted by Steven, 07-12-2004, 09:46 PM
somebodys been reading wht rules
Posted by iFlash, 07-12-2004, 09:50 PM
I was just reading DNOW's public annoucement of the move found here - http://www.nac.net/statements/statem...nouncement.pdf And this struck me as odd. Why would they not also mention the outsourced IT firm that specializes in this sort of thing? That seems like something that would be relevant information and something you would want your customers to know? You mention the movers, your in-house team, but not the IT firm? Last edited by iFlash; 07-12-2004 at 09:56 PM.
Posted by kickmybutt, 07-12-2004, 09:50 PM
Gump, where is my $100 ? I take PayPal or CC's via 2CO. CLICK HERE TO READ THE ENTIRE POST Gump, where is my $100 ? I take PayPal or CC's via 2CO. Or will you chicken feet out of this somehow???
Posted by dswallow, 07-12-2004, 09:57 PM
These are extraordinary circumstances for any colo/dedicated server provider. Did it go smoothly? No. Do we know the whole story of what led to the break between Pegasus and NAC? Probably not. About all we'll know is that each has issues with the other. I've had my server located with Pegasus for just over 3 years. And during that period, have had nothing but good things to say about them, until this move. So 2 days of downtime due to a major data center move that didn't go smoothly out of 1,100 days of service is a blip. I don't see myself moving the server to a different provider. If anything, I will finally locate a second server at a second provider to give myself redundancy and a real/true live backup. As many have said, you never put all your eggs in one basket, for just this very type of reason. I've resisted the second server idea for some time, simply because I don't need the capacity that's going to give me; I need it only for the redundancy. And frankly, I don't really need the redundancy since I have access to enough capacity through other means that I could put my sites back up anywhere within a day if I felt the need to do so. But it'd be sorta nice to maintain them in a way I literally could adjust DNS and essentially "flip a switch."
Posted by The3bl, 07-12-2004, 09:57 PM
)*&)(&(**(&$#*&@#@#$&)@#&$ @# One dedicated server customer is still down and he is about to lose all his clients and no response from anyone. Man I feel horrible.
Posted by twidnet, 07-12-2004, 10:04 PM
I know the mods are monitoring this thread just as closely as they monitor others, and I want to thank them for letting things take their course and for indulging those of us with a lot to say about this. And with that, I'm going to say even more. Those of you who don't like what I have to say can just skip right by while I dissect what Ed of DN wrote in (at last!) an update to the forums they had set up. First step in a psychological ploy. Empathize. Sorry, won't work here, Ed. Too long between updates, and dishonesty at that. So it has taken between July 11 11:42 AM and July 12 8:59 PM to move from a claim of 80-85% of servers up (posted by Matt) to 99.5% of servers online. That's almost 36 hours. It's also very hard to believe. I think the post by Matt was just another miscalculation on DN's part, or there were more issues than just "hardware failures". Both, maybe. At least one of your colo people never received any notices. One would think you would take even more special care with them, considering that it's not your equipment. You and the staff are paid to assist your customers. This doesn't excuse your total lack of communication. This message I'm quoting actually appears to be the ONLY update posted to the forums that DN themselves said in the reminder right here would be updated. I think everyone can believe the migration was much, much more than you could handle. You've ably demonstrated that above everything else. One would think this is exactly the sort of thing a company would mention right up front, if it is even true. It would even lend credibility that more planning was done than is quite obvious ever occurred. No comment about your claim that NAC sprung on you that only 8 people would be allowed in the NOC when it's clearly spelled out in the notices you got from them that only 5 would be allowed in? Guess you have to cross that one off your "blame NAC" list, huh. You were notified previously that 5 people would be allowed in. What you "personally felt" and what NAC rules state are two very different things. You knew they weren't going to allow all those people in. You lied to everyone when you said this was some last minute verbal thing. At least one person says this is not true and that his machine shows it was just powered off instead of shutdown correctly. I'd like to dig up that colo guy that doesn't provide you with root passwords and ask him what his logs show. What's the name of this mysterious company? And why wasn't this communicated at the same time you claimed NAC suddenly wouldn't allow a bunch of people in the NOC? Pity parties aren't going to happen. It's YOUR JOB. In a major metropolitan area like the NJ/NY area, you're saying there are no emergency labor providers to draw from. I've been to smaller towns that have companies whose job it is to do this sort of thing. I know a moving company who can be on a site in an hour with 25 people if needed. But not quite caring enough to post updates like you said you would. Not quite caring enough to give more notice than four days. Not quite caring enough not to try and move with no notice to most people back in May. If that's what you consider caring about your customers, you haven't learned a damn thing. No. DN made the decision to move. The reasons are clearly outlined in the legal papers DN themselves filed. And by late Sunday, you couldn't get someone to temporarily man the phones? Monday morning? ANy temp agency could have sent over someone to take calls an pass messages. The better question is: should we have planned at all? The answer to that is definitely yes. It seems like DN is just awash in bad decisions. Or did someone else pick this supposed company at random out of the blue? Temps. Temps. Anyone can type a message when instructed what to say. Relay information. You know, information? What your entire business relies on via your customers? He says "upper management". So this "upper management" can talk to Marty, who is apparently buying everything you guys say, but can't just type up a quick response, taking a couple of minutes, to post an update on the forums? That makes perfect sense in the DN world. It makes no sense in the real world. It is YOUR JOB to work hard. What is frightening about all of this is that by a very far margin past your own estimate, it was clear that DN had no idea what machines exactly were still downed, was not responding to tickets, and had no one to answer the phone, even to take messages. That's ridiculous. Then why are people still complaining about downed servers and no responses from DN if you're down to .5% of the servers left in limbo? You're darned right you're going to lose customers. Anyone willing to stand by this pile of bs is not anyone I'd trust either. We shouldn't have to "understand" your situation. Had you learned anything about communication, or aborted the move when this "IT firm" YOU chose didn't show up, we wouldn't have to be having this discussion. You should be apologizing for your total lack of communication at critical junctures during this fiasco. It is inexcusable.
Posted by thedavid, 07-12-2004, 10:07 PM
Your dedicated server guy is *still* down? With your tickets you've put in, did you get a auto-ack ticket number response? I've put in 2 so far, 1 for the server being down for 16 hours (at that point, to see what the status was - came back up after 20 hours). That ticket got closed, but I did see that someone at the console logged into the server post-powerup. So the techy folks are checking things. The other is a ticket to rona in billing - who is answering at her normal pace, and is as always helpful to work with. Here's what I would've liked to have seen, had I planned it: A webpage (not a forum, those are too 'noisy' for this) with a graphical/textual display of all of the racks/breadboards that pegasus had. Then, move one rack at a time, updating that page with details on that racks movement. Once it's up 100%, move the next one and mark off the first as complete. Yes, this would've been more work. But I feel that it would have fixed one of everyones main concerns - 'where is my server, and why isn't it back online?' as well as the 'how come they're not responding on the forums?' Just my opinion.
Posted by iFlash, 07-12-2004, 10:07 PM
Well, that's an understatement! LOL Yeah, we do. It's in those court documents.
Posted by Samuel, 07-12-2004, 10:14 PM
Certainly its not consoling now, but I think if you stick with DNow, you'll end up with a much better experience. Its going to be rocky at first (As it is now) but they are moving into a new spread, new shiny objects everywhere for people to OOO and owww over =) You'll be singing thier praises when this dies down, seen it before, and its only a matter of time before the threads start. It will start with "You know, I kinda actually love this new DC" "Yes, it was hell previously, but now WOW! its a shiny thing, right here!"
Posted by jnestor, 07-12-2004, 10:16 PM
.5% of 1500 servers = 7.5 (lets say 8) servers that are still down. I have 2 down personally. Why do I not believe that I (who until now haven't posted in this thread) am unlucky enough to have 25% of the servers still down? Admittedly the three servers I have are supposed to be cross connected (via a switch) and they claim that servers that are still down are awaiting vlan configuration. Perhaps there really are so few servers down but if it's really only 8 or so servers you'd think someone would have a minute to send me an e-mail saying "hey we're working on it now". I've been with PWebTech for years and have generally been happy with their service but given the sorry state of communication (and not just with this fiasco) and the number of other things hanging over them I'm going to have to seriously consider moving. - my primary server is on a NAC IP which the courts can take away at any minute - the horrid situation will certainly cost pegasus customers and could put them in a precarious financial situation - the ongoing legal battles could put them in a further bad business state. I'm not a web host so I don't hang out here or really think about my web provider often. I run 2 websites that run across 2 dedicated servers. My losses from 48 hours downtime will exceed a months hosting bill. I really feel for those that are going to lose customers and even more money over this. Poor show Pegasus.
Posted by iFlash, 07-12-2004, 10:23 PM
How insulting. And based on how they handled a planned move how well do you think they'll fare during a true emergency? (Without NAC to blame)
Posted by The3bl, 07-12-2004, 10:24 PM
Yep still one server down I have submited tickets, I emailed support, Matt, Jay, Ed, I have sent IM's I have called the busy all the time numbers. I have replied to the last ticket of mine they answered when they brought another server up and in it they promised to keep me up to date from that point on and if i had any more problems to write back on this ticket etc etc... Yet no comunication form anyone on this last server down. Now this guy is losing his customers right and left, he has spent over a year buidling his business to get to the point of having his own server and now this. It is a disaster and I feel horrible. I have offered him another server but without his customers files he is sunk.
Posted by twidnet, 07-12-2004, 10:27 PM
Are you on crack? Give me any good reason to trust DN. It may be a new NOC, but it's the same people, and those people are not exactly confidence-inspiring.
Posted by Joshua, 07-12-2004, 10:29 PM
That's what NAC was hoping for...
Posted by thedavid, 07-12-2004, 10:34 PM
Understood - it's got to be real discouraging for him, seeing all of his hard work go up in smoke due to the actions of those 'upstream' (pegasus) that he's put his trust in. I hope his customers are as resilliant as ours (that were affected) proved to be, though after this timeframe I don't know how much more can be reasonably expected. I feared a scenario like above, and did the moves as best I could before the server was unplugged. Luckily our other 8 servers are elsewhere, otherwise that wouldn't have been possible and it could have possibly been me posting the quoted info above. I feel for you. Hope it comes back to life soon...
Posted by mrzippy, 07-12-2004, 10:34 PM
I wonder if they're considering selling that shiny new noc to pay for their bills.. and upcoming legal fees? I know 1500 servers seems like a lot, but let's say they lose 10% of their business as a result of this. That's down to 1350 server remaining. They appear to owe a truckload of money, aren't taking new signups, and have problems up the yingyang due to impending IP renumbering issues, etc... It would be fascinating to see their balance sheet. I also wonder where they get the majority of their clients from.. if it is from here, then they're going to be digging out of that hole for a while. Enough people were poorly affected by this debacle that it'll be long while before DN gets a decent review again. Sure the dc might be "shiny and new", but who in their right mind would trust a company that has such obvious disrespect for their customers? There's plenty of other "shiny and relatively new" datacenters out there... I wonder what the bookies would place odds on pwebtech/fortress going under in a few months? With new management, I think the new dc would be pretty sweet.
Posted by Samuel, 07-12-2004, 10:35 PM
Why are the insults flying at me? Are your comments any more valid than mine? I've seen this before, the way you're bandying your comments so loosely leads me to believe that what I have seen before, will happen again. I do not use illegal drugs of any sort and while it can be taken as an offensive comment towards me, I don't take it to be as I know you're just shooting your mouth off twidnet. I never stated I asked you to trust them, but eventually you will be with them, It is a fact that the most vocal, tend to want to stay more than those are not. So for everyone repeating the same thing over and over, they got the message, you want better service. Its the silent ones that they fear will leave =)
Posted by whmcsguru, 07-12-2004, 10:46 PM
Long story short. Bills, bills, bills and more bills went unpaid (read the legal docs for more). Of course, this may have been primarily related to price hikes placed by NAC, but UCI knew long ago about all that, and only NOW is doing something? I'd say "poorly planned move" is the understatement of the decade, really. And who among us will actually SEE the new dc ehh? Not I, and I'm sure that (maybe) 1% of the individuals hosting through them will see it. New and shiny isn't impressive in this industry. Communication is, and UCI has failed that , miserably. Quite honestly, I'm surprised those staying have been so understanding. UCI has all of the signs of a business going out of business, bankrupt as it were. Not paying the bills, having extremely LARGE bills ($100k+), bank accounts overdrawn frequently. Top that off with no new revenue (DN halted sales over a year ago), and you have a disaster waiting to happen. Again, very poorly planned. A year ago, DN was great, so great that they happened to land one of the larger clients as far as webhosting is concerned, CPanel. Now, look at what it has become, a literal mess of negative ballances, back bills and the like. Doubt me? All you have to do is read the documentation made public by BOTH parties. I would strongly advise any clients even thinking about staying there to reconsider this position. If (when) they go belly up, your business is going to take yet another hit when whatever corporation comes in and cleans up the mess.
Posted by The3bl, 07-12-2004, 10:50 PM
Ahh last customers server is back up... I can now take a break.. I hope eveyone gets online soon and can put this all behind. Monte
Posted by Samuel, 07-12-2004, 10:57 PM
Wolfstream, you know its going to happen =) You dont have to see physically a shiny thing to "have" a shiny new thing. Its how it will start, wow, its great, I forgive them for handling that badly, I mean they did have this and that to contend with, they actually aren't half bad. Then, I LUV dem, they are so fast etc. We choose datacenters and server management companies based on decades of experience, not 5 ot 6 or 10. Should one say "you get what you pay for?" not sure, but it sure comes to mind.
Posted by twidnet, 07-12-2004, 10:59 PM
It's a phrase, not some sort of comment that you use drugs. And you're wrong anyway. I'm not with them any more. See, I'm not like the suckers who are willing to turn a blind eye to what DN did here. I see plenty of other vocal people moving on, too. I'm asking you why on earth you think anyone should trust them. Even one reason would do. The hardest thing in the world to do is build trust. And when you tarnish that, it takes more effort than DN is willing to expend to rebuild it. The fact they don't seem to have any financial management skills either is just icing on the cake. You know what is the sorriest thing about all of this? Not once has anyone from DN taken any responsbility at all for any of this. It was NAC. It was this supposed IT firm. It's always someone else's fault. We could all be just like them. Lie to our customers (when we bother to give them updates at all). Always claim it's the other guy's fault. Always promise more than we can actually deliver. That may work for you, but it is not the way we do business, and we won't do business with anyone who acts like that either.
Posted by Samuel, 07-12-2004, 11:01 PM
Meaning isnt in things, its between things. They know they have boffed this up, its pretty much a given.. haha. I still haven't asked you to trust them, you wil though, you're so compassionate about it. If you're gone, then move on (As they say) Do I use them as a provider? : NO Would I use them as a provider?: Maybe, depending on what level come up to, right now too bargainish for my customers Have I been in this type of situation before? Yes Was pegasus involved? Yes Do I trust them? Nope, too many mistatements and incorrent assessments over their current situation. lack of updates, and responsiblity towards their custoemrs. (Sorry for typos, can hardly see the screen)
Posted by iFlash, 07-12-2004, 11:05 PM
When you give someone money to provide a service to you there is implied trust. You are saying that people should stay with them and continue to pay them and things will get better. Again implying trust - trusting in them that they are not going to screw up this badly again.
Posted by NexDog, 07-12-2004, 11:06 PM
I know you must be severely angry but I see no need to continually insulting people. Samuel, Marty or myself do not work for Pegasus so kindly direct your annoyance elsewhere.
Posted by Samuel, 07-12-2004, 11:10 PM
Understandable, but I wasn't talking about trust, I was referring to how people look at providers. If they trust them, its dynamic. Will a bad thing stop them from trusting them? Nope, its a fact. If ford releases a vehicle with a KNOWN defect, will people not trust ford? Nope, they will trust em. If Dnow/Pegasus let you down, will you trust them in the future? Yep, you sure will. You've chosen that level of provider, in the future unless you change your target, you will find a paralel provider and make the same mistake "trusting them". take some responsibility yourself, and trust your instincts about a situation and move when its under your control, not theirs.
Posted by twidnet, 07-12-2004, 11:11 PM
Give it up, Laurence. If you're so sensitive, report my posts to a mod, or skip them. Ever hear of an idiomatic phrase? Sheesh.
Posted by Samuel, 07-12-2004, 11:13 PM
Asking me if I am on crack, on a very public forum only shows your contempt for all that are around you. Stating its idiomatic does not mitiate its bad timing and improper use. Please refrain from asking if I am on crack, its a waste of mine and your time as you are sure I am not on crack. I, my loose commented WHT user eat fruit.
Posted by NexDog, 07-12-2004, 11:13 PM
I don't believe this at all! All I see are signs of a company regrouping for the future. It's something we did when we left Rackspace. There enormous prices and piss poor management almost put us out of business and Pegasus swooped in and saved us in more ways than one. Hey, they just built a new DC. Is that the sign of a company going out of business? By moving, they are saving themselves a truckload of cash due to NAC's ridiculous price hikes. Of course they have new revenue. We have got at least 7 servers since they stopped new sales and I'm sure other existing customers (like BeachComber) have bought alot more. Also, FortressITX will open to new business very soon and bring in lots of revenue.
Posted by iFlash, 07-12-2004, 11:14 PM
It's not that they let you down. It is how they handled their screw up that determines if I trust them or not. Based on how DNOW handle this and how they are not taking responsibility, no, personally I would never trust them again. But that is something each person let down by them has to answer. I would certainly not come on this forum though and tell them, don't worry, I know they really screwed up big this time but look at all of the shiny new things. Just stay and things will get better. How do you know this? What are you basing it on? And why should anyone trust things are going to get better?
Posted by Samuel, 07-12-2004, 11:16 PM
I started programming in 1981, starting hosting web sites in 1995 commercially. I've seen it before, and I'll see it again. This is minor.
Posted by twidnet, 07-12-2004, 11:18 PM
People really can be dense. By the way, where did this fantasy come up that they "built" this DC? They are obviously leasing space in a building someone else owns, just like a lot of other people with their "own NOCs". They do not own the building.
Posted by iFlash, 07-12-2004, 11:18 PM
We are talking about DNOW and this situation. Not some company you had a bad experience with years ago. You are the one telling everyone to stay with DNOW and things will be better. And I am asking you to tell me why?
Posted by twidnet, 07-12-2004, 11:21 PM
And how about an answer to a simple question, DN? Who is this "IT firm" you chose and who failed to show? Maybe we should ask NAC what other people besides your own people showed up July 11 at midnight, huh?
Posted by mrzippy, 07-12-2004, 11:23 PM
I don't think Samuel every said to stay with dnow or that things will get better. He basically said that people will forget, etc....
Posted by Samuel, 07-12-2004, 11:23 PM
I am not telling anyone to stay with them. I know you're trying to lead me down that path but it is not the case. Tell you what? What you want to hear? I've already done that, its just my timing that is confusing you EDIT: I am referring to the same company, in the last few years.
Posted by dswallow, 07-12-2004, 11:23 PM
You build trust over time; and that foundation doesn't crumble with one mistake. In the scheme of dirt-cheap hosting, is 2 days downtime because of a data center relocation something people won't understand? In the scheme of mission-critical guaranteed-uptime, redundant servers, it probbaly would be a significant blow. But to others, probably not so much. And sure, there are people who might be running such mission-critical applications at dirt-cheap hosting locations, but that doesn't mean they had, given their configuration of servers, any right to be doing so and be considered truly mission-critical/redundant-capable. DedicatedNow couldn't sell additional servers and respond to the demand for them because they didn't have the space (or electrical service) in their old location to do so. That would create a precarious financial situation when you're selling dirt-cheap dedicated/colo services, since you depend upon the law of averages to pay for things, you need to be able to take on new customers to increase your revenue stream. It's the basis of your business model. It wouldn't work to provide such connectivity for 20 customers; you need thousands. If you can provide your services to 3,000 clients for $X and to 3,400 clients for the same $X, you'll get more revenue having 3,400 clients. If you're unable to have more than 3,000 clients, you're losing revenue you could have had. The real question that might have more relevance to a decision to change providers might be one that looks to see if they've learned anything from the mistakes made and if they take steps so that in the future they handle things better.
Posted by whmcsguru, 07-12-2004, 11:29 PM
You don't see what? Have you READ all documentation by both sides? DN currently can't (or won't) pay their bills to NAC. They had a $100k+ bill that was owed, and for whatever reason chose to not pay it. This isn't indication of a "booming business", it's indication of failure, quite serious failure at that. How can you be that blind and ignore the statements made in a court of law? Whether it was because of NAC's pricing or not, that's irrelevant Whether NAC forced them out or not, again, irrelevant (although they didn't). The plain and simple facts: DedicatedNow knew of their "ip crisis" 2 years ago, yet did NOTHING. DedicatedNow refuses to keep their customers in the loop on things. Credible? No. In fact, they took servers without owner knowledge and authorization, relocating them without notification. Anywhere else, that'd be theft. DedicatedNow has (this has been shown, again, in court documentation) had a negative ballance repeatedly in their history of dealing with NAC, hence the reason why a security deposit was demanded. Again, growing business? not at all. DedicatedNow has acted unprofessionally. You can NOT excuse their crap, or simply write it off. Of course, they're trying to do so with the new "name" of the company. Hopefully it's been mentioned enough that everyone will notice what is really going on, instead of just blindly following whatever crap they're fed. Like someone said earlier: DN and companies like DN LOVE customers like you. They'll feed you whatever, and keep you believing it, regardless of the cold hard facts. If you want your company to go down with DN, then by all means stick around, and watch a sinking ship. Just don't blame me when I said "I told you so", after all this washes up.
Posted by jayglate, 07-12-2004, 11:30 PM
The IT Firm in question is nova-corp.com. Not only are they an IT Firm, but a datacenter engineering firm. They have built many of the exodus/cw/savvis datacenters, as well as many many of the other tier 1 datacenters.
Posted by twidnet, 07-12-2004, 11:31 PM
One mistake? I don't think so. They tried to pull a job in May, got called on it, and backed off. They learned just enough from that fiasco to give four days notice instead of two, self-created an emergency situation, failed to give timely updates, lied about certain items (like the number of people that would be allowed in the NOC), etc. That's not one mistake. That's the sign of a company who can't get it through their heads that these things are important and that communication should never be left as an afterthought. They have taken no responsibility for anything, like I already posted. Why couldn't DN sell more servers? They say it was due to lack of space or power or something. Based on the new documents that are available, I bet at least part of it, and maybe even a good part of it, had nothing to do with space and power, and everything to do with the financial side of things. It's clear by every factor that they don't learn any lessons, they continually try to shift the blame to someone else, and their financial management should be worrisome.
Posted by mrzippy, 07-12-2004, 11:38 PM
Can you provide a contact name/email for someone there? (ie: Your account rep or someone familiar with the contract you have/had with them?) What reason did they give you for not showing up? Has anyone invited them to this forum to post their counterpoint to your accusation? Their response to your accusation may make or break you. if they publicly assert your claim then you will regain some of your lost truth. If they deny your claim, then should we expect a lawsuit against dn/pwebtech for libel?
Posted by jayglate, 07-12-2004, 11:43 PM
This wasn't an issue we were going to bring up with them until the dust has settled. But we expect to be dealing with this issue tomorrow or the next day.
Posted by iFlash, 07-12-2004, 11:43 PM
Yeah, shiny things make it all better. Customers will forget a screw up of this magnitude because of the new shiny things dangled in front of them.
Posted by twidnet, 07-12-2004, 11:49 PM
I dropped a note to their contact address, inviting them to comment on the issue. If I contracted a firm to assist, and they failed to show, I'd be on the phone the very NEXT business day (Monday, in this case) when the crap was still hitting the fans, asking them why they didn't show, and where their people were right then. Surely you could have taken ten of the minutes you weren't updating anyone here or at your forums and called them, considering that Matt claimed midmorning Sunday you only had 15-20% of the servers to still get up and running.
Posted by arubenstein, 07-12-2004, 11:51 PM
NAC does not hope for this. A security agreement holder only wishes to be paid; the last thing that they want to do is repossess used computer equipment. Regarding the posting http://clifton.fortressitx.com/forum...topic.php?t=24 I have the following comments. The size of the datacenter is irrelevant (and, the figure that Mr. Barna presents is factually incorrect. UCI held less than 25% of the space in only one of our datacenters in this building). In a close-quartered open-rack datacenter environment, it is not possible to allow a large amount of people in with unrestricted access. To maintain the security and continuity for all of NAC's customers, it is important to control and limit access. NAC was extremely reasonable in the amount of the people that it allowed into the datacenter at any one time. And even with our restrictions, there were two major instances of disruption to several other NAC customers. As previously mentioned, UCI was well aware of the move out policies (http://www.nac.net/statements/statem...move%20out.pdf). In the spirit of good-faith and cooperation, NAC added staff at its own cost to allow UCI double the original amount of people that were to be allowed in the datacenter. And, again, this did not place any restriction on the amount of people (movers, helpers, etc.) that could be located in the hallway immediately outside the datacenter. I, myself, was even present for the entire move out (from midnight to 7:30 AM EST Sunday morning), and even sustained an injury in doing so. Last edited by arubenstein; 07-12-2004 at 11:57 PM.
Posted by twidnet, 07-12-2004, 11:54 PM
Show me the mon-aaaaaay!
Posted by jbfung, 07-13-2004, 12:01 AM
As someone who generally lurks here at WHT rather than post, I continously find it interesting how quickly people are to accuse and attach based on insufficient information. I know I don't yet feel comfortable stating an opinion about my level of trust for DN because I lack the information necessary to make such a decision. Documentation seems to support the following propositions 1. UCI was behind on several payments 2. NAC raised their prices quickly (unfairly?) 3. UCI originally wanted to move in May 4. Communication was lacking But in all reality, these propositions are really insufficient to draw a conclusion. Here are some counterpoints to some arguments I've seen tossed around: 1. Re. the $100K figures being lauded as evidence of a business about to go bankrupt - we have no idea what the balance sheet for UCI is. $100K is not a lot of money in reality, based on some other numbers I've seen, it's less than 1/3 of a month's revenue for them (again - just conjecture based on incomplete evidence)- which is not a large debt to have in a business. I don't think we can feasible claim they're going bankrupt, when those opinions are based on our own, probably much smaller, income statements 2. Re. the IP allocation and changes - UCI has repeatedly drawn attention to how IP's are given out - I know they've been working to move people over to new IPs for a while - is it possible that they simply were not able to obtain all the IPs they needed from ARIN (APIC/whoever?) (this is an honest question - I'd love more info on the topic) 3. I'm not surprised to hear excuses of having fewer people than anticipated. I can easily envision a scenario of them having a plan where one group of people were to do tech support, one group moving, a few others maintaining open lines of communication - then them realizing that the movers were insufficient and EVERYONE needed to jump up to get the servers in. Like I said in another board - I'd rather have them working to fix my server, than sitting watching a discussion board. 4. I don't know the technical details in a move this large, and I don't fully understand the restructions of the restraining order, but as far as I understand, the TRO was only for a few days - and the network rerouting had to be cut off at a root point - thereby eliminating any feasible option to do the move in chunks. So I don't think saying "they should have done it in waves" is a feasible option. Ultimately, my point of view comes down to the fact that I trust that UCI isn't run by a group of brain-dead neanderthals - I'm sure they though of most of, if not all the things we've discussed her. I'm sure they made the best decision possible and I believe unexpected things happened. As was said before, trust is built on repeatedly delivering good service - although this was a distressingly long downtime (yes, I was affected too) - up until now I have had nothing but positive feedback for them. Never once have I moved a server from there service - rather I've moved hosting accounts from servers with other service providers because of how much better I like their product and service. I guess this post is simply recommending that people try to keep an open mind, and realize that there are probably facts and factors you probably haven't thought of. Last edited by jbfung; 07-13-2004 at 12:05 AM.
Posted by jbfung, 07-13-2004, 12:07 AM
Oh yeah - I think it's Jason's job to deal with his IT company btw, not ours...just my 2/100 $
Posted by mrzippy, 07-13-2004, 12:09 AM
You don't think it's important to find out why they didn't show up? After all, you're now practically blaming this entire screwup on their shoulders. If they can validate your story, then you will gain a lot of credibility. But if they post "against" you, then you're really screwed. I would love to you YOU make an open invitation to them to post here. After all, YOU hired them and you say they screwed you... so why not let us see YOU invite them here to explain why they screwed you and take some of the blame off your shoulders.
Posted by twidnet, 07-13-2004, 12:16 AM
I was keeping an open mind up until oh, about the 12 hour mark when Matt gave the infamous 80-85% figure and it was clear things had gone wrong from the start. With the information over the past 48 hours, it's easy to draw certain conclusions. Your number 2 is wrong. There's nothing unfair about raising prices. Jay signed the contracts. So, if they're complaining about high prices, they're not going to win points for signing a contract they didn't like. The financials. DN was required by NAC to give financial statements. They state facts in their own filings that show things aren't all that rosy in DN-land. The fact that Jay had to sign as a personal guarantor is just scary, and tells me as a financial type person that the company's books probably look even worse than we might think considering that the parent company is an incorporated business. Nobody said they should be sitting around watching forums. They started this very thread, or did you miss that? They said they'd update to their own forums. Ed in his apologia made the very first post from anyone at DN at those forums, and updates here were few and far between. Two minutes to type up something every little while is not asking them to sit there and constantly refresh the thread. IPs. Their largest block from ARIN was issued September 2003. Depending on where you read in the DN legal filings, somewhere between 35% and 50% of their customers are (still) using NAC ips at the time they filed for the TRO. That's a long time to have that many ips (do the math, it's 32,768 ips) and not be able to get the job done. My research says it's doable even if they were using all NAC ips when they got that from ARIN (even though they have other blocks as well, and they were not using 100% NAC ips by Sept 2003, which a google shows, mostly because of the abuse complaints you can find). The moves in waves. Yeah, this jibes with what another host explained to me about it, technically.
Posted by arubenstein, 07-13-2004, 12:17 AM
First point: The public record shows that UCI obtained an initial ARIN allocation in March of 2003. http://ws.arin.net/cgi-bin/whois.pl?...216-67-224-0-1 Second point: ARIN will generally give you IP space as needed, as quickly as you need it, assuming that you are using the IP space properly and documenting it properly, based upon my own experience. Third point: You should read the following certification, and draw your own conclusions. http://www.nac.net/statements/statem...d%20Kovach.pdf There is nothing that NAC did or did not do that would have interfered with UCI's ability to move in 'waves', or even over multiple days. UCI chose the schedule, not NAC.
Posted by AussieHosts, 07-13-2004, 12:17 AM
It was impressive how quick they did it. Now...if only they had forewarned us... Best move United ever made though. Sagonet are solid. Gary
Posted by NexDog, 07-13-2004, 12:21 AM
Why is twisted, sorry twitted, sorry twidnet involved in this thread if he isn't even with DN. Anybody smell a troll?
Posted by iFlash, 07-13-2004, 12:25 AM
Is there some law about only people currently with DNOW being able to post here in this thread?
Posted by twidnet, 07-13-2004, 12:26 AM
I suppose I should whine about your name-calling, like you have about me. Like you've never barged in on threads where you obviously are not with whoever is being complained about. Maybe you missed this. I am not with them ANY MORE. Not clear enough for you? Maybe it was the close to 48 HOURS OF DOWNTIME that made me go. Maybe it was around the 24 hour mark when no updates were coming that I made other arrangements. And maybe you shouldn't be drawing conclusions when you obviously have no clue about anything.
Posted by NexDog, 07-13-2004, 12:33 AM
You show all the hallmarks of a scare mongerer. I had respect for you until this thread. The way some people have behaved in this thread is a real eye-opener. Anyhow, were those servers colo'd with Pegasus or NAC? Hmm, not exactly rocket science. Are you a DN customer or just another troll? I can excuse "their crap" because I've enjoyed stellar service for almost 2 years. People like you come here and try and twist things around like the above statement. What has the name change got to do with anything? You're saying the name change is to cover up this fiasco? The name change was decided ages ago. DN remains DN but Pegasus is now FortressITX. Yep, good company in that quote - sums you up. Of course they love loyal clients like us. Always enjoyed amazing service for an amazing price. Helped us grow to where we are today and will stand behind them all the way and "take bullets" (not that the amusing cheap shots by yourself and that other guy even closely resemble bullets - more like hot air being blown up my *** that tickles). Why would I blame you if DN fails? Do you think you have power in this situation or something? You're a nobody to me, mate.
Posted by halldorr, 07-13-2004, 12:33 AM
Why is it wrong? So if they raised it several times at horribly unfair rates it's not price gouging? It has been more than implied in this huge thread that the DN crew should have been sitting here replying to people with downed servers giving up to date status information. Besides, jbfung was simply stating he'd rather them working on servers than on a forum answering everyone personally.
Posted by NexDog, 07-13-2004, 12:39 AM
Ah, nothing like a vendetta. Isn't that the same Dave Kovach that got fired from Pweb/DN? So the guy gets fired from Jay's company for what I hear was "incompetence", and now gladly files an affadavit on behalf of NAC against his former company. Not like he has a grudge or anything?
Posted by twidnet, 07-13-2004, 12:41 AM
Who says it was unfair? DN? They have an option if they don't like the pricing, don't they? They did not exercise that option when they signed the contract did they? They think signing a contract they didn't like and then whining about the pricing is going to sway a judge? Who has implied that DN should be sitting around, refreshing, constantly posting a reply to anyone who posted here? Who says they should have been answering everyone personally? What people have been saying, me among them, is that they should have been posting SOMETHING, especially to the forums they referred people to in their own announcement! How hard is that to understand? Do you realize that until Ed posted to those forums that not a single update had been posted there by DN, even though DN said they would be posting updates there? I know exactly what jbfung was saying, but posting nothing at all, and then limited information only when there's clearly some major issue is wrong. When you've said you'll post updates to your own forums, and don't (and even have people comment on it there when rare updates are posted to WHT!), that's also wrong.
Posted by twidnet, 07-13-2004, 12:43 AM
Open yourself up to lawsuits much? How about opening DN up to one ("what I hear", like it would have come from anyone other than DN) with yourself called as a witness? I'm sure NAC would be glad to engage their own counsel on behalf of their employee.
Posted by NexDog, 07-13-2004, 12:49 AM
I heard from NAC so put that in your pipe and smoke it. I have lawsuits everyday. Breakfast, lunch and dinner. Yummy.
Posted by twidnet, 07-13-2004, 12:55 AM
You think that makes you special? I think it makes you someone to avoid like the plague. Someone cheering themselves for having lawsuits swirl around them doesn't give a good aura. You could ask SCO about that. Like I said in the other forum, whether he was fired or not doesn't matter if what he says is true. I bet we could find someone else who could also attest to the fact that DN knew, like all providers know, that they would have to reassign ips if they left NAC.
Posted by rois, 07-13-2004, 01:02 AM
How about getting back to the topic and maybe stop insulting each other Attacking each other will not get your server problem solved any quicker. my 2c
Posted by twidnet, 07-13-2004, 01:10 AM
Sorry. Back on topic. I'd love to have a peek at DN's books. If the credit worthiness check was serious enough that Jay had to sign a personal guarantee to NAC (no matter who they reassigned the contracts to), it would be interesting reading. I still think their communication stinks. I'm still surprised that anyone offers any support for this fiasco. If that IT place responds to me, I'll post what they send.
Posted by dswallow, 07-13-2004, 01:16 AM
Just to follow up on something I'd said earlier about not having received the notification email... Now that my server is back up, I searched the archives of email that was classified as SPAM and found 3 copies of it from 7/7. It got there because it failed an RFC-based test to verify all CR and LF characters were together at the end of each line. So indeed Pegasus did do as they claimed... notified me. It was mostly my fault for not having them whitelisted prior to such a test; though I'll give them say 5% of the blame for sending non-RFC-compliant formatted email. You'd think Unix people would pay more attention to RFC's.
Posted by NexDog, 07-13-2004, 01:29 AM
Thank you, twidnet, for taking time out of your busy schedule to duel for NAC and fish through all the evidence. I think it might be time to sum up a few things here. I'm totally amazed how some members have conducted themselves in this thread. But most of the noise has come from members with no servers down and nothing to do with this move. It's almost as if Alex from NAC is channeled his goons in here to screw up this thread. We have people screaming for updates from Pegasus and when they come, they are pounced on, disected and twisted. Hell, if I was Pegasus, I wouldn't be in a hurry to post public updates either. The lack of support shown to Pegasus and the way the community has pounced on this damn fine company with a spotless record till now is downright DISGUSTING. Everyone is speculating over Jay's personal finances and his ethics. No-one has applauded him for going out on his own and setting up a new datacenter. Instead, all the focus has been on the IP dispute that has NOTHING TO DO WITH ANYBODY except Pegasus and NAC. The IP dispute doesn't affect anybody and the courts aren't going to overrule the situation on Friday (as someone suggested). The way members that have tried to put a positive light on things and put in a good word for Pegasus have been set upon is also a disgrace. There has to be balance. But I will continue to post in support of the Pegasus crew. I know the company is not going to fold in the next few months. I know I still have at least 2-3 weeks to co-ordinate IP migrations. I know that FortressITX will continue to provide first-class service and support. And I know that I'll be ridiculed for saying so. But what I will say is that at least I'm involved. I'm a customer of Pegasus/DN with about 20 servers and will add 5 more this month so Jay can feed his fish. No troll here.
Posted by Perfecthost, 07-13-2004, 01:42 AM
Another thread sucked down the toilet.
Posted by Steven, 07-13-2004, 01:42 AM
I agree with you 100% NexDog. WHT is full of so much drama
Posted by akuo, 07-13-2004, 01:50 AM
Best of luck to you Laurence, there's a lot to be said for loyalty. I sincerely hope DN can put this whole mess behind them and rebuild their previous reputation
Posted by Thraxion, 07-13-2004, 01:57 AM
I've been with a reseller for dedicatednow, and can never thank them as much as I want to. Their service has been superb throughout the years, and I will stick with them due to their past reputation. They may be in a small hole, but I see a great milestone they will soon look back on and wonder why they didn't invent wireless server transit sooner
Posted by NexDog, 07-13-2004, 02:00 AM
Loyalty is a 2-way street. Pegasus have gone way beyond what was promised when we first joined them. They've been loyal to us and shown time and time again that they value our business and are willing to work with us on custom configurations etc. Plus, as I said before, when things looked the most bleak, they were there for us. This will always be a taint on their reputatution but who doesn't have taints? Two months from now when FortressITX is taking on new customers and the IP issue is history, they will do really well. DN/Pegasus is a "hands-on" provider - unlike Managed, The Planet etc. I bet you'll hardly ever (if at all) see threads about bad support.
Posted by WebSun, 07-13-2004, 02:52 AM
Our server was done for 33 hours, we have maybe lost some business but if customers were interested in our product, they will come back. Also its look that there is a lot of freshman in business here who thing that to be down for 12 to 24 hours will drive them to the bankrupt, I thing that any event in their live will put them in bankruptcy if they are unable to manage a situation like this. Stop to do business and try a career in the public administration I wish the best to Jay and his crew for they new venture
Posted by thaphantom, 07-13-2004, 02:54 AM
thats only if they make it out of the TRO, what if the judge revokes it and all your servers go down on the 17th? How happy will you be then, still saying how they are so great. Dont think ARN is gonna jump out and give em a bunch of class C's right then either... I am sure they arent too happy bout the court order either.
Posted by Thraxion, 07-13-2004, 02:57 AM
I think the NAC would be on the losing end if they were to take all the ip addresses back- most likely ticking off a lot of customers at DNow. In my mind the NAC would look great if they were to give the addresses to DNow, but maybe that is to fantastic to dream..
Posted by WebSun, 07-13-2004, 03:05 AM
First notice of migration come on April 1 2004 and official notice on June 26, 2004. There was a lot of time to make the move
Posted by qps, 07-13-2004, 03:05 AM
They can't give the addresses to UCI because they are not NAC's to give. NAC does not own IP addresses. IP addresses are not property.
Posted by Aussie Bob, 07-13-2004, 03:17 AM
Was there a bit of biffo? Do tell, Alex.
Posted by thaphantom, 07-13-2004, 03:29 AM
yea what happened alex?
Posted by The3bl, 07-13-2004, 03:29 AM
You assume the world is in the same shape as you. We lost a few customers, we will get over it, we can take that hit. We lost a couple of grand in revenue as we are having to give a free month to those on the worst affected servers and some of our dedicated customers want to move so we have to buy them new servers and set them up in another data center. But again we can take the hit and survive it. What we cannot take and hurts more than anything is our resellers they depend on us to keep their sites up some of them have lost many clients and income they will not be able to recover from. Quite a few of them have worked their butts off to grow their business to get their own dedicated server only to be shot down by this and now they may not be able to hang on. Sure we can take the hit but the small guys with one server or that were resellers on servers that were down for 45 hours might not survive, it is plain wrong to say just get over it you are a hosting novice. I know if something like this had happened to me when I was starting out and only had one server I was struggling to make the payments on each month I might not be here today. I take my share of the blame for the problems I should have done more to make sure this would not happen to them, but I trusted a crew I had always had excellent service and raport with, never in my wildest dreams did I think that Jay, Matt and crew would do this to us and not respond to us in any fashion. What the future holds for us and Pweb I have no idea I am only concerned at this point in helping my resellers and dedicated clients get back on their feet once we do that and the dust of this move settles we will see where we are at ..
Posted by NexDog, 07-13-2004, 03:35 AM
Wise words and spoken with a cool head. Techark were hit bad by this situation and still managed to conduct themselves with self-respect and decorum. An example to many.
Posted by Samuel, 07-13-2004, 03:51 AM
I heard Alex punched 30 of the outsourced crew in the face, knocked em flat, and then laughed so hard he spilled his coffee with his other hand and burned his right toe, which might explain why the crew were unable to get in.
Posted by Shaw Networks, 07-13-2004, 03:54 AM
I don't think I could ever see myself that calm during downtime like this
Posted by NexDog, 07-13-2004, 03:54 AM
That is something I would have loved to have seen.
Posted by whmcsguru, 07-13-2004, 03:56 AM
Much unlike yourself, as usual? It's pretty obvious to me who the "trolls" are in this case, that would be the insulting individuals, such as yourself, or the ones that insist upon attacking others merely because they don't like their post. Personally, I don't really care whether you like the posts I made or not, or whether you "respect" me or not. Doesn't mean 1 thing to me. In fact, it's quite amusing to me really, how you cna blindly ignore facts, and continue to have so much crap and lies pumped into you, yet being so gullible that you actually buy their crap. I guess, there really IS one born every minute. This is a public form, and as such, advice is given publicly, freely. It's not required that one have any experience with the company at hand, however, in my case I did and do. Because of UCI's monumental screwup, I had at least 1 client down 24 hours. Now, that's all well and good, but considering I had to be up for those 24 hours with this client, their crap affected MY business and MY person. I, like everyone else affected by this screwup (every cpanel user included) have every right to be upset, voice our frustration and anger against the one corporation that DID screw this up. You can NOT blame this screwup on anyone but UCI. They've tried blaming NAC, they've TRIED blaming their "outsourced" help, hell, I'm sure Jay would try blaing his MOM if he could get her into it somehow, anything to avoid taking responsibility for his own actions. Like I said, the records show it all. Court records, et all. This business is on a very downhill road. Sure, stay with 'em, that's fine, just don't be surprised when your ip gets yanked, or you go for days without service because Jay refused to pay the power (or connectivity) bills like he has here. Again, this was all shown PUBLICLY, in PUBLIC documentation.
Posted by Samuel, 07-13-2004, 04:00 AM
That is what sucks, having to babysit problem servers over night, but really isn't that what life is anyway? If I have someone out that is sick, what do I do? I cover for them. I am getting old very fast and while Dnow might be, or might not be going into the toilet, at least this is enough public info to make a wise choice either way. If they do go into the shitter (No disrespect inteded DNow), you've had warning.
Posted by whmcsguru, 07-13-2004, 04:06 AM
Yeah, but this isn't 1 person sick, this is like hundreds, if not thousands (in the case of CPanel, which admittedly should have learned their lesson @ Burst) being sick, and was completely avoidable.
Posted by NexDog, 07-13-2004, 04:10 AM
Aw, poor cpanel users couldn't get an update. Life surely does suck.
Posted by NexDog, 07-13-2004, 04:11 AM
You didn't see me crying like a girl over this.
Posted by iFlash, 07-13-2004, 04:24 AM
You know, it's great that you are happy with 18 hours of downtime and that you are satisfied with the way DNow handled this, but a lot of people aren't. Making fun of them because they are concerned and upset and rightly so, is about the most trollish and childish thing I've seen in this discussion yet.
Posted by NexDog, 07-13-2004, 04:36 AM
I wouldn't say I was happy with it but I'm capable of looking at the bigger picture and I'm also capable of forgiving this incident. I don't think you or wolfstream really understood what troll what means but we'll let it slide.
Posted by The3bl, 07-13-2004, 04:42 AM
I think we have had enough of the flames over who is right or wrong and the name calling. The thread is here for anyone that may still be having problems with the move. Lets leave it at that and stop flaming each other, or I will clean the thread and issue warnings. You are welcome to express your opinions on the the move and the company but stop the name calling and flaming of each other.
Posted by ElrondTwg, 07-13-2004, 05:03 AM
This talk of another potential downtime on the 17th has me a little worried. What IP ranges would be affected, anyone know?
Posted by NexDog, 07-13-2004, 05:06 AM
I agree entirely. But it's difficult being in the minority and wanting to support Pegasus against the masses. And lets face it, with almost 20 servers there, we were affected more than most in this thread.
Posted by NexDog, 07-13-2004, 05:08 AM
Don't let the scare mongerers worry you. But if you got your server after March of last year, you should be ok. Anything in the 69.57, 69.72 and 216.67 class B ranges doesn't need migrating.
Posted by ElrondTwg, 07-13-2004, 05:16 AM
Thanks NexDog... much appreciated.
Posted by newyorker, 07-13-2004, 07:28 AM
what about: 66.246 ?
Posted by RossH, 07-13-2004, 07:52 AM
Did you people even read how much they were paying? No you just heard that they were paying unfair prices. They were paying $80/mbit for DCJN and $100/mbit for pure nac. That is not price gouging by any means as many colo companies even around here start out at $80/mbit and most are in the $125-175/mbit range.
Posted by Samuel, 07-13-2004, 07:57 AM
lol@ "You People"
Posted by rmMark1, 07-13-2004, 08:09 AM
Those are NAC's and will be returned rightly so.
Posted by NexDog, 07-13-2004, 08:12 AM
Yep, that class has to be migrated. If you haven't received your new IPs yet, email network@pwebtech.com to them. I'll ignore the attempted dig "returned rightly so" by monsignor newbie.
Posted by Aussie Bob, 07-13-2004, 08:19 AM
The price hikes were not as much to do with bandwidth pricing, as they were to do with electricity charges, from what I understood in reading those docs.
Posted by rmMark1, 07-13-2004, 08:23 AM
Its not a dig, but the truth. Once the TRO is removed, everyones site are coming down. Again, poor planning on DN part. Hell, I think NAC should give all the IPS to DN. That way NAC can get lifted from all the abuse blocks that DN caused form not nuking its spammers, I know many anti-spam groups already placed blocks against DN new range. here is one; spews.org/html/S2649.html
Posted by Samuel, 07-13-2004, 08:28 AM
bah depreciated spews, means nothing.
Posted by rmMark1, 07-13-2004, 08:35 AM
There not the only ones. DN has polluted the net with those blocks and anti-spammers are blocking DN new Arin space. My point is, NAC should if they could ( They cant under ARIN policy) transfer those dirty IP's to DN, as it would help clean up NAC's own space.
Posted by NexDog, 07-13-2004, 09:13 AM
Jesus, rmMark1, ASAIK, you ain't welcome here with your off-topic BS.
Posted by susannad, 07-13-2004, 09:16 AM
signed up just to post that ?
Posted by rmMark1, 07-13-2004, 09:26 AM
Nope, been following this thread from the 11th and see all the bashing on both sides. I know that other reasons that NAC was having a hard time with DN not handeling its abuse complaints, just more facts being brought forward. Last edited by rmMark1; 07-13-2004 at 09:32 AM.
Posted by rmMark1, 07-13-2004, 09:35 AM
Do you know if Cpanel is already migrated? Thanks
Posted by NexDog, 07-13-2004, 09:40 AM
Well, layer1 and layer2 are: Cpanel.net itself isn't hosted with Pegasus (as as I understand it).
Posted by mainarea, 07-13-2004, 09:41 AM
That's just IP migration, that doesn't mean that they're changing datacenters. It simply means that Pegasus was moving you from NAC's IP blocks to theirs. I believe DN still has NAC connectivity, so unless they drop NAC connectivity, I don't think the IPs are going anywhere. Of course, that's only a guess. Yes, CPanel has migrated to new IPs. All of CPanel is hosted at Pegasus, they don't have any backup servers online yet. CPanel.net = 69.9.165.24, and while that is part of Voxel's IP block, those are the temporary IPs that PWebTech got from Voxel (see http://www.fixedorbit.com/cgi-bin/cg...ine=cpanel.net ) - Matt - Matt
Posted by Samuel, 07-13-2004, 09:44 AM
Voxel.net http://uptime.netcraft.com/up/graph?...el.net&probe=1
Posted by rmMark1, 07-13-2004, 09:52 AM
Yes, correct..That is how I see, but after reading the legal docs,. The TRO is to announce DN existing IP blocks provided from NAC. Once the TRO is lifted, NAC has ever right to remove those blocks unless orderd to continue. Cool...Good to know.
Posted by NexDog, 07-13-2004, 09:52 AM
Yah, I was wondering about the Voxel connection. Thanks for clearing that one up.
Posted by twidnet, 07-13-2004, 10:43 AM
Since you obviously won't read all of the documents, someone has to point these things out. Most of the noise has come from people who were directly affected. You're playing DN's game, by blaming Alex and NAC. Yeah, that's great business relations there. Say you're going to post updates to a certain forum, and then never post a thing until over 40 hours later. In another forum, post that things are rosy, even when ech passing hour says they're not. Hold on just a minute. People may have not liked the sudden, four day notice. I didn't. But people have a right to expect that the provider knows wtf they're talking about and that the things they say are ACCURATE. How can you possibly support DN for not only grossly misleading their clients but just flat out lying to you? They KNEW NAC was not going to allow all those people into the NOC. They knew way beforehand. But let's say NAC hadn't told them that, although they did, well in advance. Let's say NAC halted them at the door and told them at 12:01. You don't think it woud have been a benefit for people to know that? "Sorry, folks, our plans have changed because we're not going to be able to get as many people into the NOC as we originally thought." How hard is that? What is there to suport of a company in this fiasco when they didn't plan well, omited information it would have been useful to know, didn't communicate well at all, and lied? What is disgusting is that anyone has to defend how pissed off they are about all this to people like you because you just blindly support their actions. Jay must schmooze you pretty good. People can comment on public records like court documents, and sometimes, like with providers, it's idiotic if you fail to look over what's presented and make certain conclusions, even privately. And what's to applaud about leasing space in a building and turning it into a small datacenter? People do it all the time, you don't see others trying to say they've done somethig no one else could imagine. Gosh, hope you don't have any servers with NAC ips, Laurence. It has EVERYTHING to do with other people who might still be using those NAC ips. I can't even believe that someone in the hosting buiness would say something like this. Believe what you want. But the people who have been sorely mistreated in this by DN won't forget, and many have already decided to leave. Google also never forgets. Luckily, some people like yourself have posted the old and new names together so searches will turn up all the details. There are rules people draw up for themselves. Those rules change from time to time, but one of my rules is to not do business with any company that treats their clients with contempt. DN falls into that category for obvious reasons, not the least of which is their total mishandling of this move. You fall into this category because of your blind acceptance of unacceptable behavior from DN, without any indication that you disagree with what they've done here, and your constant berating of people who are asking the questions that need to be asked.
Posted by jbfung, 07-13-2004, 02:26 PM
I find it interesting how many people here are recommending..nay..proselytizing making a snap decision about the ethics and blame associated with this situation, far, far too early. Many people have been disapointed with the way problems and comunication happened - but I don't think any of us here are in a position to pass judgement based on so-called "facts". (this goes for posts coming from both sides of the fence) All the "facts" I've seen people post here are conjecture and perceptions of court documents, which are themselves perceptions of the authors and/or filing parties. My recommendation to everyone here, is rather than telling people to stay, or telling people to run, we try to make sure everyone her has a thread where they can get the information they need. Part of that involves waiting to see what happens once UCI and NAC get the time to pause, take a breath, and start posting valid information. I don't believe either company is making posts they'll be proud of later this week
Posted by PhMatt, 07-13-2004, 04:41 PM
We have posted an industry announcment in regards to all ongoing situations presently located at: http://www.webhostingtalk.com/showth...hreadid=296125 If the MODS feel it best moved into other threads, that is fine with us. Thank you,
Posted by Samuel, 07-13-2004, 06:54 PM
http://www.digikitten.com/playhousev...ith/revive.gif
Posted by The3bl, 07-13-2004, 07:06 PM
I think this thread has served it's purpose.