theplanet problems [Merged]

Portal Home > Knowledgebase > Articles Database > theplanet problems [Merged]

Posted by TotalChoice - Bill, 09-24-2003, 03:22 PM
Anyone else having problems @ The planet? Very odd tracerts and server is down. Just wondering as we await a reply from the support department.
Posted by Protollix, 09-24-2003, 03:22 PM
one of my servers is down... the trace route loops back from router to router indefinately.
Posted by TMX, 09-24-2003, 03:23 PM
Apparently a switch went down. No ETA as of 3:20pm eastern, but they're working on it. -B
Posted by TotalChoice - Bill, 09-24-2003, 03:23 PM
Did you have down time yesterday as well?
Posted by inteltechs, 09-24-2003, 03:23 PM
yeah, all my servers are down too. They said there were having a network issue.
Posted by frozen, 09-24-2003, 03:24 PM
2 of mine are down as well, outage report submited.
Posted by Eric M, 09-24-2003, 03:24 PM
Both of my servers at the Planet are down. 69.56.xxx.xx
Posted by lastminutehosting, 09-24-2003, 03:24 PM
Just got off phone with noc said they have 3 techs dispatched on it should be back soon.
Posted by TheVoice, 09-24-2003, 03:24 PM
My servers are no longer with us as well.
Posted by sheryip, 09-24-2003, 03:25 PM
so how much downtime does it require to drop to 99.9%?
Posted by TotalChoice - Bill, 09-24-2003, 03:26 PM
The only reason I started the thread was this same thing happend yesterday, and although the outage was short it appears to be the same issue.
Posted by inteltechs, 09-24-2003, 03:26 PM
15 mins i think
Posted by frozen, 09-24-2003, 03:26 PM
Hopefully just a minor glitch, I am sure they will make it up to us
Posted by TotalChoice - Bill, 09-24-2003, 03:26 PM
Depends on SLA but 99.9% = 43 minutes of downtime per month
Posted by inteltechs, 09-24-2003, 03:27 PM
I noticed it too, I think it was more than 10 mins downtime last night
Posted by frozen, 09-24-2003, 03:28 PM
It is at 19 right now
Posted by LiveRack, 09-24-2003, 03:28 PM
Ditto. 3 of my servers are down there too. Their support team says they're having network problems and that they're working on it.
Posted by percent5, 09-24-2003, 03:29 PM
Yes...our servers are down too. Does the Planet have a place where they report network troubles to their customers, or did those of you who heard about the network problem hear it from tech support?
Posted by Matt, 09-24-2003, 03:29 PM
Hmmm, we had no downtime yesterday or last night, but it is down right now. Monitor checks every minute.
Posted by Matt, 09-24-2003, 03:30 PM
You can submit it through Orbit, which I am sure is getting flooded right now, mine included
Posted by frozen, 09-24-2003, 03:31 PM
We had no downtime last night either, just today. I have submitted a ticket via orbit for each server I have there, if they are being flooded with tickets, props to em, because they responded to me within 5 minutes.
Posted by ElysiumNet, 09-24-2003, 03:31 PM
if im not mistaking there was a 2 or 3 hours of downtime in the beginning of september/end of august, but it was during the night though
Posted by ElysiumNet, 09-24-2003, 03:32 PM
no downtime yesterday, just today.. (for us)
Posted by trustedurl.com, 09-24-2003, 04:03 PM
still down for us.... it's been 20+ minutes... 15:38 is when we first were alerted.... 99.9% SLA? Anyone have a link to it. I can't find it very quickly, but I'll be requesting everything allowed under the SLA. last known working connection was 15:01. I couldn't reply to the other thread
Posted by belindaj, 09-24-2003, 04:08 PM
I couldn't reply to the other thread either. My servers are also down. Came here to see if it was just me before entering a ticket. I do see the announcement of the trouble on the Orbit site - sure hope it gets fixed fast - my phone is going to start ringing off the hook any moment I'm sure as people start to realize they are not getting to their mail & sites.
Posted by DrCool, 09-24-2003, 04:09 PM
Is this a new trend in the industry.. every dedicated server company has to have downtime, one after the other.. this is getting ridiculous.. arn't any of these companies taking notice and being more prepared? Note: my server with the planet is also down! websitepluse monitoring says, for about 1 hour now.. Dr Cool Last edited by DrCool; 09-24-2003 at 04:16 PM.
Posted by jay23, 09-24-2003, 04:11 PM
So do we all move to Rackspace now ?
Posted by Eric M, 09-24-2003, 04:11 PM
Both of mine have been down for more than an hour now.
Posted by cxreg, 09-24-2003, 04:13 PM
I wonder if unlike some other providers *cough*burst*cough*, they will actually respect their SLA I too have been down just over an hour
Posted by htremblay, 09-24-2003, 04:13 PM
Same thing for me, I received my first monitoring email alert at 15h40. Everything is down...
Posted by TotalChoice - Bill, 09-24-2003, 04:13 PM
We are well over a hour down now.
Posted by ElysiumNet, 09-24-2003, 04:13 PM
siteuptime says 1hour for me, my client also noted that there was downtime in the morning but then it went back up
Posted by frozen, 09-24-2003, 04:32 PM
Something with ElysiumNet is messing with the ability to postreplies to the threads, I've seen this bug on a few other sites as well, i is always triggered by the same usernames.
Posted by frozen, 09-24-2003, 04:33 PM
Oops, this should be moved.
Posted by cxreg, 09-24-2003, 04:35 PM
Being new to theplanet, I'm curious if anyone here has ever claimed downtime against their SLA? And is this pretty rare? Prior to switching, I had never seen a single complaint about them on this forum..
Posted by Dathorn-Andrew, 09-24-2003, 04:35 PM
I talked to Jeff R. and the problem is with a switch apparently. He said he'd give me a call back as they should have a solution in the next 15 minutes or so but knowing the type of problem it is I would count on longer.
Posted by frozen, 09-24-2003, 04:36 PM
It is back up now.
Posted by percent5, 09-24-2003, 04:36 PM
It took a while, but they seem to be up again.
Posted by ElysiumNet, 09-24-2003, 04:38 PM
yup, its back up
Posted by roccol, 09-24-2003, 04:38 PM
I am still down.
Posted by lastminutehosting, 09-24-2003, 04:38 PM
down for me still
Posted by TotalChoice - Bill, 09-24-2003, 04:38 PM
Still down here
Posted by Dathorn-Andrew, 09-24-2003, 04:38 PM
And the one time that I do expect the worst I'm wrong
Posted by frozen, 09-24-2003, 04:39 PM
Hmm, expect the worst more often when these kind of problems arise then
Posted by TotalChoice - Bill, 09-24-2003, 04:40 PM
Why cant they just reboot their network
Posted by ThePrimeHost, 09-24-2003, 04:40 PM
One of our servers at The Planet is still down. Anyone else, still down? Best Regards, -Darrell
Posted by Matt, 09-24-2003, 04:40 PM
I'm back, on 69.56.172.**
Posted by belindaj, 09-24-2003, 04:42 PM
LOL - I think I'll trust that the exemplary service The Planet/SM has stood for all this time will continue. I hate this downtime too but I trust they are working their butts off to get it fixed. Mine are now back up - but yes, it was over an hour. Glad it's back up though.
Posted by twastudios, 09-24-2003, 04:42 PM
Did anyone not recieve a page? I just noticed this and my box is offline as well, and I have not been paged. From Orbit:
Posted by TotalChoice - Bill, 09-24-2003, 04:44 PM
My blackberry went nuts about an hour ago (from our own monitor), still waiting for the "host is alive" pages. Just a personal note, I wouldnt rely on the provider to monitor your server(s). You should consider a off site monitoring tool/server.
Posted by TotalChoice - Bill, 09-24-2003, 04:47 PM
Is anybody still down?
Posted by ElysiumNet, 09-24-2003, 04:48 PM
1 server is still down for us, the rest are up..
Posted by lastminutehosting, 09-24-2003, 04:48 PM
Im down still
Posted by HappyManEd, 09-24-2003, 04:49 PM
Still down for me 69.56.171.***
Posted by ElysiumNet, 09-24-2003, 04:49 PM
so did this go over the SLA?
Posted by BigJ406, 09-24-2003, 04:50 PM
up for me now
Posted by twastudios, 09-24-2003, 04:53 PM
We have been working on our monitoring and had it offline when this issue popped up with SM. How's that for timing
Posted by ElysiumNet, 09-24-2003, 04:55 PM
yup, we still have 1 down.
Posted by frozen, 09-24-2003, 04:57 PM
69.56.187.* good to go. Wow it let me post after you. lol
Posted by trustedurl.com, 09-24-2003, 04:57 PM
ditto.
Posted by vivac, 09-24-2003, 05:00 PM
Server is still down for me too... 69.56.168.***
Posted by ElysiumNet, 09-24-2003, 05:02 PM
hehe
Posted by WebHostingNeeds, 09-24-2003, 05:06 PM
My servers is back online now
Posted by kingpcgeek, 09-24-2003, 05:06 PM
my two servers were never down, last night or today
Posted by TotalChoice - Bill, 09-24-2003, 05:11 PM
yep still down here 69.56.169.***
Posted by Mrdredd, 09-24-2003, 05:12 PM
69.56.133.*** has been up consistently
Posted by wakkow, 09-24-2003, 05:14 PM
I have a colo there that went down for a few minutes but seems back up now. 69.56.172.xx
Posted by lastminutehosting, 09-24-2003, 05:26 PM
69.56.179.*** still down
Posted by TotalChoice - Bill, 09-24-2003, 05:31 PM
I feel your pain We are still down as well. Going on well over 2 hours
Posted by ElysiumNet, 09-24-2003, 05:34 PM
same here 69.56.165.***
Posted by coastweb, 09-24-2003, 05:42 PM
Also still down on 69.56.188.xxx but I trust that they are just as unhappy as we are especially since there has been nothing but praise for ServerMatrix for quite a few weeks now. I also trust they are making every effort humanly possible to rectify the situation. I just moved to them a couple of weeks ago and from the sounds of their SLA I should see some money off my next bill.
Posted by ThePrimeHost, 09-24-2003, 05:47 PM
...I suppose this happens to all networks at some point. must..............remain......................positive............ Best Regards, -Darrell
Posted by Nessun, 09-24-2003, 05:50 PM
wow started reading this post and was scared my server was down checked it and never went down thank god
Posted by HostingScene, 09-24-2003, 06:00 PM
Wow...two hours of downtime thus far! Anyone have a copy of their SLA?
Posted by TotalChoice - Bill, 09-24-2003, 06:13 PM
Still down.. yea
Posted by frozen, 09-24-2003, 06:13 PM
It is on their site last time I looked. If you are still not up I recommend logging into orbit and asking for a update on the situation. I've had times where the network one of my servers were on went down and in the meantime my server crashed for some odd reason. The network comes back up, yet I never knew because my server was still down.
Posted by TotalChoice - Bill, 09-24-2003, 06:15 PM
Update from The Planet IMPORTANT NOTICE! Update: Most of the customers who were affected by today's incident are currently online. However, in order to bring the rest of the affected customers online, we will have to reboot a router which will cause a momentary outage for the customers whose server had already been restored. This will only affect those customers who were affected by the earlier incident. This reboot will occur approximately between 17:00 and 18:00 CST. Thank you for your patience.
Posted by Alex042, 09-24-2003, 06:16 PM
I wonder what happened anyway. Maybe I should move my secondary DNS server outside of their network when I get a chance.
Posted by TotalChoice - Bill, 09-24-2003, 06:18 PM
Wondering why they are allowing a 1 hour window on rebooting a router. You would think they would reboot that router asap and get us back up. THAT makes little sense to me.
Posted by Mans, 09-24-2003, 06:18 PM
They took four days to setup my server three weeks ago and now two hours of downtime.... what a good company! It happened twice for me! Once with MCHost and now with this TP just bad luck i guess
Posted by YourHost, 09-24-2003, 06:19 PM
Just an update we will be rebooting some core Server Matrix gear within the next 45 minutes. This will bring all customers that are currently up back down for a very, very short amount of time (5 minutes). This is needed to bring everyone up and back to normal. We do apologize for the problems. If you have any questions feel free to contact me directly. jreinis@theplanet.com Thank You
Posted by Mans, 09-24-2003, 06:29 PM
My server has been unreachable for THREE (3) hours!! THIS IS TOOO MUCH TP!!
Posted by ElysiumNet, 09-24-2003, 06:36 PM
just called infact, they said a network issue and thats as specific as they described it, they said that they werent told the real issue so thats what he told me ...
Posted by 123finder.com, 09-24-2003, 06:48 PM
Mine is still down. I was just doing some testings with the new server and this outtage makes me reconsider my server choice to ThePlanet, RackShack somehow is still very stable with its network. We need an explanation why it happened and what ThePlanet will do to prevent such problems.
Posted by trustedurl.com, 09-24-2003, 06:50 PM
Still down... want to hear something funny? We were using this server to monitor our servers on top of alertra... Sigh.
Posted by runboy27, 09-24-2003, 06:54 PM
Yourhost Why is it that both you and the website refers to the problem as only being a problem on the Server Matrix segment ? My servers are on the "The Planet" segment and has also been down.
Posted by ThePrimeHost, 09-24-2003, 06:54 PM
The router reboot is supposed to be between 1700-1800 CST (that is 6 more minutes). Everyone cross your fingers. Best Regards, -Darrell
Posted by Mrdredd, 09-24-2003, 06:55 PM
Is anyone in the 69.56.133.*** down? My box there is down now, but it wasnt before.
Posted by TotalChoice - Bill, 09-24-2003, 06:55 PM
Wonders same
Posted by TMX, 09-24-2003, 06:56 PM
If I had to guess, I'd say that they were probably an hour or so away from being ready to reboot the router when they posted that notice. -B
Posted by nogi, 09-24-2003, 06:57 PM
Yep, it took appx 5 minutes. But the other server is still down for about 3 hours now. John
Posted by TotalChoice - Bill, 09-24-2003, 07:03 PM
Looks like they missed the window.
Posted by frozen, 09-24-2003, 07:06 PM
They rebooted it at 43 after the hour and was back up at 56 after.
Posted by lastminutehosting, 09-24-2003, 07:07 PM
Well im still down here 69.56.179.***
Posted by ideafusion, 09-24-2003, 07:07 PM
and down again
Posted by TotalChoice - Bill, 09-24-2003, 07:08 PM
Were still down as well, so much for the router reboot fix all trick.
Posted by frozen, 09-24-2003, 07:08 PM
Trying again, it appears.
Posted by nogi, 09-24-2003, 07:10 PM
I am actually a refugee from another data center where we experienced outages regularly for about 2 months. And now, exactly the same - arrggh! I hope that this is an isolated incident. John
Posted by 4web-space, 09-24-2003, 07:11 PM
This is our 3rd set of downtime this week with theplanet its really starting to effect our business and the excellent reputation we have built up over the past years Robbie
Posted by TotalChoice - Bill, 09-24-2003, 07:13 PM
lovely
Posted by ThePrimeHost, 09-24-2003, 07:14 PM
still down here as well.....
Posted by brav0, 09-24-2003, 07:19 PM
Down again. I wonder why they don't post an announcement at the Orbit. I thought my server was down and enetered a reboot request, after getting no response for half an hour I called them and only then they told me it was on their end and were working on it, they wouldn't provide any details though. Hopefully they straighten this out soon. Just came back up as I was posting.
Posted by HostingScene, 09-24-2003, 07:19 PM
Methanks god for having servers spread out amongst other datacenters. This is ridiculous though, there is the need for some BIG time explaining!
Posted by htremblay, 09-24-2003, 07:20 PM
Up again... Hope it will stay up for a while...
Posted by ThePrimeHost, 09-24-2003, 07:20 PM
Same here. I go with the "eggs in basket" theory. Kind Regards, -Darrell
Posted by TotalChoice - Bill, 09-24-2003, 07:21 PM
Still down...
Posted by Mans, 09-24-2003, 07:26 PM
i think tp/sm should change their sla to 77.7% after this
Posted by TotalChoice - Bill, 09-24-2003, 07:28 PM
How about a update D' Planet !
Posted by HostingScene, 09-24-2003, 07:31 PM
Back!!
Posted by TotalChoice - Bill, 09-24-2003, 07:31 PM
Were up!
Posted by lastminutehosting, 09-24-2003, 07:32 PM
Not me yet
Posted by nogi, 09-24-2003, 07:32 PM
I go with the "eggs in basket" theory Me to - I can only imagine how the help desk would have looked if all servers were in one data center John
Posted by BaddaBing, 09-24-2003, 07:33 PM
Still Down
Posted by TotalChoice - Bill, 09-24-2003, 07:33 PM
and were back down again.
Posted by lastminutehosting, 09-24-2003, 07:36 PM
Glad mine stays down atleast I dont get some hope then fails again
Posted by TotalChoice - Bill, 09-24-2003, 07:36 PM
LOL
Posted by HostingScene, 09-24-2003, 07:36 PM
DAmn, down again
Posted by TotalChoice - Bill, 09-24-2003, 07:39 PM
Were back!
Posted by lastminutehosting, 09-24-2003, 07:39 PM
Back
Posted by BaddaBing, 09-24-2003, 07:39 PM
LOL
Posted by Eric M, 09-24-2003, 07:40 PM
Been down the entire time.
Posted by TotalChoice - Bill, 09-24-2003, 07:40 PM
and down
Posted by lastminutehosting, 09-24-2003, 07:40 PM
down again the horror
Posted by ThePrimeHost, 09-24-2003, 07:41 PM
Were in the same boat. Been down for over 3 hours. Lovely.
Posted by ElysiumNet, 09-24-2003, 07:42 PM
this sure isnt going to make our customers happy
Posted by ThePrimeHost, 09-24-2003, 07:46 PM
No kidding.
Posted by thalaric, 09-24-2003, 07:46 PM
I've been down since about 12:00 PST, which makes it almost 5 hours for me. Anyone else? 69.56.188.*
Posted by mark1hos, 09-24-2003, 07:47 PM
Ours have not been up ONCE! Been down for over 4 hours now! Its funny we submitted a support ticket but also an outage report and they responded with a 'copy and paste' statement: "We are currently having a network issue that may prevent you from accessing your server. We will update this ticket when this has been resolved. Please accept our apologies for any inconvenience." HA! I have also asked them when I can expect to receive a refund as accordingly to their site they offer a 99.9% SLA! No response to this one yet...... The only good thing is that we have not moved clients over yet nor placed our own site on their yet! (phew!) Chris
Posted by Alex042, 09-24-2003, 07:48 PM
I noticed a few minutes ago it looked like a tracert revealed the path cycled a couple of times between 2 ips at theplanet then timed out. Oh lookie, I just got an email from my server.
Posted by TotalChoice - Bill, 09-24-2003, 07:51 PM
well were up again...
Posted by Eric M, 09-24-2003, 07:52 PM
Both of my servers are back up. First time in over 4 hours. *crosses fingers*
Posted by thalaric, 09-24-2003, 07:52 PM
I'm up again also. EDIT: I shouldn't say again since this is the first time.
Posted by HostingScene, 09-24-2003, 07:53 PM
back up....after 4 hours down
Posted by lastminutehosting, 09-24-2003, 07:53 PM
Back up now 4 hours and 30 minutes later yippie
Posted by Alex042, 09-24-2003, 07:54 PM
Interesting, the route changed.
Posted by ThePrimeHost, 09-24-2003, 07:54 PM
Wow. Up for the first time in hours!!!!! Woo Hoo!!!!
Posted by TotalChoice - Bill, 09-24-2003, 07:55 PM
4:39:43 of downtime today. Ouch....
Posted by BaddaBing, 09-24-2003, 07:57 PM
Im still down Edit: it's up
Posted by Alex042, 09-24-2003, 07:57 PM
How does that downtime fit into y'all's SLA to your customers? At least I don't have to resolve a server issue this time cuz it looks like the server itself stayed up.
Posted by BaddaBing, 09-24-2003, 08:06 PM
My Server is sorta up lol it's like 25% up 75% it's down right now
Posted by Mans, 09-24-2003, 08:07 PM
up here :||
Posted by nogi, 09-24-2003, 08:07 PM
Glad mine stays down atleast I dont get some hope then fails again Lol, it's refreshing to meet a real optimist John
Posted by 1AsianChic, 09-24-2003, 08:08 PM
it's been up for a while already
Posted by HappyManEd, 09-24-2003, 08:15 PM
I have been down scense around 12:00 PST and has not come up yet. I have the remote reboot port, but alot of good that has done me :-) Its 5:20 PST now
Posted by BaddaBing, 09-24-2003, 08:21 PM
Your not back up yet dang that has to suck
Posted by ElysiumNet, 09-24-2003, 08:33 PM
We have been back up for awhile now.
Posted by LiveRack, 09-24-2003, 08:37 PM
One of my 2 servers at thePlanet came back about 90 minutes ago and the second finally came up about 10 minutes ago. In total, there was 4 hours and 49 minutes of downtime. Very dissappointing to say the least. Fortunately, my main client servers weren't there.
Posted by BaddaBing, 09-24-2003, 08:40 PM
Dang when I said this my server went down
Posted by lastminutehosting, 09-24-2003, 08:41 PM
Bummer back down again
Posted by TotalChoice - Bill, 09-24-2003, 08:44 PM
yep were down again as well.
Posted by LiveRack, 09-24-2003, 08:46 PM
Super. Down here too.
Posted by nogi, 09-24-2003, 08:48 PM
The servers are up but one of them are very slow. A traceroute shows *** for the last 5-6 hops. John
Posted by Mans, 09-24-2003, 08:53 PM
Hahahahah D O W N A G A I N Bahahaha
Posted by TotalChoice - Bill, 09-24-2003, 08:59 PM
This is a becoming a joke!
Posted by DrCool, 09-24-2003, 09:01 PM
it appears my server is accessible once again... but it's going slower then usual.. and all my pings come back empty.. also the tracert seems to be boucning around: 6 dfw5-core3-p5-0.atlas.algx.net (165.117.69.125) 34.549 ms 34.313 ms 34.270 ms 7 dfw5-core2-p6-0.atlas.algx.net (165.117.48.125) 34.329 ms 34.407 ms 34.280 ms 8 dfw5-edge1-pos7-0.atlas.algx.net (165.117.53.2) 34.164 ms 34.086 ms 34.040 ms 9 theplanet-gw.customer.algx.net (206.181.190.10) 34.181 ms 34.302 ms 34.228 ms 10 ge-0-1-0.ibr4.dllstx2.theplanet.com (12.96.160.233) 35.178 ms 35.323 ms 35.129 ms 11 core0-v1.dllstx1.theplanet.com (12.96.160.8) 41.825 ms 42.498 ms 42.717 ms 12 dsr1-v1.dllstx2.theplanet.com (12.96.160.9) 42.273 ms 43.430 ms 43.017 ms 13 core0-v1.dllstx1.theplanet.com (12.96.160.8) 50.401 ms * * 14 * dsr1-v1.dllstx2.theplanet.com (12.96.160.9) 222.685 ms * 15 core0-v1.dllstx1.theplanet.com (12.96.160.8) 60.257 ms 57.607 ms 58.998 ms 16 * dsr1-v1.dllstx2.theplanet.com (12.96.160.9) 58.076 ms 57.389 ms 17 * core0-v1.dllstx1.theplanet.com (12.96.160.8) 64.007 ms * 18 dsr1-v1.dllstx2.theplanet.com (12.96.160.9) 65.870 ms 66.332 ms 63.642 ms 19 * core0-v1.dllstx1.theplanet.com (12.96.160.8) 70.971 ms 74.360 ms 20 * dsr1-v1.dllstx2.theplanet.com (12.96.160.9) 71.580 ms * 21 * core0-v1.dllstx1.theplanet.com (12.96.160.8) 82.587 ms 81.043 ms 22 * * dsr1-v1.dllstx2.theplanet.com (12.96.160.9) 82.848 ms 23 core0-v1.dllstx1.theplanet.com (12.96.160.8) 90.919 ms 93.391 ms * 24 dsr1-v1.dllstx2.theplanet.com (12.96.160.9) 89.218 ms 91.394 ms * 25 core0-v1.dllstx1.theplanet.com (12.96.160.8) 97.834 ms 96.780 ms 99.772 ms 26 * hops 10 -25 are just back and forth, back and forth.. Dr Cool
Posted by TotalChoice - Bill, 09-24-2003, 09:03 PM
Looks just like my first tracert from 5 hours ago!
Posted by HostingScene, 09-24-2003, 09:03 PM
ahhhh
Posted by binaris, 09-24-2003, 09:11 PM
Running fine here... is this isolated to DLLSTX1? We're in DLLSTX2 and haven't had any problems today.
Posted by EKR, 09-24-2003, 09:12 PM
You know there's a new forum for Network Outages and Updates, right? (Sorry if someone pointed this out already...)
Posted by FHDave, 09-24-2003, 09:12 PM
I have a friend whose servers on DLLSTX2 and he was affected as well. Looks like it's back up normal now.
Posted by TotalChoice - Bill, 09-24-2003, 09:15 PM
We have servers both on tx2 and tx1 and they are all down atm
Posted by binaris, 09-24-2003, 09:18 PM
Any in the 64.x ?? We have our own routers, etc. but this looks like a core problem. And yes, you can make claims on the SLA. We have several times before. TP was great until about a month after they opened Server Matrix.
Posted by Nessun, 09-24-2003, 09:23 PM
still no problems here good luck to you others :/
Posted by TotalChoice - Bill, 09-24-2003, 09:32 PM
Well they suck atm. Forget about the SLA, what about my lost monthly revenue.
Posted by Justin Reel, 09-24-2003, 09:42 PM
That's what I'm thinking too. I just promised all of my clients that things would be much improved after the move to theplanet/servermatrix, and all this happens not even a month after we move. Now I've got a bunch of unhappy people to explain myself to.
Posted by TotalChoice - Bill, 09-24-2003, 09:45 PM
Thing is tell us whats going on. Either here or in email or somewhere. Most clients will understand if you tell them whats going on. Looks like we are back up.
Posted by Shingen, 09-24-2003, 09:46 PM
Been down for at least 6 hours so far today. This is ridiculous, and no serious statement from them.
Posted by 1AsianChic, 09-24-2003, 09:56 PM
thanks god, mine is still up
Posted by LiveRack, 09-24-2003, 10:06 PM
Been over 5 hours for one of my servers. This is getting pathetic.
Posted by lastminutehosting, 09-24-2003, 10:07 PM
Mines up for now but o well we will see later. You guys need to calm down not like tp did this on purpose things happen.
Posted by LiveRack, 09-24-2003, 10:12 PM
True it was probably not forseen or on purpose but over 6 hours of downtime is enough to get anyone upset. Money is literally going down the drain for every hour that a server is down for most people.
Posted by Shingen, 09-24-2003, 10:20 PM
It's understood 'things' happen, that doesn't concern me too much. But they're doing a bad job of communicating with customers right now. i.e. what is wrong, why, and how long until it's *actually* fixed? That's just common sense customer relations. My server was down for 6 hours, and all I got from them was a lame 'we are experiencing technical difficulties' email. Normally they've been very good about communication; this is the time where that should really shine, yet it isn't.
Posted by ruiner, 09-24-2003, 10:20 PM
3 of my servers are back, but one is still down :\
Posted by YourHost, 09-24-2003, 10:33 PM
At approximately 2:00PM Central Standard Time, a core ServerMatrix Cisco Series 6500 switch rebooted and service to 67 customer switches lost connectivity. Engineers were notified of the problem within 1 minute and were deployed to resolve the issue. After several reboot attempts on the core switch, Engineers located a faulty FE blade with 19 customer switches connected. The blade was removed from the router and service was restored at 3:30PM Central Standard Time for the remaining 48 customer switches. A hot swap Cisco 6500 switch was deployed to move the remaining 19 customer switches to. As the config on the new switch was being loaded, Engineers worked with programmers to remove VLANs from the current switch to restore on the new switch. At 6:21PM Central Standard Time, all service was restored to all existing VLANs and traffic resumed to normal levels. Secondary IP ranges and static routes were complete as of 9:00PM and all service is now 100% normal. Engineers continue to monitor the process and no further outages are expected.
Posted by lastminutehosting, 09-24-2003, 10:35 PM
Thank you MR Reinis hopefully calm some of these people down now. It was a inconvience but however I still am happy with service. Thank you SM for getting us all back good work.
Posted by Shingen, 09-24-2003, 10:36 PM
Nice. Thank you Jeff. For me, that makes all the difference. Always great to know why.
Posted by LiveRack, 09-24-2003, 10:43 PM
Not quite 100% apparently because one of my servers is down and the SM forums show that a number of others are still having the same problem. I agree with Shenigen though--it's nice to finally get an explanation.
Posted by HostingScene, 09-24-2003, 11:13 PM
How are refunds going to be handled for this?
Posted by ElysiumNet, 09-24-2003, 11:16 PM
one of ours have been down for 6hours now
Posted by inteltechs, 09-24-2003, 11:16 PM
one of my servers just went down again. Today isn't a good day for sm's customers.
Posted by anon-e-mouse, 09-24-2003, 11:28 PM
Testing fix.
Posted by frozen, 09-24-2003, 11:35 PM
Can probably be merged with this http://www.webhostingtalk.com/showth...hreadid=190328 and this http://www.webhostingtalk.com/showth...hreadid=190343 hehe
Posted by motl, 09-25-2003, 12:09 AM
We were down for almost six hours.... we've been up since around 9ish.
Posted by LiveRack, 09-25-2003, 12:09 AM
Not a good day for SM either. Midnight right now and one of my servers have been down for 7+ hours now.
Posted by inteltechs, 09-25-2003, 12:12 AM
oh boy 7 hours? Did you call them and find out what is going on with your server?
Posted by motl, 09-25-2003, 12:15 AM
try calling them..... we placed almost 10 calls to them over the many hours
Posted by LiveRack, 09-25-2003, 12:17 AM
Sure did. Called multiple times. Also have had an outage report support ticket sitting there for the last couple hours too. One of my servers there came up a couple hours ago but the other one is still down (literally over 7 hours). Calling just gives me a "our techs are working on it" or "our network engineers are looking into it."
Posted by StartAnISP, 09-25-2003, 01:46 AM
We had 3 servers out of commission for over 4 hours about 2 weeks ago. They claimed a switch problem then too..
Posted by LiveRack, 09-25-2003, 01:49 AM
That's what they're claiming this time too...except this time, I've had almost 9 hours of downtime so far with the server still being down after multiple calls and support tickets.
Posted by DrCool, 09-25-2003, 01:57 AM
It appears everything is back to normal.. my server is up, and the network is now responding to pings and tracerts normally... overall I'm very impressed with how The Planet has handled this issue.. the only thing I think that could have helped things would have been Jeff's explaination early on.. so we knew why it was taking so long... Dr Cool
Posted by StartAnISP, 09-25-2003, 02:02 AM
We had some servers up and some down. Then the ones that were up would go down and the ones that were down would go up and so on. The whole time even the ones that were up were basically unusable. Could not load web pages from that. So, when I say up I mean barely responding to a ping. We have lots of backup servers in place in different areas of the country but this still caused us a ton of hassles and we could have done without it. We are for the most part very happy with them but between the problems we experienced a few weeks ago and now this. I am starting to wonder if I want to keep adding servers with them or not. It's a little scarey when they allow dedicated customers to be down for periods of time as long as these. They had a good thing going, I hope this is not the result of some executives greed. Sounds like some serious growing pains to me. I hope they work out the kinks so I can get back the confindence I once had with them.
Posted by StartAnISP, 09-25-2003, 02:04 AM
You are impressed with more than 10 hours of downtime this month? Wow, I wish all my ISP's were as tolerant as you I am not bashing them and I was not down today either (see a few post above) but I would not say I am impressed at this point.
Posted by innova, 09-25-2003, 02:06 AM
Having experienced nearly a 9 hour shortage, I am going to give them the benefit of the doubt. We have been a customer for 6+ months now, and have nothing bad to say until today. However, it makes me wonder, What is the contingency planning? What will happen next time to avert this again? I admit to knowing little about routing hardware, preferring to leave that to who know it best, but... Shouldnt there be some kind of failover, hot-spare type redundancy? I think how TP/SM handles that last point will really define how the future turns out. I am betting they will not go the way of Nocster, etc, due to their HONESTY and normally fast support. One thing I am left with is being impressed with Jeff Reinis's posts on the matter, both here and their forums. He did not sugarcoat, just said it like it was and didnt run for cover when people got angry at him / the company. He was apologetic without trying to pacify anyone. Good job to the techs and support staff who handled this one. Lets hope TP/SM implements preventative measures to ensure this doesnt happen again.
Posted by LiveRack, 09-25-2003, 02:16 AM
The communication from ServerMatrix has been virtually nil. The least they can do when a server (like mine) is down for almost 9 hours straight is tell us what's going on and keep us updated on what is happening. I literally had to make so many calls and so many support tickets till they finally got my 2nd server there back up (came up about 3 minutes ago (2:15AM) after going down at around 4:30pm EST yesterday).
Posted by DrCool, 09-25-2003, 02:50 AM
actually the real test will be if they honor thier SLA of 99.9% uptime.. cause as of right now, our server was down a total of 6 Hrs. which would breakdown to approx. 99.1% uptime (24 hrs in a day X 7 days in a week X 4 weeks in a Month) on average so aprox. 672 Hrs. in an average month - 6 Hrs. of downtime = 666 Hrs. of uptime take the new number (666) and divide by hours in an average month (672) = .99107 then convert to percent (move decimal 2 places to the right) 99.1% uptime for this month (so far) which would be .8% lower then thier SLA Guarantee.. Dr Cool
Posted by bloom, 09-25-2003, 03:45 AM
My server was "only" down for a little under 2 hours before they rebooted the router. Since then it's been mostly down - only up for a few minutes at a time. The server itself has been up, of course, the entire time. It's just that no one can get to it. Now the DNS lookups from here are failing for my domains. Refunds for their SLA wouldn't amount to much - from my understanding they would simply be required to compensate us for the downtime - even if they refunded the entire day, that still only amounts to around $5 for most customers. I lost more than just for being down an hour.
Posted by Nime, 09-25-2003, 05:51 AM
That's from the http://www.servermatrix.com/about_us.html Nothing explicitly said about number of dollars off the bill...
Posted by Nessun, 09-25-2003, 06:18 AM
I personally have been hearing quite a bit of bad things about SM since they opened up the SM section I kind of think it would be a good idea for them to sell what servers they have in stock and then stop selling a little while so they can get there things straightened out. I was about to purchase 2 new servers in the next 3 days and now im very weary. Although my server did not go down any today it still makes me nervous to order more...
Posted by HostingScene, 09-25-2003, 06:54 AM
I am interested in what they are going to do...
Posted by ElysiumNet, 09-25-2003, 07:01 AM
I hope they make alot of improvements, because i liked them and still like them but if there is 1 more downtime like this, ill have to reconsider our providers..
Posted by Alex042, 09-25-2003, 07:55 AM
This is what I got too, but this IP set was apparently removed at one point and the trace went through so must've been when they did the temporary swap. The telecom division of a company I am currently working with designs their networks to be redundant so in case of this kind of issue, a piece of hardware could be removed or a line could be cut and the trace would automatically reroute through another path and would go through. I guess they don't have the same redundancy plan.
Posted by roccol, 09-25-2003, 08:46 AM
Who has this type of redudancy? I have a test server at the server matrix. I am done, I want a redudant network with fail safes, I will pay more.
Posted by TMX, 09-25-2003, 09:31 AM
Assuming a direct correlation between the amount of downtime below the SLA and the amount of $$ one would be compensated, a $200/mo server would get you exactly one dollar and sixty cents of downtime credit on a six hour outage (assuming your math is correct, I didn't check it). People have an understandable reason to be upset about the downtime, but the way some are getting worked up over what amounts to chump change is pretty silly. don't you think? IMO, several contributors to this thread have wasted several times that while incessantly making "up again, down again" posts. -Bob
Posted by BigJ406, 09-25-2003, 09:34 AM
Are you guys still down? Mine was down for more than an hr yesterday, and that's it
Posted by Alex042, 09-25-2003, 09:39 AM
I'm not sure what public network does this. The group I work with has their own private national network. They have a multi-million dollar annual budget to make network upgrades. Don't most providers usually offer more than that in compensation? That little of a compensation hardly makes any guarantee worthwhile so what would be the point in even offering a guarantee unless there's more compensation?
Posted by trustedurl.com, 09-26-2003, 12:37 AM
Even the planet does. They made a fair compensation that I can live with. And it's more than the $1 people mentioned. I'm giving them another month before I make up my mind.
Posted by Qizeny, 09-26-2003, 04:01 AM
Here's the in-depth description of the failure and steps taken. http://forums.servermatrix.com/viewt...ml?p=2226#2226
Posted by OverlordQ, 09-26-2003, 04:09 AM
Nice description, not quite as easy as "plug it in and turn it on" that I bet a few people here expected.
Posted by SaudiOnMoon, 09-26-2003, 06:59 AM
Are servers still down guys? mine is still!
Posted by nogi, 09-26-2003, 09:15 AM
Here we go again. One of my servers are down again. Didn't they fix the problem fully or am I the only one experiencing this? John
Posted by frozen, 09-26-2003, 09:25 AM
My servers are on. try calling them, sure people say thay don't want to waste their time just to get a "we are working on it" answer, but the problem could be something completely different only effecting you. Posting on here that it is still not up is not going to get it back on alone.
Posted by LiveRack, 09-26-2003, 09:32 AM
None of our servers there are having problems now either. You should definitely call.
Posted by nogi, 09-26-2003, 09:43 AM
My message here were not a request for support. Off course I've contacted them before posting here to see if anyone else experienced anything similar. They have already responded that they can view the server in a browser. I can't access it via WHM and SSH though, at the same time a traceroute shows no packet loss John
Posted by nogi, 09-26-2003, 09:45 AM
I must say that servermatrix are responding very fast to outage reports. I received two responses within 10 minutes. Thumbs up. The server is being rebooted now. John
Posted by frozen, 09-26-2003, 09:46 AM
Try a reboot request instead of a trouble ticket, stating the box is pingable, but you cannot access its services.
Posted by nogi, 09-26-2003, 09:47 AM
Ok, thanks, that's what they're doing now John
Posted by frozen, 09-26-2003, 09:59 AM
great now to remember what I needed to ask them as well. lol
Posted by HostingScene, 09-26-2003, 02:11 PM
How much % credit did you all get? We got 40%....
Posted by inteltechs, 09-26-2003, 02:13 PM
interesting. How did you get it?
Posted by HostingScene, 09-26-2003, 02:21 PM
Submitted a support ticket from within Orbit to the accounting department.
Posted by TotalChoice - Bill, 09-26-2003, 04:41 PM
The reboot tickets we entered took one hour.
Posted by trademe, 09-26-2003, 10:39 PM
Ring the tech support hotline, and request a reboot.
Posted by rusko, 09-27-2003, 04:34 PM
welcome to the paul no spin zone. running a 7000+ server network without vlans would be literally impossible/insane. dont try to dress it up as a benefit to customers (which it is), since you need to do that anyway. its like saying that you use managed switches for the customers' benefit. mistakenly provisioning a load of 'theplanet' customers on servermatrix handoffs? uhuh, right. add to this some horror stories ive heard privately and am not going to relay here and the verdict is clear as day. i was considering colo'ing an offsite box with theplanet (not servermatrix), but the spin you put on what should have been a 'we are very very sorry' post turned me off from that idea permanently. paul
Posted by YourHost, 09-27-2003, 05:29 PM
Show me where I posted that. I am very interested in seeing this post from me. Could have been my evil twin or could be someone putting words in my mouth.
Posted by Shiek, 09-27-2003, 06:15 PM
Jeff's evil twin Awesome. I met that "evil twin" a couple of times, though the "good" Jeff always pops up Last edited by Shiek; 09-27-2003 at 06:26 PM.
Posted by rusko, 09-27-2003, 06:49 PM
actually, my mistake. it was posted by William Chernock, your VP of Engineering in this thread: http://forums.servermatrix.com/viewtopic.html?p=2226 it was my mistake. i assumed, naturally, that you wouldnt let your techies comment on the business side of issues, such as per-customer costs etc. the point still stands. paul
Posted by serve-you, 09-27-2003, 10:39 PM
I think that you are missunderstanding the VLAN situation. As I'm sure you are aware, most "low cost" dedicated hosting companies, do not do private VLANs for each customer/server. A VLAN usually will consist of a rack or two of servers, occupied by several different customers. What servermatrix did, is put each server on it's own VLAN, to better segregate it from the servers around it. This is infact a benefit to the customers, and they do NOT need to do this. -Dan
Posted by StartAnISP, 10-03-2003, 11:53 AM
Here we go again.. Our servers at The Planet are experiencing 37% packet loss on average.. Servers are unusable AGAIN!!
Posted by lastminutehosting, 10-03-2003, 11:59 AM
Im fine here no loss
Posted by StartAnISP, 10-03-2003, 12:06 PM
Looks like it's back.. Feeeeuuuuu..
Posted by StartAnISP, 10-03-2003, 12:08 PM
They said it was some sort of attack but they blocked the IP's and all is well now.. Problems over there get me all knotted up now because of the recent events. They fixed it quick this time and I am a happy camper, at least for now
Posted by bloom, 10-06-2003, 10:09 PM
I noticed a little downtime tonight, too (in fact, access to the server is still a little slow) But it's not long enough to complain. And the server itself is up - so I don't know if it's The Planet or my DSL connection (or somewhere in between). I did notice my hits decrease for an hour (by about half) but that could be a fluke.
Posted by Asverse, 10-17-2003, 02:35 PM
How was everyone compensated for that 6+ hours of downtime? Also, to everyone thinking of leaving or not going with ServerMatrix: Yes, they have had some recent problems. But every place is going to have problems. Hardware fails, it's simply a fact. But you can guarantee NO other datacenter will be working as hard as SM will be to get you back up. If it took them over 6 hours, I can guarantee most places would have taken 18+ or more probably. I have experience with a lot of datacenters, including Rackshack. Granted, Rackshack does have nice network uptime, their spiraling bad support overweighs anything good about them. ServerMatrix has excellent support, most of the time less than 5 minute responses to problems. They have always been willing to help. I still, and I believe most here do, still recommend ThePlanet/ServerMatrix.