Network down at hivelocity [Merged]

Portal Home > Knowledgebase > Articles Database > Network down at hivelocity [Merged]

Posted by thomor25, 11-13-2005, 03:22 PM
anyone elses server down cause their network is down in a part of the datacenter?
Posted by RyanD, 11-13-2005, 03:24 PM
this is the wrong forum for this type of message, please use the host downtime / outage forum
Posted by dex101, 11-13-2005, 03:36 PM
My server is unreachable as well. I hope this is fixed soon but I am relieved that's its not something wrong with my server .
Posted by Alex Fernandez, 11-13-2005, 03:39 PM
Aye mine too, bit poor to have the whole network die. Consideringits Cogent & L3 BGP as well. But, I'm moving to LT end of month anyway (LT server up already), so I'm not that fussed really.
Posted by thomor25, 11-13-2005, 03:39 PM
yeah well after 45 mins on chat waiting for my reboot the tech finally tells me that it is a network problem after 1 or 2 cuss words and saying that if it was at liquidweb it would have been rebooted in 15 mins and not an hour cause its not hard to get up and push a button. well I have an unmetered box so.
Posted by nicmo, 11-13-2005, 03:41 PM
mine is down too
Posted by Alex Fernandez, 11-13-2005, 03:42 PM
I'm on metered box heh
Posted by Servax, 11-13-2005, 03:45 PM
Mines down as well, although they haven't admitted its an network issue to me. Sigh..
Posted by Rick-WHSuite, 11-13-2005, 03:45 PM
Just an FYI: For some reason hivelocity are having a network issue with a small part of their DC their support rep wont give me an ETA he just said the following: (Rick.M is me Vick.R is the support guy) Anybody else having this problem?
Posted by CNSERVERS, 11-13-2005, 03:46 PM
same here with my two servers.
Posted by thomor25, 11-13-2005, 03:46 PM
it would be nice for them to tell us what is exactly going on, the tech won't tell me anything, and its like pulling teeth to get info or an eta.
Posted by nicmo, 11-13-2005, 03:46 PM
Vick R.: Currently the section of the data center your server is in is having network issues Vick R.: Our bandwidth providers are currently working on the issue. IT will be fine asap. yes they have admited
Posted by saukwebsolutions, 11-13-2005, 03:47 PM
down for me as well.
Posted by ceridius, 11-13-2005, 03:47 PM
We are having a software issue on our DC2 core switch , our techs are working to resolve as quickly as possible.
Posted by Servax, 11-13-2005, 03:49 PM
Haha, yep I just talked to Vick as well.. Maybe we'll make it for the 99.9% Uptime Guarentee we got.. (Least I think we got one..)
Posted by thomor25, 11-13-2005, 03:49 PM
I wonder if we will be credited for this downtime or do they not credit accounts like liquidweb. Man I wish liquidweb had unmetered servers.
Posted by Rick-WHSuite, 11-13-2005, 03:50 PM
Is Vick.R the only support rep? If so he must have a lot of requests because it took him 5 minutes to answer my question.
Posted by thomor25, 11-13-2005, 03:51 PM
yeah i think he is, i got him on the line over an hour ago now I'm losing lots of money from this.
Posted by Servax, 11-13-2005, 03:52 PM
Yeah, hopefully we do get credited.. Guess only HiVelocity knows. And from what I've seen right now, it seems like he is the only one. Took him a while to reply/answer my questions as well.
Posted by Rick-WHSuite, 11-13-2005, 03:52 PM
Tell me about it - finaly actualy get the server going the way it should do and this happens...fab! Never mind - hopefully i wont be with them much longer.
Posted by thomor25, 11-13-2005, 03:52 PM
http://www.hivelocity.net/Network/nocStatus.asp damn thats not good.
Posted by Rick-WHSuite, 11-13-2005, 03:55 PM
Nothing there :O
Posted by thomor25, 11-13-2005, 03:55 PM
i don't know what it was by my server load was at 80.00 before I tried to reboot it and it locked up so could it be a possible attack? My server never goes above 3 and rigth befor ethe netowrk went down it was at 80.
Posted by Servax, 11-13-2005, 03:55 PM
Nope.. All blank for me as well.
Posted by Rick-WHSuite, 11-13-2005, 03:57 PM
Mine was super high too thomor - im guessing what ever it is, its not good
Posted by Servax, 11-13-2005, 04:00 PM
My server is an Dual Opteron server, was at 0.35 when its usually 0.01 or such.. So I dunno, possibly it was a network attack. Dunno for sure.
Posted by saukwebsolutions, 11-13-2005, 04:00 PM
I noticed all this morning my connection via remote was really bad, but I thought it was just me at the time. My website was also very slow to load again I thought it was me, but I guess I know why
Posted by thomor25, 11-13-2005, 04:02 PM
fkin great, I restarted apahe and it started going back down and then bam connection gone. Yeah I think they got hit. wtfmate. ugh its nice to find these things out from the actual ustomers then hivelocity themselves. I sent a message to vick about this thread but strage we have yet to have a response except in another thread in the down host forum. in other news I can't type cause my keyboard is stiff as well you know I need otg et a new keyboard.
Posted by Energizer Bunny, 11-13-2005, 04:06 PM
Hmm i see the site , so what is been talked about?
Posted by Alex Fernandez, 11-13-2005, 04:07 PM
So we are magically all in the same part of DC and somehow BOTH carrers go down (as if), and apparently L3 and Cogent are working to fix it. Explain to me how L3 & Cogent can do anything when its within the HV DC that the problem is? paidhosting, thier site is clearly in the "good" part of DC, customers go into the crappy part. Since: Thats from my LT server to my HV one. Love the L3 -> L3 ping there, but the Cogent -> Cogent ping is low. Last edited by Alex Fernandez; 11-13-2005 at 04:13 PM.
Posted by ceridius, 11-13-2005, 04:13 PM
This is not a carrier issue , this issue is a problem with the dc2 core switch. I appologize for the confusion, I have updated the level1 technicians.
Posted by swiserv, 11-13-2005, 04:14 PM
Looks like all my servers on 10 mbps are down, all 20 and 50 mbps are up.
Posted by ceridius, 11-13-2005, 04:15 PM
All 20 , 50 , and 100 mbps unmetered are in DC1
Posted by thomor25, 11-13-2005, 04:15 PM
interesting cause I'm in a 10meg rack but I'm at 20 and I was supposed to be moved to the 20meg rack and I have not been obviously.
Posted by thomor25, 11-13-2005, 04:19 PM
I was supposed to be moved to a 20mbit rack cause my bansdwidth was upped but now I am down cause I wasn't moved like I was supposed to. Nice I better get credited for all this mess cause I ain't happy and I was looking to get another server. Going on 2 hours of downtime now.
Posted by WebHostingNeeds, 11-13-2005, 04:38 PM
2 of my servers are down. Thanks God, it is not my server problem. Browsing at WHT is better than waiting for HV support ticket :-)
Posted by saukwebsolutions, 11-13-2005, 04:41 PM
by the way have they given a time to when they will be up, or a guesstimate
Posted by Apolo, 11-13-2005, 04:45 PM
Moved to Providers and Network Outages and Updates forum from Dedicated Servers forum. Edited: Threads related to HiVelocity have been merged. Last edited by Apolo; 11-13-2005 at 05:12 PM.
Posted by WebHostingNeeds, 11-13-2005, 05:10 PM
Just got a mail from Hivelocity
Posted by marmoset01, 11-13-2005, 05:16 PM
or am I the only one? Its been down for over 2 hours now... due to "network problems".
Posted by Whocares, 11-13-2005, 05:17 PM
-------------------------------------------------------------------------------------------------------- Attention: If you are unable to reach your dedicated server at this time it is likely due to a software problem we are experiencing on the DC2 core switch. We apologize for the inconvenience. Our network engineers are working to resolve this issue as quickly as possible. Thank you for your patience during this time. Please check our network status page at http://www.hivelocity.net/Network/status.aspfor updates. or please put in a trouble ticket at http://support.hivelocity.net Sorry for the inconvenience HiVelocity Support. --------------------------------------------------------------------------------------------------------
Posted by c0bra, 11-13-2005, 05:21 PM
That's too long outages with hivelocity in less than a month
Posted by RyanD, 11-13-2005, 05:22 PM
there is already a long thread in the provider downtime / outage forum. Please follow-up on it there and don't create duplicate threads
Posted by c0bra, 11-13-2005, 05:24 PM
Think I'll be cancelling my new server with them too many network issues but to expected really with the pricing you get what you pay for
Posted by c0bra, 11-13-2005, 05:25 PM
They should make that forum more visible I can never find it... I suppose that was done at the request of advertisers who don't want prominent network outage information to hurt their sales efforts.
Posted by WebHostingNeeds, 11-13-2005, 05:28 PM
http://www.webhostingtalk.com/showth...7&goto=newpost Yes, it is some what hidden :-)
Posted by marmoset01, 11-13-2005, 05:30 PM
I didnt realize such forum existed, lol.
Posted by marmoset01, 11-13-2005, 05:34 PM
My first day at HV Servers, and this happens. Not a good start at all....
Posted by Rick-WHSuite, 11-13-2005, 05:43 PM
I cant beleve this - nearly 3 hours now , if they dont offer some kind of compensation (and 10% off the bill certenly does not cut it) then im off to LT
Posted by marmoset01, 11-13-2005, 05:46 PM
Yah.... Tis total bs....Im losing traffic and money from ads...
Posted by Rick-WHSuite, 11-13-2005, 05:48 PM
Tell me about it, thats why 10% wont cut it - i have customers & advertising to keep going here - i know its only my little server but still!
Posted by thomor25, 11-13-2005, 05:57 PM
yeah well I've lost more then the cost of the server in the time its been down. THis is ot worth it to me. They told me they are waiting on a software patch from the vendor.
Posted by marmoset01, 11-13-2005, 06:00 PM
woooo.... so we can expect about a week of downtime?
Posted by Rick-WHSuite, 11-13-2005, 06:01 PM
Well i suppose as long as it isnt a micro$oft product it will only be another day as opposed to 5 years. I am going to be wanting a refund for this month + compensation - this is totaly unacceptable.
Posted by thomor25, 11-13-2005, 06:01 PM
same here I better get a month free cause of this.
Posted by marmoset01, 11-13-2005, 06:09 PM
Why do they need software update? It was working before... and something got screwed. Cant they put it back the way it was until this update arrives?
Posted by Servax, 11-13-2005, 06:09 PM
Yep, this is my FIRST month with them and this happens.. Great. Going to be requesting a full month's credit as well, this is crazy. ~3 hours and servers are still down.
Posted by thomor25, 11-13-2005, 06:12 PM
they said they were working on one of the blades in the switch and it fk'd up.
Posted by Rick-WHSuite, 11-13-2005, 06:17 PM
So because of some ****'s slip of the hand we are stuffed for a few days.
Posted by marmoset01, 11-13-2005, 06:25 PM
Its my first DAY! I moved into the server a few hours ago!I posted a message on my site, saying move was flawless with 0 downtime... and then BOOM, this bs happens.
Posted by c0bra, 11-13-2005, 06:29 PM
I'd say we'd be lucky to get 1% service credit for this outage. Definitely from my perspective nothing less than a full month service credit will be enough to retain my custom after this fiasco.
Posted by Rick-WHSuite, 11-13-2005, 06:30 PM
Looks like Hivelocity will be losing around 10-20 customers then Last edited by Rick-WHSuite; 11-13-2005 at 06:37 PM.
Posted by marmoset01, 11-13-2005, 06:33 PM
Profit loss from 30 customers for 1 month <<<<<<<<<< loss of 30 customers.
Posted by probonic, 11-13-2005, 06:41 PM
I'm in the same boat as you - signed up on Thursday. Not good at all. If it wasn't for the fact I got a great deal on the server and it's not mission critical for me, I would be thinking of moving away ASAP. However, I am willing to give them another chance. They just need to get this damn switch issue sorted...
Posted by thomor25, 11-13-2005, 06:41 PM
30*200=6000
Posted by c0bra, 11-13-2005, 06:44 PM
I can't believe anyone would be using any budget provider for anything seriously mission critical. I certainly would neither run a hosting business nor any site where every few minutes down = serious amounts of lost revenues at hivelocity or anyone else at this end of the market. Luckily I'm not losing money over this at all just time. And my time is too valuable to me to waste dealing with issues like this on a regular basis so I'll just shutdown our server with them.
Posted by dirtbag, 11-13-2005, 06:59 PM
WOW, you make enough in 3 hours to pay for your server each month? That's a helluva revenue stream to have running on a bargain-basement host. BOY, I wish I could make $30,000 a month with a $130 server!
Posted by Servax, 11-13-2005, 07:00 PM
Yeah, I too think that a too unbelievable
Posted by thomor25, 11-13-2005, 07:18 PM
the income is not even through out the day but its close in price it was a figure but there are other expenses I have but anyway its been down 4 hours, peak time when most visitors are on in the usa you got people in the west just waking up and people in the east just getting out of church this couldn't have happened at a worse time and on a weekend when most are at home. The thing is it should be on the dc1 switch cause its a 20mbit unmetered not a 10 but they don't hve space for adding antoher 20 to the 20 racks right now so my server is down cause someone goofed and cause osmeone didn't move my server when I upped the bandwidth.
Posted by marmoset01, 11-13-2005, 07:29 PM
Lets not debate thomor25's revenue, focus on the topic of this thread! Death to HV techs!
Posted by thomor25, 11-13-2005, 07:30 PM
:Werd: still no answer and its been 4.5 hours I'm not happy I better get a credit at the end of this tunnel.
Posted by c0bra, 11-13-2005, 07:39 PM
Apparently they are waiting on a software patch from the hardware vendor so you better ready yourself for a wait until tomorrow at the earliest.
Posted by xploitfan, 11-13-2005, 08:12 PM
My Server Is Down More Than 3 Hours Until Now . they tell me they have core network problem. Dont Believe A New Companny.it Is My Exprience
Posted by RyanD, 11-13-2005, 08:13 PM
thanks for your first post. However, it's in the wrong forum and there is already a large number of posts in the provider downtime forum <<< Mod Note: Thread merged again. >>> Last edited by SoftWareRevue; 11-13-2005 at 08:18 PM.
Posted by L3e, 11-13-2005, 08:13 PM
This is ridiculous. It’s been about five hours and they don’t even have an idea when it will be back up. As soon as things are back to normal I am requesting compensation for the price of one month. If they don’t give it to me I am just leaving, and even if they do, I am thinking twice. I have had a number of other networking problems besides this and those problems are not completely fixed either. Ridiculous…unprofessional…anything else you want to call it.
Posted by datruesurfer, 11-13-2005, 08:15 PM
Have you tried calling them? Theres a Phone # on their website to get to their support department.
Posted by Scott.Mc, 11-13-2005, 08:27 PM
Is this still going on or what? My newest server that is only 10mbps is down, and theres nothing on it yet. Did all servers have to be rebooted or somthing? I have no idea whats going on I was not available yesterday to even notice this. How ever all 16 other servers seem to be fine which are 100mbps. I understand what you guys where saying about the live chat guy he is pretty poor, infact virtually all the live chat people are very very poor and need to be trained, even talk to a sales guy they have no idea what you are talking about half the time. Vick always tells you wait because he doesn't know what somthing is. Myabe this is somthing Hivelocity should look at fixing because its pretty annoying if somthing goes wrong and you get landed with some idiot making assumptions. I will be talking to my account rep Kevin about credit too, even though its only got 1 domain running and even that is active yet, not the point though. Although I doubt I will be switching. Nobody is answering my support ticket to reboot, nobody is answering on live chat either. Please wait for a site operator to respond. All operators are currently assisting others. Thanks for your patience. An operator will be with you shortly. All operators are currently assisting others. Thanks for your patience. An operator will be with you shortly Pretty shoddy that I had to find out on here, but I bet if I was too sign up for a server they would answer pretty quickly.
Posted by SarcasticDwarf, 11-13-2005, 08:29 PM
Just to comment as the same things get said every time an outage happens with any company: 1) Making demands of "they must give me compensation or I am leaving" is both childish and unproductive. If you seriously desire this, wait until the issue has been resolved and then submit a ticket. Hosts are not going to be reading this thread and thinking "hey, I better send this guy a regund for a month". That is simply not how things work. 2) If you have managed to lose more revenue than the cost of the server for the entire month, then you probably need to change providers to one with a more redundant infrastructure. Sorry, but you won't be getting much sympathy when you say that your $80/month server going down for five hours cost you $900 in revenue. 3) This is not the support center. Please keep this in mind.
Posted by c0bra, 11-13-2005, 08:32 PM
Why is the outage forum not accessible to guests?
Posted by Valman, 11-13-2005, 08:34 PM
Are they still down?
Posted by SarcasticDwarf, 11-13-2005, 08:39 PM
The answer has already been stated in this thread by HiVelocity. It is standard in almost any industry to divide support into tiers when a company has more than one or two employees. It is not worth it to any company to pay an expert technician to answer questions which consist mostly of things which can be resolved by almost any technician. A low tier technician does not always have access to the latest developments. Things generally filter down fairly quickly. Also, low tier support personel are not going to ask the owner about every single question that comes up. Again, fairly standard for sales and support to be two different groups. They are probably getting a huge number of support requests at the moment. Let me give an example of how hiring practices often work: A company can hire five technicians, which are usually answering questions say 70% of the time. In rare instances, such as a major hardware failure, the volume of requests increases around 200%. This might happen an average of every six months. When this happens, there is little the support technicians can do to fix it. Their work will mostly consist of giving updates to customers (this applies in every single industry). In order for the company to be able to immediately answer every question, they would need to double the number of people working. There would cause two things to happen. First, the costs are *greatly* increased for both the company and therefore the customer. Second, these ten employeed spend most of their day idle.
Posted by c0bra, 11-13-2005, 08:40 PM
Hello it's a discussion forum. If people want compensation, why is it a sin to discuss this on the WHT discussion forum? As has already been pointed out in this thread. As far as I can tell people have been discussing the issues as customers and trying to share what little info there is with one another. I don't think anyone here has confused this as a customer support forum or thread. We're just venting at the lack of information coming from hivelocity and the very vague status message they posted hours ago.
Posted by marmoset01, 11-13-2005, 08:51 PM
I want my mommy!
Posted by Valman, 11-13-2005, 08:55 PM
So they are still down?
Posted by RyanD, 11-13-2005, 08:56 PM
yes, they are still down
Posted by Scott.Mc, 11-13-2005, 09:03 PM
A level 1 tech should know what is going on and not make incorrect assumptions. Its that simple, theres no other arguing to the matter. You are saying it is ok for a level 1 tech to just make reasons up to satisfy people which is exactly what happened, and if you ask anyone else who has spoken to HV techs about something, I will bet its not the first time because I have experienced it myself where they just make things up trying to satisfy you. More than 3 people says about a problem with there server in a space of a few minutes, and you think nothing of it? Even sales staff should know what's going on, every single staff member should know what's going on with a issue thats affecting possibly hundreds of customers, and hundreds of thousands of people that attempt to visit the customers servers. If you cannot get through to a tech you try another department, you keep trying until someone answers you. So even if its a sales person they should be able to inform you what's going on. Again, the very fact they are split into groups is a well used business practice and brining it up is virtually useless. A tech guessed what was up with peoples servers and said something that was completely incorrect. He even blamed bandwidth providers. They can tell you what's going on and tell you that there is no ETA. How would I know hivelocity is down, they have not told me. If it was not for WHT I would not know a thing, nobody is answering and its downright annoying. I couldn't careless that its down, and it would not have been a problem if someone had told me from hivelocity, all it would have taken is someone to respond to my requests/tickets and said they where having problems and there would not be a problem. My question still has not been answered, Is hivelocity still down? yes or no Its evident you don't even know, and your reply is completely useless and does not help the situation. The status page might not have been updated, and its very easy to forget in a situation like this where the first thing you do is get things working. I have still yet to hear anything at all from hivelocity directly.
Posted by marmoset01, 11-13-2005, 09:24 PM
The 99.9% uptime guarantee has been breeched, since after 5 hours of downtime we are at 99.3%, they are legally obliged to compensate us.
Posted by Alex Fernandez, 11-13-2005, 09:25 PM
Well my server is still down, so I guess YES
Posted by dirtbag, 11-13-2005, 09:36 PM
Well I first heard about it in this mass e-mail to their customers. So really, all this FUD is unnecessary. -------------------------------------------------------------------------------------------------------- from cp.hivelocity.net (localhost [127.0.0.1]) by cp.hivelocity.net (8.12.11/8.12.11) with ESMTP id ************* for <********@*****.com>; Sun, 13 Nov 2005 16:10:45 -0500 Message-ID: <********.**************.JavaMail.cpanel@cp.hivelocity.net> Date: Sun, 13 Nov 2005 15:23:32 -0500 (EST) From: info@hivelocity.net Add to Address Book Add Mobile Alert To: ********@*****.com Subject: HiVelocity Network Status Update - 11/13/2005 3:20PM EST. Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Mailer: Content-Language: UTF-8 Content-Length: 543 Attention: If you are unable to reach your dedicated server at this time it is likely due to a software problem we are experiencing on the DC2 core switch. We apologize for the inconvenience. Our network engineers are working to resolve this issue as quickly as possible. Thank you for your patience during this time. Please check our network status page at http://www.hivelocity.net/Network/status.asp for updates. or please put in a trouble ticket at http://support.hivelocity.net Sorry for the inconvenience HiVelocity Support. --------------------------------------------------------------------------------------------------------
Posted by Scott.Mc, 11-13-2005, 09:39 PM
I have already sent an email to Kevin, I couldn't care about their SLA, I wouldn't have asked for credit if someone from HV had told me or answered my requests. Thats the whole issue, the fact that nobody from HV has told me a thing, I had to find out from WHT. I have gathered that now, again not the point, its very poor that we have to go on assumptions. I finally got through to someone on live chat after 2 hours+ of waiting. I asked why I was not informed and why they have not answered me at all, Rob T gave me some message about it must be fixed in the next hour and that they apologize. I said read what I asked, I am not interested that its down. He waited 5-6 minutes, closed the session. I am not having that, so I will be canceling for that very reason as soon as Kevin responds too me. Am I really asking for too much? Is someone from the company telling me, or even a mass email, really too much?
Posted by Scott.Mc, 11-13-2005, 09:41 PM
Its nice that you got an email.
Posted by Servax, 11-13-2005, 09:43 PM
As per the https://www.hivelocity.net/Network/guarantee.asp. Yeah, we are getting at least 10%.
Posted by RyanD, 11-13-2005, 09:46 PM
No you aren't, read that carefully it says that the guarantee only applies to virtual servers it says nothing about dedicateds.
Posted by marmoset01, 11-13-2005, 09:48 PM
And by the looks of it, it will be down for at least 24 hours.
Posted by Servax, 11-13-2005, 09:51 PM
Notice what page it is on, if they are willing to give out 10% credits/refunds for late tickets.. This doesn't apply? That page applied for every hosting service is what I am aware of as I asked Kevin about this before I signed up. Also, please notice this asvertising they had here: http://www.webhostingtalk.com/showth...ght=HiVelocity Last edited by Servax; 11-13-2005 at 09:54 PM.
Posted by Scott.Mc, 11-13-2005, 09:56 PM
I will vouch for Hivelocity that they do infact give credit. We got full credit $x,xxx without any real difficulties from Kevin the account rep, when Hivelocity did not configure the HardWare firewall properly and it was causing short periods of downtime.
Posted by thomor25, 11-13-2005, 10:01 PM
well I have called 4 times and everytime I get to 1 in the wait list I get my cal dropped. No person is answering the phone all 10mbit servers are down I was suppose dto be in the 20mbit reack but I am not and this proves it cause my server is downa nd I am supposed to be moved I have called 4 times and had to recall againc ause I was on hold for 20 mins and then my call was dropped 4 times after that after reahing the person who is answering.
Posted by Servax, 11-13-2005, 10:01 PM
I shall be talking to Kevin tomorrow morning when he starts his shift then
Posted by Sir Drake, 11-13-2005, 10:02 PM
HiVelocity has a datacenter of their own in downtown Tampa (the one shown in the pictures on their website) but it's very small and filled up pretty fast. Instead of building out more space of their own, which is expensive, they've begun to colocate a large number of servers a few miles away in Sagonet's datacenter. (presumably what they call their "DC2") Sago has been mostly offline since around 2:00 a.m. due to a core switching/routing software upgrade gone seriously wrong. If you want to read about the current disaster at Sago, check out thread number 460513 in this same forum, entitled "Sagonet / Allmanaged Down". I'd post a link, but this forum won't let new members post URLs until they have posted at least 5 times.
Posted by marmoset01, 11-13-2005, 10:02 PM
.......................
Posted by thomor25, 11-13-2005, 10:03 PM
I can't type I'm so livid I'm calling drew in the morning I cannot take this.
Posted by Servax, 11-13-2005, 10:05 PM
Link to the thread if anyone needs it: http://www.webhostingtalk.com/showthread.php?t=460513
Posted by marmoset01, 11-13-2005, 10:11 PM
I really hope they don't all go home once their shifts are over, and just leave this until tomorrow morning....
Posted by Servax, 11-13-2005, 10:15 PM
Looks like I got an EST from Sam, within 90 minutes.
Posted by marmoset01, 11-13-2005, 10:20 PM
*crosses fingers*
Posted by saukwebsolutions, 11-13-2005, 10:25 PM
Guys the threats wont do much, I think they know how bad it is, and I am sure they know something will have to be done for their customers. Accidents happen, its not like they said today, hey lets go take a bat to our servers and make our customers mad lol Lets be calm and know that I am sure they will do something at the end of the day for the customers
Posted by Valman, 11-13-2005, 10:32 PM
So Hivelocity just colos?? How can they lie about that when theyre really just coloing with Sagonet? Sure they have 1 datacenter but they arnt putting our servers in them, just in a colocated space at sago... Why didn't they just build out their space further instead of lying and being greedy?
Posted by Servax, 11-13-2005, 10:33 PM
Only DC2 is colocated, they have their own datacenter but I guess its pretty much filled up.
Posted by tomz17, 11-13-2005, 10:44 PM
My 10mbps server is down as well... It has been spotty since last night... Tech supports answered quickly, but has been incapable to provide a solution. I just got a response to my trouble ticket :: The DC2 switch is still down. We have technitians on site from the manufacturer. I am extremely sorry for this. It will be up ASAP. Please reopen this ticket if your machine is not back up in the next 90 minutes. -Tom
Posted by Scott.Mc, 11-13-2005, 10:48 PM
I got that aswell, you would think they would atleast get someone who can type properly to do the message. A simple read over would have spotted that, now they have probley sent it to hundreds of tickets.
Posted by Servax, 11-13-2005, 10:48 PM
Yep, so hopefully it means our servers will be up in ~70 minutes now (Since I got mine a while ago). Last night SSH was lagging and I kept on getting ping loss, so it looks like they tried to fix it up today and instead messed it up. The techs from the manyfacturer are there so we can only hope the best.
Posted by probonic, 11-13-2005, 10:56 PM
Well my server with HiVelocity is now up again and pinging absolutely fine - no packet loss and normal ping times.
Posted by tomz17, 11-13-2005, 11:01 PM
still down here... 10mbps unmetered -Tom
Posted by Servax, 11-13-2005, 11:01 PM
Mines still offline, congrats to yours coming back online probonic.
Posted by saukwebsolutions, 11-13-2005, 11:01 PM
it is still going up and down for me. oh well
Posted by c0bra, 11-13-2005, 11:07 PM
Still down here
Posted by l0ck_up, 11-13-2005, 11:09 PM
up & down for me as well
Posted by Scott.Mc, 11-13-2005, 11:11 PM
Just came back up for me there, lets see how long it stays. Since its a sago problem, I am still unsure as to why Hivelocity could not have just said that and why are they ignoring people, its not like they are the ones fixing the problem.
Posted by Servax, 11-13-2005, 11:13 PM
Mines back online as well, no packetloss so far (Pinging it 2005 times..).
Posted by marmoset01, 11-13-2005, 11:15 PM
Mine is up too... hurray
Posted by c0bra, 11-13-2005, 11:18 PM
Seems like it's something they don't want to discuss. I guess they prefer if people believe they are in full control of their operations and not relying on some other DC management team not to screw things up like has happened today. I can't believe this work was authorised on a Sunday when it was bound to be difficult to reach the required expert personnel to assist in resolving the botched upgrades. All in all an appauling lack of communication and an appauling lack of foresight from those who took the necessary decisions today in handling this whole fiasco.
Posted by CNSERVERS, 11-13-2005, 11:19 PM
both of my servers are back now.
Posted by dirtbag, 11-13-2005, 11:33 PM
As for people expecting a personal answer to every single person affected, what do you expect really? They posted here when they didn't have to, they sent a mass email to customers, they put info up on their network status page. What do you expect? Has anyone here experienced an outage affecting hundreds of users? I have. Do you know what it's like to have the phone ring off the hook incessantly with people wanting you to kiss their *** and appease their tempers for 10 minutes each while you try in vain to get something back up and running? I do. It's an impossiblilty.. especially when people don't get the answer they want to hear. They just go on and on and on with the same question, expecting to brow-beat you into the answer they want to hear. You guys are like a bunch of animals chomping at the bit thirsting for blood. Outages happen, and I don't care where it is... they happen. Cut the place a little slack eh? They are back up now so the rumors about they sky falling have been unfounded
Posted by Scott.Mc, 11-13-2005, 11:42 PM
dirtbag, ok so lets clear this up. The people that put food on your table are upset, so you inflame the situation more. Is that what you are saying?
Posted by c0bra, 11-13-2005, 11:46 PM
Don't feed the trolls To be honest we shouldn't really be complaining though. You can't expect top notch service and customer care at the prices we're paying and that's why I'm bailing out. Having to deal with bottom feeder providers is bad for my health and I'm going make a new years resolution to never do it again. :p
Posted by ceridius, 11-14-2005, 12:08 AM
We have resolved the network issues at the Hivelocity DC2 core switch and at this time your server should be without issue. If you are still experiencing any problems, please contact us immediately so we may resolve any open issues. Thank you for your patience during this period. To open a support request, please visit http://support.hivelocity.net
Posted by Valman, 11-14-2005, 12:26 AM
So I also take it that you are non redundant then?
Posted by ceridius, 11-14-2005, 01:09 AM
We deeply apologize to any customer of ours who experienced downtime today. As you already know, we have resolved the issue and those customers that were affected are back online. Our Account Managers will be in the office tomorrow morning bright and early to discuss today with each and every one of you. We value your business very much and appreciate your patience today. Please contact your Account Manager via email for a personal phone call tomorrow. If you do not know who your Account Manager is please email sales@hivelocity.net with your ip and contact information.
Posted by saukwebsolutions, 11-14-2005, 01:19 AM
Everyone and everything messes up in life. Things happen, lets relax and move on. If it happens again then people should not be happy, but like I said things happen and to think they will never happen is just wrong
Posted by dirtbag, 11-14-2005, 01:21 AM
What the heck are you talking about? What I said in a nutshell was basically that these things happen and there's no need to go overboard about it and act like Hivelocity is suddenly some kind of horrible evil company that should over compensate everyone for what they think is some kind of massive inconvenience. I'm passionate about this issue because I have worked as a sysadmin in large networks environments for 12 years now and I therefore have great sympathy to a provider's plight when things unexpectedly go wrong. People can become very vicious and absolutely unreasonable under such circumstances and it really isn't fair to expect a personalized response to hundreds of users. Oh nevermind, you'll never get it, you've already made up your mind. Carry on...
Posted by ceridius, 11-14-2005, 01:22 AM
Remind me to send you a christmas card.
Posted by HostBreak, 11-14-2005, 01:24 AM
I agree - Things can go wrong and not always you have control on them. Sometime you can fix the mess quickly while at times you cannot. I think instead of criticizing and getting impatient one should give positive suggestions and feedback to help the provider do better next time -- Yes, this can happen next time and we have to make the provider be better prepared for that. My 2 cents..
Posted by saukwebsolutions, 11-14-2005, 01:35 AM
Well like I said it happens to the best of them. Heck The Planet has it happen, shoot LEVEL 3 went down didnt it? You cant get better in life or in what you do unless you make mistakes and learn from them.
Posted by RyanD, 11-14-2005, 01:36 AM
Hopefully Sago will work to make things right with all parties involved.
Posted by Scott.Mc, 11-14-2005, 02:42 AM
Aguring about how these things happen, that is clear. As I stated to my account manager, I don't care that it was down, I do however care about how the situation as handled. I would have liked to have found out from Hivelocity instead of finding out on WHT. Although it seems they did send out an email, I must have not got it, however I did get an email about it being back up now. There are many problems about this whole situation, that NEED to be addressed by Hivelocity and I have no doubt that they will, but it still does not excuse it. While I understand alot of people are requesting help, I waited 2 hours on live chat only to get a message that was being sent to everyone, I don't need to be told my server is down, I am fully aware of that. All I wanted was for Hivelocity to tell me what happened, because at that time I did not know the network was still down since when I posted the last post regarding the server being down was marked as Yesterday. It also goes to show you that if you are getting the same staff member over and over again when requesting, how many actually where handling it. Why where there not staff there to handle the extra requests, long delays should not have happened, a delay yes, hours more than normal no. Especially when hivelocity where not even the ones fixing the problem. However theres no point beating a dead horse. I feel sorry for Kevin and the other account reps who will have to try salvage whats left of relationships. I have more respect for Kevin than to even bother asking him for ways to make me stay, he is a very nice guy and is one of the most helpful guys I have ever had the pleasure to meet.
Posted by Rick-WHSuite, 11-14-2005, 03:32 AM
Well after all that happened i've just emailed Steve regarding compensation for falure - also saying that performing work like this is not best done on a sunday afternoon when software venders are closed!
Posted by easyarabia, 11-14-2005, 03:57 AM
Too bad... My server is down again now.
Posted by HostBreak, 11-14-2005, 04:00 AM
Mine as well and this happened at the time when I was comforting an angry reseller! Kevin, this reseller is gone!!
Posted by dex101, 11-14-2005, 04:01 AM
My server is down again
Posted by CNSERVERS, 11-14-2005, 04:01 AM
seems down again? anyone else?
Posted by marmoset01, 11-14-2005, 04:02 AM
yup, down again.
Posted by cococrust, 11-14-2005, 04:02 AM
I guess mine is down also. Just created a ticket. Hopefully the problem'd be resolved soon. this is prob my 2nd major outage other than the last time when my server was having hardware problem. customers of the 20 and above mbit servers: are you guys having stable connections or is it also a bit spotty for the day??
Posted by easyarabia, 11-14-2005, 04:08 AM
server is up! downtime: 10 mins
Posted by cococrust, 11-14-2005, 04:23 AM
yeah. its up. I think I counted 15 mins. I hope this problem is not sporadic.
Posted by marmoset01, 11-14-2005, 04:28 AM
Any news on those compensations?
Posted by l0ck_up, 11-14-2005, 04:41 AM
My inbox is full of alert messages from my monitoring service, up & down....
Posted by CNSERVERS, 11-14-2005, 05:24 AM
seems... down again...
Posted by probonic, 11-14-2005, 05:26 AM
My server is also down again - I was hoping to wake up this morning to find everything still fine - What the hell is happening over there??
Posted by probonic, 11-14-2005, 05:37 AM
I got the following response from HV tech support:
Posted by probonic, 11-14-2005, 05:39 AM
I seem to be back up with 0% packet loss again. Wonder how long it'll last this time...
Posted by Rick-WHSuite, 11-14-2005, 06:48 AM
I had about 35 mins this morning - the issues are "ongoing" according to the reply to my ticket.
Posted by skyaus, 11-14-2005, 07:10 AM
Downtime total 9 hour(s) 5 min(s) 10 sec(s).
Posted by skyaus, 11-14-2005, 07:29 AM
do you guys get this email? -------------------------------------------------------------- Attention: If you are unable to reach your dedicated server at this time it is likely due to a software problem we are experiencing on the DC2 core switch. We apologize for the inconvenience. Our network engineers are working to resolve this issue as quickly as possible. Thank you for your patience during this time. Please check our network status page at http://www.hivelocity.net/Network/status.aspfor updates. or please put in a trouble ticket at http://support.hivelocity.net Sorry for the inconvenience HiVelocity Support. -------------------------------------------------------------------------------- My server is down today. Downtime total 9 hour(s) 5 min(s) 10 sec(s). and i got another email. ---------------------------------------------------------------------------- Attention: We have resolved the network issues at the Hivelocity DC2 core switch and at this time your server should be without issue.If you are still experiencing any problems, please contact us immediately so we may resolve any open issues. Thank you for your patience during this period. To open a support request, please visit http://support.hivelocity.net Hivelocity Support ===================================== but my server is still down. what the hell is going on?
Posted by l0ck_up, 11-14-2005, 07:29 AM
It's getting too long with up & down network...my server is down again, unfortunelly it's happened when I'm in the middle advertise my hosting, some potential clients might left me because of this
Posted by skyaus, 11-14-2005, 07:30 AM
i don't know what is the problem Last edited by skyaus; 11-14-2005 at 07:34 AM.
Posted by Energizer Bunny, 11-14-2005, 07:31 AM
Hi there is also another thread stating there downtime , so i guess lots of people were hosted with them , its like 2 4 posts below this post , stating something about there ongoing downtime, although there site works for me.
Posted by Rick-WHSuite, 11-14-2005, 07:31 AM
Yup mines gone down again too - this is total rubbish now, there is no excuse for not giving support or a reason! What the hell is up with them? IF they cant provide support - dont offer it!
Posted by skyaus, 11-14-2005, 07:35 AM
It's very frustrating.
Posted by gate2vn, 11-14-2005, 07:37 AM
and who is your provider? Did you contact them? I am not sure to understand why you have posted a thread likes this
Posted by Energizer Bunny, 11-14-2005, 07:38 AM
I think his provider is hivelocity.net as he has posted another thread about same problem
Posted by Energizer Bunny, 11-14-2005, 07:40 AM
Yes any sort of downtime is frustating , i get frustated even when i send reboot signal to server , so such long downtimes are headaches , and u get customers pming why site was down and hell breaks loose.
Posted by net, 11-14-2005, 07:42 AM
It is better to contact hivelocity. Is it their network, hardware problem or something you did in the server?
Posted by skyaus, 11-14-2005, 07:44 AM
i've already opened a ticket.. waiting for their reply
Posted by LBX, 11-14-2005, 08:07 AM
Did you open a support request like it says?
Posted by Alex Fernandez, 11-14-2005, 08:11 AM
That e-mail was after 3 hours of downtime, by which time this thread on WHT already existed. Nice to see it takes HV 3 hours to realise thier network is down Oh, and Sir Drake I dont really care if they are in Sago DC, in thier advert threads they claim to own thier own DC AND, saying - well its not our DC, so ... is not excuse. And I wasnt aware Sago had an L3 & Cogent mix (correct me if I'm wrong). saukwebsolutions, the e-mail came 3 hours after servers went down ... AND, when customers asked what was wrong initially they go wrong/no info (carriers being down - remember that?). Thats what people are upset about, about the 24/7 support taking time, and the information being provided being wrong. Last edited by Alex Fernandez; 11-14-2005 at 08:20 AM.
Posted by Rick-WHSuite, 11-14-2005, 08:13 AM
My server is back online (finaly), not quite sure how long it will last though.
Posted by skyaus, 11-14-2005, 09:44 AM
my server have down for more that 12 hours
Posted by ceridius, 11-14-2005, 09:48 AM
I have a tech looking at your server as we speak. If anyone else is having an issue please send in a ticket at http://support.hivelocity.net , I have extra staff on hand to deal with any lingering issues.
Posted by Sir Drake, 11-15-2005, 03:51 PM
Ok, I will correct you. First, HiVelocity does have their own datacenter on the 10th floor of the Lykes building in downtown Tampa. It's a VERY small facility and filled up pretty fast, so instead of building out more space of their own, they elected to colocate the overflow with Sago (who has plenty of room) a few miles away. Second, nowhere did I say to "excuse" HV because it's not their problem. My criticism with HV is for pretending the problem was in their OWN datacenter, with their OWN switch, and claiming they THEMSELVES were working on it, when in reality they were just waiting for Sago to fix it. So anyone whose server is in "DC2" has been lied to, in the context that your machine is not actually located inside HiVelocity's datacenter. Third, I never said Sago had Cogent bandwidth. They do not. Sago provides a loop between HV's colocation space at Sago (DC2) and HV's datacenter (DC1) at the Lykes building. Any connectivity for HV's servers originates from the circuits at DC1. When Sago had the meltdown, the only effect it had on HV, to the best of my knowledge, was an outage on the transport between DC1 and DC2. I believe the only customers affected are the ones whose servers are colo'd at Sago. I'm not aware of anyone being affected whose servers are physically at HiVelocity's facility in the Lykes building. Last edited by Sir Drake; 11-15-2005 at 04:05 PM.
Posted by Alex Fernandez, 11-15-2005, 05:56 PM
Hmm, odd that HV transit has to go through Sago core routers, you'd think they'd have own equipment in there and only rely on Sago power.
Posted by Sir Drake, 11-15-2005, 07:45 PM
I believe that at Sago's core, both Layer 2 and Layer 3 functions are performed by the same equipment. (Foundry switch-routers) So the Layer 2 transport between the Sago and HV facilities would have been affected, along with the rest of Sago's traffic.
Posted by Alex Fernandez, 11-16-2005, 04:49 AM
As I said, thats "odd", as you'd expect HV to use thier own equipment to limit reliance on Sago.
Posted by Sir Drake, 11-17-2005, 02:04 PM
They have no reservations about pretending DC2 is their own facility. According to other posts in this thread from people who have asked "do you colo?" and were told "NO", it shouldn't come as any surprise they might not own the equipment carrying the transport between facilities. If you've had the opportunity to go inside the HV facility as many times as I have, your perception of the "real" HV might be somewhat different than the impression you get based on what their website tells you. Last edited by Sir Drake; 11-17-2005 at 02:17 PM.