hivelocity.net down ?

Portal Home > Knowledgebase > Industry Announcements > Web Hosting Main Forums > Providers and Network Outages and Updates > hivelocity.net down ?

Posted by ttgt, 05-27-2011, 06:16 AM
Hi,

one of my server is unreachable,

i do tracert,it seems the node of hivelocity's router do not get response,

do you get the same issue?

thanx
Posted by kimboy, 05-27-2011, 06:18 AM
same issues here hivelocity's router do not get response, tried from different networks
Posted by tnedator, 05-27-2011, 06:25 AM
I can't reach my server or the support website. Clearly they have a network problem this morning.
Posted by twhiting9275, 05-27-2011, 06:28 AM
3 different networks, the same response all 3 times:
Code:
traceroute to hostname (ipaddr), 30 hops max, 40 byte packets
 1   (122.201.72.2)  1.130 ms  1.123 ms  1.156 ms
 2   (122.201.64.228)  1.443 ms  1.463 ms  1.527 ms
 3   (121.200.226.78)  4.498 ms  4.610 ms  4.603 ms
 4   (121.200.224.87)  2.395 ms  2.686 ms  2.777 ms
 5  216.6.124.145 (216.6.124.145)  1.636 ms  1.629 ms  1.619 ms
 6  Pos-channel1.mcore4.PDI-PaloAlto.as6453.net (207.45.213.133)  173.667 ms  173.017 ms  173.030 ms
 7  if-0-0-0.mcore3.PDI-PaloAlto.as6453.net (216.6.29.37)  552.095 ms  552.101 ms  552.090 ms
 8  if-1-1-0-0.tcore1.PDI-PaloAlto.as6453.net (216.6.29.66)  255.450 ms  255.827 ms  255.819 ms
 9  if-4-0-0.core1.DDV-Denver.as6453.net (66.198.127.70)  190.191 ms  190.264 ms if-0-0-0.core1.DDV-Denver.as6453.net (66.198.127.58)  213.899 ms
10  if-1-0-0.core1.CT8-Chicago.as6453.net (66.110.27.5)  211.711 ms  211.407 ms  211.449 ms
11  if-5-921.tcore2.AEQ-Ashburn.as6453.net (66.110.27.54)  263.884 ms  263.901 ms  264.162 ms
12  216.6.42.14 (216.6.42.14)  260.303 ms  260.515 ms  260.188 ms
13  tpa1-ar3-ge-1-0-0-0.us.twtelecom.net (66.192.247.242)  275.169 ms  274.268 ms  274.277 ms
14  * * *
15  * * *
16  * * *
17  * * *
18  * * *
19  * * *
20  * * *
21  * * *
22  * * *
23  * * *
24  * * *
25  * * *
26  * * *
27  * * *
28  * * *
29  * * *
30  * * *


[root@server1 ~]: traceroute linux-tech.net
traceroute to linux-tech.net (68.233.248.163), 30 hops max, 40 byte packets
 1  gw.BS2-30.as29550.net (213.229.103.65)  1.074 ms  1.102 ms  1.103 ms
 2  the-a.3.magic.as29550.net (92.48.95.49)  1.105 ms  1.155 ms  1.157 ms
 3  vl667.hex-7600.as29550.net (92.48.95.9)  1.384 ms  1.402 ms  1.399 ms
 4  xe-1-2-0.lon21.ip4.tinet.net (77.67.67.137)  1.396 ms  1.399 ms  1.392 ms
 5  xe-0-0-0.lon14.ip4.tinet.net (89.149.183.162)  2.219 ms xe-1-0-0.lon14.ip4.tinet.net (89.149.183.166)  1.382 ms  1.387 ms
 6  213.200.77.234 (213.200.77.234)  30.179 ms  1.648 ms  1.633 ms
 7  * * *
 8  * * *
 9  * * *
10  * * *
11  * * *
12  * * *
13  * * *
14  * * *
15  * * *
16  * * *
17  * * *
18  * * *
19  * * *
20  * * *
21  * * *
22  * * *
23  * * *
24  * * *
25  * * *
26  * * *
27  * * *
28  * * *
29  * * *
30  * * *

[root@server ~]: traceroute linux-tech.net
traceroute to linux-tech.net (68.233.248.163), 30 hops max, 38 byte packets
 1  newlevelhosting.com (75.126.47.81)  0.411 ms  0.292 ms  0.285 ms
 2  po101.dar02.sr01.dal01.networklayer.com (66.228.118.157)  0.289 ms  0.274 ms  0.287 ms
     MPLS Label=11669 CoS=6 TTL=1 S=0
 3  ae6.bbr02.eq01.dal03.networklayer.com (173.192.18.212)  0.292 ms  0.252 ms  0.236 ms
 4  * * *
 5  vlan80.csw3.Dallas1.Level3.net (4.69.145.190)  0.514 ms vlan70.csw2.Dallas1.Level3.net (4.69.145.126)  0.621 ms vlan90.csw4.Dallas1.Level3.net (4.69.145.254)  1.825 ms
 6  ae-71-71.ebr1.Dallas1.Level3.net (4.69.151.138)  0.769 ms ae-61-61.ebr1.Dallas1.Level3.net (4.69.151.126)  0.632 ms ae-81-81.ebr1.Dallas1.Level3.net (4.69.151.150)  0.627 ms
 7  ae-1-12.bar2.Tampa1.Level3.net (4.69.137.117)  24.234 ms  52.605 ms  24.368 ms
 8  ae-0-11.bar1.Tampa1.Level3.net (4.69.137.109)  24.185 ms  24.286 ms  24.283 ms
 9  * * *
10  * * *
11  * * *
12  * * *
13  * * *
14  * * *
15  * * *
16  * * *
17  * * *
18  * * *
19  *
They are definitely down, have been down since aproximately 4:40(ish) am CST
Posted by tnedator, 05-27-2011, 06:31 AM
I use Total Server Solutions for linux management, they had put in a ticket with HiVelocity prior to me receiving a page/text from Hyperspin monitoring, as TSS do their own server monitoring. HiVelocity just responded to the ticket saying that they have a network issue in part of the DC that they are working on. Hopefully, it's quickly resolved.
Posted by databloc, 05-27-2011, 06:32 AM
yep.. server down..

68.233.249.*..
Posted by twhiting9275, 05-27-2011, 06:32 AM
Well, it's already not been 'quickly resolved', it's been going on for close to an hour now.
Posted by Hendrickson, 05-27-2011, 06:37 AM
Yes I'm down too. Its been offline for sometime now
Posted by tnedator, 05-27-2011, 06:39 AM
Quote:

Originally Posted by linux-tech

They are definitely down, have been down since aproximately 4:40(ish) am CST

FWIW, at least with my server, it was around 5:00 am CDT. Hyperspin successfully connected at 4:57 and then couldn't connect at 5:00.
Code:
	

May 27 2011 05:03:23 AM	Roubaix, France	DOWN	connection timeout	
		
May 27 2011 05:02:41 AM	Ashburn, Virginia, USA	DOWN	cannot connect to port 80								

May 27 2011 05:01:20 AM	Frankfurt, Hesse, Germany	DOWN	connection timeout								
									
May 27 2011 05:00:00 AM	Seattle, Washington, USA	DOWN	connection timeout								
	
May 27 2011 04:56 AM	May 27 2011 04:56:55 AM	Maidenhead, Berkshire, UK	UP		202ms	161ms	0ms	258ms	476ms	1097ms	View Response	
	
May 27 2011 04:53 AM	May 27 2011 04:54:10 AM	Ashburn, Virginia, USA	UP		7ms	35ms	0ms	129ms	108ms	279ms	View Response
Posted by twhiting9275, 05-27-2011, 06:44 AM
You're right, I misread the time on the alert, was in fact closer to 5am.
AND WE'RE BACK!!! Yay, now I can start getting all my "service is down" notifications. Uggh
Posted by kimboy, 05-27-2011, 06:47 AM
my server are back online now
Posted by tnedator, 05-27-2011, 07:39 AM
Down for 39 minutes according to Hyperspin. Fortunately, outages like this have been rare with HiVelocity.
Posted by HivelocityGM, 05-27-2011, 08:55 AM

Quote:

Originally Posted by tnedator

Down for 39 minutes according to Hyperspin. Fortunately, outages like this have been rare with HiVelocity.

From what I can tell the network issue was for around the 39 minutes you are stating. It was not our entire network that was affected but specific vLans. I will be posting a full explanation on this event on our forum shortly once I have all the information.

I am very sorry for any inconvenience this caused those of you who were affected. Feel free to email me direct if you would like to discuss this more.
Posted by tnedator, 05-27-2011, 08:57 AM

Quote:

Originally Posted by HivelocityGM

From what I can tell the network issue was for around the 39 minutes you are stating. It was not our entire network that was affected but specific vLans. I will be posting a full explanation on this event on our forum shortly once I have all the information.

I am very sorry for any inconvenience this caused those of you who were affected. Feel free to email me direct if you would like to discuss this more.

As always, thanks for the update and great job of communicating with us on WHT (and other mediums).
Posted by databloc, 05-27-2011, 09:19 AM
Thanks communications Hivelocity.
Posted by ttgt, 05-27-2011, 10:50 AM
the router get down again ?
Posted by twhiting9275, 05-27-2011, 11:18 AM

Quote:

Originally Posted by databloc

Thanks communications Hivelocity.

Quote:

Originally Posted by tnedator

great job of communicating with us on WHT (and other mediums).

What communication? 3 hours after the event occurs? really, that's "great communication"?

Truth be told, this is getting beyond tiring. This is just the latest in a long string of routing events at hivelocity since midnight tonight. Once again, routing is completely and totally down, yet we get nothing for communication at all. Even their "status" page lies and says "we're up", did before, does now.

Come on Hivelocity, get with the program. Find and fix whatever's going on!!!
Posted by kimboy, 05-27-2011, 11:22 AM
down again here
Posted by databloc, 05-27-2011, 11:24 AM
there we go back up again
Posted by twhiting9275, 05-27-2011, 11:26 AM
Yup
Like I said, been this way all damn night.
Give it an hour or two, it'll go right back to 'down' again
Posted by ttgt, 05-27-2011, 11:29 AM
over many times fail,i really need to move my servers out...
Posted by kimboy, 05-27-2011, 12:08 PM
are back again, I contacted chat support tell me this are fixed now.
Posted by HivelocityGM, 05-27-2011, 12:48 PM
We have posted an EOD and follow ups here- http://forums.hivelocity.net/network...html#post15659

Once again, we are very sorry to those of you who have been affected today.
Posted by TannerG, 05-27-2011, 12:52 PM
Glad to see this resolved, Hivelocity is defiantly quick to get their services back up and running; especially the fact that it went down so early in the morning.
Posted by tnedator, 05-27-2011, 01:55 PM

Quote:

Originally Posted by linux-tech

What communication? 3 hours after the event occurs? really, that's "great communication"?

Truth be told, this is getting beyond tiring. This is just the latest in a long string of routing events at hivelocity since midnight tonight. Once again, routing is completely and totally down, yet we get nothing for communication at all. Even their "status" page lies and says "we're up", did before, does now.

Come on Hivelocity, get with the program. Find and fix whatever's going on!!!

I disagree. This is three hours coming to a third party forum to give us an update. I consider that good communication. I received an update in my trouble ticket within 15 minutes or so.

As to downtime, the only downtime that HiVelocity registered was from 5:00am-5:40am or so this morning. The, around 11:00 this morning there was another brief outage, which was apparently the rebooting of the router. I did not experience a series of problems since midnight. I know I was on my site until about 12:40 central time, and all was fine then, and I didn't get my server down alert until 5:12 (12 minutes after it went down, which is how my alerts are setup).
Posted by twhiting9275, 05-29-2011, 07:48 AM

Quote:

Originally Posted by tnedator

I disagree. This is three hours coming to a third party forum to give us an update. I consider that good communication. I received an update in my trouble ticket within 15 minutes or so.

Let's see here.
A> Their own status, through ALL of this said the services were 'up'.
B> Their 'forum', through ALL of this said nothing.

And once again, their network is DOWN. Rather than say anything about this, what have they done? Oh, nothing. They'd rather pretend that the downtime doesn't exist

Who CARES if they respond to a helpdesk ticket within 15 minutes. That's their job to do. That's not "excellent communication", that's a staff member doing their job.

"Excellent communication" is staying AHEAD of the incident, not hiding it, not denying there's multiple outages, it's publicly staying ahead of the incident!

edit:
yay, it was only down for 20 minutes this time. Good LORD, this is getting embarrassing for these people. "Network upgrade" my tail end!!!
Posted by kimboy, 05-29-2011, 08:44 AM
down again here now anyone experience same problem ?
Posted by twhiting9275, 05-29-2011, 08:46 AM
Yup, same route, down again. Second time this morning
Does Hivelocity acknowledge this downtime? Of course not. Same as before. Nothing in forums, nothing on status page, we're just forced to guess what the problem is and when it's going to be resolved.
Posted by kimboy, 05-29-2011, 08:53 AM
back again ! router respond now, but this situation became unacceptable why not just replace this fault hardware and keep good service again.
Posted by ttgt, 05-29-2011, 08:53 AM
It seems unstable again
Posted by ttgt, 05-29-2011, 10:02 AM
down again
Posted by twhiting9275, 05-29-2011, 10:03 AM
Here we go again
Posted by ttgt, 05-29-2011, 10:13 AM
i really think they may not think it is a big/important issue.

i open live chat,and they close the chat quickly

David: yes, i am aware, our admin is working on it now
Chat session has been terminated by the site operator.
Posted by JLHC, 05-29-2011, 10:32 AM

Quote:

Originally Posted by ttgt

i really think they may not think it is a big/important issue.

i open live chat,and they close the chat quickly

David: yes, i am aware, our admin is working on it now
Chat session has been terminated by the site operator.

They may be swamped by too many chat requests about the same issue.
Posted by HivelocityGM, 05-29-2011, 10:54 AM
I apologize for the delay in getting everyone an update on this morning's network instability. I have updated the network status forum on our site explaining the situation. Most of our customers were not affected by this mornings issue, but to those of you that were I am very sorry.

http://forums.hivelocity.net/network...html#post15660

Please know that we do take this very seriously and that we understand that any interruption in service, short or long, adversely affects our customers ability to run their businesses. We care deeply for each of our customers and providing you with a reliable hosting solution is always our primary focus.