Softlayer down anyone?

Portal Home > Knowledgebase > Industry Announcements > Web Hosting Main Forums > Providers and Network Outages and Updates > Softlayer down anyone?

Posted by Amphibulus, 12-30-2010, 08:16 PM
Hi,

Here softayer (Washington DC location) looks to be down, up down up

Something going on?
Posted by JSCL, 12-30-2010, 08:39 PM
I'm having troubles getting to my shared host in SoftLayer Dallas - not the only person either.
Posted by Kevin2001, 12-30-2010, 09:48 PM
Yep, down here too. arg
Posted by sirius, 12-30-2010, 10:06 PM
There's an issue in WDC and SEA, specific to AT&T/SBC - not sure of the impact in Dallas. My environments in DAL01 and DAL05 are all fine, but having some on again/off again issues in SEA and WDC, but both directly related to the AT&T/SBC issue.

Quote:

Originally Posted by notification

Engineers have completed the reload of BBR01.EQ01.WDC02, BBR02.EQ01.WDC02, and BBR01.WB01.SEA01. This has resolved the immediate issues with customer connectivity.

During work on these routers, it was determined the same issue is occurring at a much smaller scale at a number of network POPs around the SoftLayer backbone. Engineers will be reloading each of those routers later tonight. Prior to each reload, traffic will be shifted off to redundant paths. Engineers expect no interruption to customer services during these reboots; however, some customers may experience higher latency as their routes shift to other paths.

==================================================

SoftLayer Engineers will be performing emergency maintenance on BBR01.EQ01.WDC02, BBR02.EQ01.WDC02, and BBR01.WB01.SEA01 in an effort to correct routing issues customers are currently experiencing to various destinations including AT&T residential services. In conjunction with Cisco, Engineers have identified a bug that has resulted in the router entering a degraded performance state. To correct this degraded state, the router must be reloaded.

Customers can expect to see latency / packetloss as routes reconverge across redundant routers on both the public and private networks. While the maintenance window is set for 2 hours, we expect the maintenance to take no longer than 20 minutes during each router reload.

Customers in the Seattle datacenter will lose backend connectivity to all other datacenters (local Seattle private network traffic will not be impacted).

==================================================

SoftLayer Engineers are aware of a routing issue impacting a subset of customers globally on the SoftLayer backbone. Engineers have traced this issue back to a hardware forwarding limitation on a number of Cisco routers in the backbone. Engineers are currently in contact with Cisco to determine the best course of action to resolve this issue.

Posted by 3sgte_power, 12-30-2010, 10:13 PM
Am having issues too. I think its a routing issue. I can't access it on my home network (BT provider), but i can through my mobile phone network (TMobile).
Posted by Michaels75, 12-30-2010, 10:58 PM
We Received Early Maintenance notification.

Quote:

=======================================================================
SoftLayer Emergency Maintenance - GLOBAL [12/30/2010]
Date: Thursday, December 30, 2010 (12/30/2010)
Start Time: 05:00 PM CST
End Time: 07:00 PM CST
Services affected: Public / Private Network
Location: Global
Duration: 2 hours

==================================================
SoftLayer Engineers will be performing emergency maintenance on BBR01.EQ01.WDC02, BBR02.EQ01.WDC02, and BBR01.WB01.SEA01 in an effort to correct routing issues customers are currently experiencing to various destinations including AT&T residential services. In conjunction with Cisco, Engineers have identified a bug that has resulted in the router entering a degraded performance state. To correct this degraded state, the router must be reloaded.

Customers can expect to see latency / packetloss as routes reconverge across redundant routers on both the public and private networks. While the maintenance window is set for 2 hours, we expect the maintenance to take no longer than 20 minutes during each router reload.

Customers in the Seattle datacenter will lose backend connectivity to all other datacenters (local Seattle private network traffic will not be impacted).
==================================================

If you have any problems after this time frame with regard to connectivity, or if you have any questions regarding the maintenance at any point, please open a ticket in the customer portal or contact our support via phone.

We appreciate your patience during this work and welcome any feedback.

Thank you,
Network Engineering

Now, everything fine from our end.
Posted by JSCL, 12-30-2010, 11:21 PM
There's still plenty of issues with UK connectivity to SL it seems - can't access my website still.
Posted by 3sgte_power, 12-31-2010, 12:43 AM
Same here