Gigenet Cloud down (again)

Portal Home > Knowledgebase > Industry Announcements > Web Hosting Main Forums > Providers and Network Outages and Updates > Gigenet Cloud down (again)

Posted by SaaSMX, 09-20-2010, 07:19 PM
Any other Gigenetcloud user having trouble ?

VM down and not rebooting either.
Posted by jasonhk, 09-20-2010, 07:47 PM
Same here, got 2 backup nodes there (backup MX and NS) both down. Will be moving those away soon, too much downtime and false promises..
Posted by Apolo, 09-20-2010, 08:37 PM
From GigenetCloud:

Message 1

Quote:

We are currently experiencing problems with SAN2B (which is a standby hotspare) and we were in the middle of repairing SAN2A. We are currently pausing all the VMs that are attached to this particular device because of connectivity issues which we are investigating right now. We are having to reboot SAN2B at this time and will get everyone online as soon as possible. In any normal situation we would have been running on SAN2A but because of the problems we experienced just recently we have been reconfiguring it so that the previous problem doesn't reoccur. Unfortunately, because we are running off the hotspare, this downtime is unavoidable.

We apologize for the inconvenience and we will obviously provide compensation for those affected and I will be happy to take any calls regarding this issue once its resolved.

--NOTE--

This does NOT affect anyone running on our other storage units. This is specifically the same storage unit (SAN2) we had problems with just recently.

Please standby for updates as I will be providing them along with a full explanation after this has been resolved.

Message 2

Quote:

Problem Resolved - Should be backup within 10 - 20 minutes
Since we were repairing SAN2A we didn't have a failover and for some unknown reason we had a problem with SAN2B to where it became unresponsive. We had to pause all VM's attached to SAN2B and are bringing them back up asap. SAN2A will be restored as soon as possible so we don't encounter problems like this again.

Please let us know if you require compensation and if you would like to talk with me directly please call me at 800-561-2656 x 2 and ask to be transferred to Chris Armer. If I am not available I will call you back so tell support to send me an email with your contact information (or you can send me an email at chris.armer@gigenet.com) and I will get back to you in the order received.

Thank you for your patience please give us a few more minutes to get everyone affected back online.

Posted by chrisarmer, 09-21-2010, 02:42 AM
Just to follow up.....

We did manage to get everyone back online. We are getting the backup storage unit online tonight (now) with new settings to prevent the previous problem from happening again. Once this is back online we can begin troubleshooting SAN2B which doesn't appear to have any problems at the moment. We will apply upgrades to SAN2B (scheduled maintenance) which should fix any and all remaining problems in the very near future. Everything is stable and should be fully redundant at this time. Failover is fully operational and should be from this point forward on this particular centralized storage clusters.

We are taking stables to further the stability and redundancy of the storage clusters immediately and should have updates on our website soon in regards to this. New code is scheduled to be released soon which will include all bug fixes known up to this point and will speed up our process queue which will make everything operate faster in our portal. I hope to be providing very good news here shortly so stay tuned.

We apologize for any inconvenience this may have caused and will more than gladly compensate any customers who experienced downtime so please email me at chris.armer@gigenet.com at anytime.
Posted by jackpx, 09-21-2010, 07:36 AM
What is the guaranteed uptime in Gigenetcloud?
Posted by rezilient, 09-21-2010, 10:58 PM
My VM is down again right now... It was down last night during the above mentioned outage, too.

I'm seeing the following errors in the kvm console, but I can't login or do anything else.

end_request: I/O error, dev xvda, sector 63
Buffer I/O error on device xvda1, logical block 0
lost page write due to I/O error on xvda1
EXT3-fs error (device xvda1) in ext3_reserve_inode_write: Journal has aborted

Looks like a storage issue... again. And I was supposed to be moved to SAN3 tonight.. haha..
Posted by SaaSMX, 09-22-2010, 12:14 AM
Yep, down AGAIN, almost 2 hours now as we speak.
Posted by SaaSMX, 09-22-2010, 01:22 AM
And they have no clue about what is going on after 2 hours and counting.
Posted by SaaSMX, 09-22-2010, 02:17 AM
Up now, no RFO yet however
Posted by rezilient, 09-22-2010, 11:18 AM
They need to just send that SAN2 back to the vendor...