opticgrid down?

Portal Home > Knowledgebase > Industry Announcements > Web Hosting Main Forums > Providers and Network Outages and Updates > opticgrid down?

Posted by Faris Aziz, 08-15-2010, 05:31 AM
I can't access their website, my server is down and I can't login Virtuozzo.

Anyone else having the same problem?
Posted by batz_one, 08-15-2010, 05:33 AM
I am.

Their main site is down too.
Posted by Faris Aziz, 08-15-2010, 05:34 AM

Quote:

Originally Posted by batz_one

I am.

Their main site is down too.

ARGH , any ways to contact them?

What services do you have with them?
Posted by batz_one, 08-15-2010, 05:35 AM
A VPS I thought they had 100% uptime :/
Posted by Faris Aziz, 08-15-2010, 05:35 AM

Quote:

Originally Posted by batz_one

A VPS I thought they had 100% uptime :/

Same. Everything was going okay for 3 weeks until today .
Posted by DeltaAnime, 08-15-2010, 06:16 AM
Have you guys given them a call?

Francisco
Posted by Faris Aziz, 08-15-2010, 06:18 AM

Quote:

Originally Posted by DeltaAnime

Have you guys given them a call?

Francisco

What's their number? Is it toll free?
Posted by batz_one, 08-15-2010, 06:20 AM
Found it from their whois, calling them now

edit: left a message
Posted by Faris Aziz, 08-15-2010, 06:27 AM

Quote:

Originally Posted by batz_one

Found it from their whois, calling them now

edit: left a message

Just wait, there is nothing we can do.
Posted by Faris Aziz, 08-15-2010, 09:45 AM
I'm getting worried, clients are complaining and there is still no sign of them
Posted by batz_one, 08-15-2010, 10:40 AM
Good news, gave them another ring, they should be back online soon (hopefully!) (Forgot to ask who picked up, doh!)
Posted by Faris Aziz, 08-15-2010, 10:48 AM

Quote:

Originally Posted by batz_one

Good news, gave them another ring, they should be back online soon (hopefully!) (Forgot to ask who picked up, doh!)

Thanks mate. I called them and they say there will be an SLA Credit Refund
Posted by Faris Aziz, 08-15-2010, 11:38 AM
Server Back online. Would like to hear from Opticgrid management what happen.
Posted by dudul, 08-16-2010, 09:49 PM
nope still down here. any news from them?
Posted by km01, 09-04-2010, 02:09 AM
optic grid down againg...
Posted by Faris Aziz, 09-04-2010, 02:18 AM

Quote:

Originally Posted by km01

optic grid down againg...

Yeah, damn it. I tried calling but no responses.
Posted by sahilchugh, 09-04-2010, 09:31 AM
Yes. Down from more than 12 hours now
Main site also down!
Posted by Faris Aziz, 09-04-2010, 10:18 AM

Quote:

Originally Posted by sahilchugh

Yes. Down from more than 12 hours now
Main site also down!

Its the 2nd longest downtime. I bet its time for me to search for a new host.
Posted by JcLusso, 09-04-2010, 12:42 PM
The issue has been resolved. The main problem we are resolving is the fact that the OpticGrid site is on this specific server that has been experiencing issues. We are currently moving the site onto a separate server to prevent any future issues in contacting our support personnel. Also the server that is experiencing issues is going to be upgraded to a newer server ASAP to prevent any future downtimes because we suspect it may be something to due with the hardware. All clients effected will be notified prior to the upgrade and downtime during the upgrade should be less than 30 minutes at a maximum. Please keep your eyes on your email as we will be sending out the notification in the coming week.
Posted by JcLusso, 09-04-2010, 12:46 PM
Also all services effected should be restored. SLA credit will be given for this downtime so please open a support ticket.
Posted by Faris Aziz, 09-04-2010, 05:13 PM

Quote:

Originally Posted by JcLusso

Also all services effected should be restored. SLA credit will be given for this downtime so please open a support ticket.

Thank you. Will open a new ticket.
Posted by Faris Aziz, 09-04-2010, 05:17 PM

Quote:

Originally Posted by JcLusso

Also all services effected should be restored. SLA credit will be given for this downtime so please open a support ticket.

Server seems down again and your site down.
Posted by km01, 09-04-2010, 05:20 PM
Unbeliveble! Down again...
Posted by JcLusso, 09-04-2010, 06:35 PM
We believe there may be hardware failure. We have support personnel looking into this issue and we will being giving out SLA Credit to all clients effected. We are greatly sorry about this downtime and it is very embarrassing for us to have this downtime occur.
Posted by dudul, 09-04-2010, 10:21 PM

Quote:

Originally Posted by JcLusso

We believe there may be hardware failure. We have support personnel looking into this issue and we will being giving out SLA Credit to all clients effected. We are greatly sorry about this downtime and it is very embarrassing for us to have this downtime occur.

I still can not access my website. is it still down?
Posted by johnemmel2003, 09-05-2010, 02:17 AM
MY SITE IS STILL DOWN. ALMOST 24 HRS NOW. OTHER VPS ARE NOW ONLINE. DO YOU HAVE PLANS TO TAKE SH1.OPTICGRID.COM ONLINE? THIS DOWNTIME HAS ALREADY CAUSED A LOT OF DAMAGES TO MY COMPANY. MY CLIENTS ARE GOING NUTS!
Posted by Faris Aziz, 09-05-2010, 04:00 AM

Quote:

Originally Posted by JcLusso

We believe there may be hardware failure. We have support personnel looking into this issue and we will being giving out SLA Credit to all clients effected. We are greatly sorry about this downtime and it is very embarrassing for us to have this downtime occur.

My server is up, but clientarea down so we cant open a ticket to request SLA
Posted by sahilchugh, 09-05-2010, 04:44 AM
More than 24 hours now! Still down!
Posted by Faris Aziz, 09-05-2010, 04:51 AM

Quote:

Originally Posted by sahilchugh

More than 24 hours now! Still down!

WOW, server went down again. I lost hope towards them.
Posted by km01, 09-05-2010, 04:51 AM
The worst thing is that we can't even access the backups , so we can move our clients to another provider.
Posted by johnemmel2003, 09-05-2010, 04:51 AM
DOWN AGAIN! IM SO ANGRY RIGHT NOW!
Posted by Faris Aziz, 09-05-2010, 04:57 AM
Relazx, i have clients in my server which is down too but the good thing is my clients do not nag when server is down.
Posted by JcLusso, 09-05-2010, 04:58 AM
All data is secure. We are working on the server since there is something wrong with it. We hope to have this server back online shortly.
Posted by JcLusso, 09-05-2010, 05:21 AM
Also the SLA credit for any client effected with be a full 100% credit for the entire months payment. Once again we are very sorry about the inconvenience this downtime may be causing and we shall have this issue resolved shortly.
Posted by Faris Aziz, 09-05-2010, 08:10 AM

Quote:

Originally Posted by JcLusso

Also the SLA credit for any client effected with be a full 100% credit for the entire months payment. Once again we are very sorry about the inconvenience this downtime may be causing and we shall have this issue resolved shortly.

Been 4 hrs since you last posted this. Your shortly is really fast.

Server still down. . . . . .
Posted by dudul, 09-05-2010, 09:31 AM
any good news?.........

My Client begin to contact me.
Posted by km01, 09-05-2010, 11:44 AM
server UP right now. I hope it remains this way
Posted by km01, 09-05-2010, 12:51 PM
DOWN AGAIN... What's happening JcLusso ?
Posted by JcLusso, 09-05-2010, 01:18 PM
We have technicians looking at the server to see why it is crashing. We are working on the issue as fast as possible and are trying to resolve this. Please note that once our site is back online open a ticket so you can receive your 100% SLA credit. Once again we are very sorry about any inconvenience this may be causing and we appreciate your cooperation.
Posted by Faris Aziz, 09-05-2010, 05:35 PM

Quote:

Originally Posted by JcLusso

We have technicians looking at the server to see why it is crashing. We are working on the issue as fast as possible and are trying to resolve this. Please note that once our site is back online open a ticket so you can receive your 100% SLA credit. Once again we are very sorry about any inconvenience this may be causing and we appreciate your cooperation.

Been almost 5 hours and your site is down and my servers is down.

.................
Posted by JcLusso, 09-05-2010, 05:37 PM
We believe we may have found the problem. After a full chassis switch and the problem persisting we are led to believe it may be the RAID Controller's Firmware. We are updating the firmware and we hope to have the server back online in the next 30 minutes.
Posted by tarheelisp, 09-05-2010, 08:02 PM
Twice in two months, the systems have gone down for over 6 hours with apparently no way to reach or hear from the management, Is this normal operations porcedures?
Posted by JcLusso, 09-05-2010, 08:26 PM
On behalf of OpticGrid Networks:

Cause & Resolution of Downtime:
The past downtimes were all related to this same issue. Our last downtime of this same server was caused due to a file system corruption which left us having to run an entire file system check and rebuild the RAID. On a large server this can be quite time consuming and therefore caused a very large period of downtime.

This downtime turned out to be very similar. For an unknown reason (possible hardware failure) the file system corrupted again and after doing the file system check the server would come online for a short period and kernel panic. After numerous attempts to find the cause we replaced the entire hardware chassis of the server except the hard drives. This process resolved the issue temporarily but after the server came up it kernel panicked again. This was due to an incompatibility between the RAID Controller firmware and the new chassis. After diagnosing the issue we upgraded the firmware and the issues have now been resolved.

Future Plans:
One of the biggest issues was the fact that none of our clients could contact us and this was because all of the OpticGrid Network sites happen to be hosted on one of the hardware nodes we have clients on. To prevent such from ever happening we are in the process of migrating the OpticGrid Network sites to a high availability server setup so that the OpticGrid Network support team will not be compromised. We realize this was a huge fault on our end.

Service Level Agreement:
While many of our clients know we do offer an SLA. This downtime will be covered by the SLA and we will be giving a full 100% credit of your monthly recurring total. Please open a support ticket if you were an effected client and we will have the credit applied to your account.

If anyone has any questions feel free to contact us at support@opticgrid.com

We appreciate the entire cooperation and patience of all of our clients and hope that we can regain your trust in OpticGrid Networks.
Posted by Faris Aziz, 09-06-2010, 03:39 AM
Thank you. But your toll free hotline was not answered by anyone. Just wondering...
Posted by johnemmel2003, 09-06-2010, 08:53 AM
ITS DOWN AGAIN!
Posted by dudul, 09-06-2010, 08:57 AM
same here, cannot access the web.
Posted by sahilchugh, 09-06-2010, 09:11 AM
Every time I tell my clients it's up, it goes down again!
Not sure what to tell them now.
Posted by Faris Aziz, 09-06-2010, 09:34 AM

Quote:

Originally Posted by johnemmel2003

ITS DOWN AGAIN!

Awesome , i bet this will never stop.
Posted by tarheelisp, 09-06-2010, 10:04 AM
I have a backup host,so, in theory, I will only have to change DNS servers to get the sites moved, but all my backups are on the 'more reliable' service, which is down, lol.

Does anyone know how to have a real-time mirror site on different servers free or really cheep?
Posted by tarheelisp, 09-06-2010, 10:08 AM
All my clients are friends and family. Right now I am in hot water with my wife because she is on call today (medical field) and gets important faxes by email. Needless to say, her patients are on hold, not a good thing!

I am transferring her email to a backup system. It won't get the old stuff, but it will allow her to work.

This is really bad!

Quote:

Originally Posted by sahilchugh

Every time I tell my clients it's up, it goes down again!
Not sure what to tell them now.

Posted by johnemmel2003, 09-06-2010, 10:32 AM
Well I bet JcLusso doesn't even know that his node is DOWN AGAIN. I am pretty sure NOBODY IS HOME to fix the problem. I don't need a 100% SLA Credit. I AM GETTING A REFUND! So much with SOFTLAYER, NEHALEM, 1GBPS, RAID10 1TB HDD, 32GB RAM. I can live with 286SX machines that doesn't go OFFLINE. Darn! I remembered my Quantum 130MB Harddisk!
Posted by JcLusso, 09-06-2010, 10:39 AM

Quote:

Originally Posted by Farisworlds

Thank you. But your toll free hotline was not answered by anyone. Just wondering...

Our PBX was going through the same server.
Posted by JcLusso, 09-06-2010, 10:47 AM

Quote:

Originally Posted by johnemmel2003

Well I bet JcLusso doesn't even know that his node is DOWN AGAIN. I am pretty sure NOBODY IS HOME to fix the problem. I don't need a 100% SLA Credit. I AM GETTING A REFUND! So much with SOFTLAYER, NEHALEM, 1GBPS, RAID10 1TB HDD, 32GB RAM. I can live with 286SX machines that doesn't go OFFLINE. Darn! I remembered my Quantum 130MB Harddisk!

You speak as if I am a single entity. I am the only one that writes on this forum but am not the only one at OpticGrid Networks. This is very embarrassing for us and we do realize it. SoftLayer is really putting us in a tough position too since this server is one in their data center. I hate to say anything bad about them but we have a boat load of servers with them and this one has just been horrible.
Posted by JcLusso, 09-06-2010, 11:03 AM
The only solution we have left at this point is to migrate clients onto a different hardware node. We are going to begin this process immediately. We are also bringing another hardware node online to ensure we have enough space for all the clients on this node to be moved.

Technical Info: All hardware but the hard drives have been replaced and the system continues to kernel panic and rebuild the RAID arrays.
Posted by JcLusso, 09-06-2010, 11:20 AM
The hardware node with issues is coming online now. We are deploying a new server now as well and are beginning the migration of the VPS's. We will be migrating VPS's over the private back-end network so the transfer speeds should be fast and we hope to have all accounts transfered fairly quick.
Posted by JcLusso, 09-06-2010, 11:33 AM
The hardware node with issues is now online. We are in the process of bringing an additional node online. We are also moving some VPS's onto another server right now.
Posted by sahilchugh, 09-06-2010, 03:13 PM
Hi JcLusso,

I understand but please let me know the exact time after which the servers will be online permanently as earlier without any issues so that I can convey the same to clients otherwise it will create problems.

I hope you understand.
Posted by tarheelisp, 09-06-2010, 04:03 PM
I am still up and down.
Posted by JcLusso, 09-06-2010, 06:14 PM

Quote:

Originally Posted by johnemmel2003

ITS DOWN AGAIN!

We are migrating the accounts onto the new server.
Posted by JcLusso, 09-06-2010, 10:42 PM
Some accounts have been moved. The additional server took a bit longer to deploy than expected due to hardware requirements. We are now migrating any remaining accounts.

OpticGrid Networks sites are being migrated onto a separate server as-well and that is why the site may be unreachable for some while DNS propagates.
Posted by PrestonHost_Jon, 09-06-2010, 10:42 PM
Whats the status of the migration?
Posted by Faris Aziz, 09-07-2010, 04:36 AM

Quote:

Originally Posted by JcLusso

Some accounts have been moved. The additional server took a bit longer to deploy than expected due to hardware requirements. We are now migrating any remaining accounts.

OpticGrid Networks sites are being migrated onto a separate server as-well and that is why the site may be unreachable for some while DNS propagates.

My server is still DOWN! It have been more than 12 hrs since yr last reply about migration started.
Posted by JcLusso, 09-07-2010, 06:10 AM

Quote:

Originally Posted by Farisworlds

My server is still DOWN! It have been more than 12 hrs since yr last reply about migration started.

We are still migrating accounts. We are moving files as fast as they can move.
Posted by JcLusso, 09-07-2010, 06:59 AM
The main reason this is not resolved is because it took much longer to deploy the additional server so some accounts have yet to be restored. Also if you are not getting replies to your tickets through our support system it is because the DNS is still propagating and we are having issues accessing it still.
Posted by tarheelisp, 09-07-2010, 07:15 AM
still ooc here. Another lost day of work
Posted by JcLusso, 09-07-2010, 07:40 AM
The server with problems crashed so we had to restart the migration from where it was last at about 10 minutes ago. Please note your VPS may come online on that server so do not be alarmed if it goes down again. We hope to have the remaining servers restored by 12PM EST.
Posted by JcLusso, 09-07-2010, 12:02 PM
Still transferring a few remaining accounts. If you submit tickets to our support system they will not process as the database is still moving from our old server.
Posted by tarheelisp, 09-07-2010, 01:16 PM
Still down here, but at least we are hearing regular updates.
Posted by JcLusso, 09-07-2010, 01:21 PM

Quote:

Originally Posted by tarheelisp

Still down here, but at least we are hearing regular updates.

If you paste your hostname into WHT I'll have the status checked and Ill update here until our support system is back online.
Posted by PrestonHost_Jon, 09-07-2010, 03:00 PM
Its 3PM est and we're still down.
Posted by JcLusso, 09-07-2010, 03:49 PM

Quote:

Originally Posted by PrestonHost_Jon

Its 3PM est and we're still down.

Please provide the hostname and I'll have the status of your specific VPS checked. Based on the size of your VPS is how long the transfer takes. We are transferring the files at Gbit speeds so we can't do anything more than allow the transfers to complete.

Due to this downtime we will also be providing all customers with an additional 50% credit on top of the 100% SLA credit. We realize this downtime is far longer than any downtime should be and we value the cooperation and patience our clients have given. If you have already requested your SLA credit please open a support ticket after 09/08/2010 and the credit will be applied to your account.
Posted by JcLusso, 09-07-2010, 03:52 PM
Please also note we have plans to do a full investigation of the failed hardware node so we can work towards preventing any future downtimes of this sort. As well we are looking into a High Availability setup between all of our hardware nodes so should one fail the others can pick up the slack until a replacement is ready.
Posted by JcLusso, 09-07-2010, 04:38 PM
Please note if your server is up there may be a short downtime of around 15-30 minutes later this evening while we install an update for Virtuozzo on the new node. It may be shorter than 15 minutes if all goes well. This will only effect a few clients but please do not be alarmed.
Posted by Faris Aziz, 09-07-2010, 04:52 PM

Quote:

Originally Posted by JcLusso

Please note if your server is up there may be a short downtime of around 15-30 minutes later this evening while we install an update for Virtuozzo on the new node. It may be shorter than 15 minutes if all goes well. This will only effect a few clients but please do not be alarmed.

Hostname - dallas.prestonhost.com
Posted by JcLusso, 09-07-2010, 05:01 PM

Quote:

Originally Posted by Farisworlds

Hostname - dallas.prestonhost.com

Will check the status and get back to you.
Posted by tarheelisp, 09-07-2010, 05:19 PM

Quote:

Originally Posted by JcLusso

If you paste your hostname into WHT I'll have the status checked and Ill update here until our support system is back online.

tarheelisp.com

It is the host in Dallas, TX your only shared reseller hosting site. (I think)
Posted by JcLusso, 09-07-2010, 05:25 PM
@prestonhost - we are still copying your server. your specific server had issues on the first copy and that is why it is taking an extended amount of time for your server.
Posted by Faris Aziz, 09-07-2010, 06:36 PM

Quote:

Originally Posted by JcLusso

@prestonhost - we are still copying your server. your specific server had issues on the first copy and that is why it is taking an extended amount of time for your server.

Okay please update.
Posted by JcLusso, 09-07-2010, 07:56 PM

Quote:

Originally Posted by Farisworlds

Okay please update.

Your server should be up now.
Posted by dudul, 09-07-2010, 08:48 PM
mine is still down, how long all will be up?
Posted by JcLusso, 09-07-2010, 09:04 PM
Please note our member center is still down. Do not be alarmed. We are still migrating the client database which is very large and that is why the system is down.
Posted by sphere, 09-07-2010, 10:01 PM
Hi Jarret,

Please check your pm, it's very urgent!

Thanks
Posted by johnemmel2003, 09-07-2010, 10:19 PM
JcLusso, Intel® Xeon® Processor X5570? you are trying to win us back. And it seems working.
Posted by dudul, 09-08-2010, 12:42 AM
It's Up Now.

Is there any chance to down again? if not, I want to inform my client that's everything is all right.
Posted by Faris Aziz, 09-08-2010, 01:05 AM

Quote:

Originally Posted by johnemmel2003

JcLusso, Intel® Xeon® Processor X5570? you are trying to win us back. And it seems working.

WOW! awesome man. How many cpus are there?
Posted by Faris Aziz, 09-08-2010, 01:50 AM

Quote:

Originally Posted by Farisworlds

WOW! awesome man. How many cpus are there?

argh mine still same.

Processor #1 Vendor: GenuineIntel
Processor #1 Name: Intel(R) Xeon(R) CPU
Posted by JcLusso, 09-08-2010, 07:09 AM

Quote:

Originally Posted by Farisworlds

argh mine still same.

Processor #1 Vendor: GenuineIntel
Processor #1 Name: Intel(R) Xeon(R) CPU

You have 5570s. Where are your reading from?
Posted by JcLusso, 09-08-2010, 07:11 AM
If anybody is having issues with ANYTHING please open a support ticket. Our member center is now live again. Please note incoming mail is still down. Any email you send will not be received so please open a support ticket directly. You will receive email from us though.
Posted by JcLusso, 09-08-2010, 07:14 AM

Quote:

Originally Posted by johnemmel2003

JcLusso, Intel® Xeon® Processor X5570? you are trying to win us back. And it seems working.

We were actually hoping to get 5670s which are 6 x 2.93GHz but they were out of stock.

Maybe they will be back in stock in a few weeks. they are tough to come by since they are still new!
Posted by Faris Aziz, 09-08-2010, 10:06 AM

Quote:

Originally Posted by JcLusso

You have 5570s. Where are your reading from?

WHM -> Server informations.
Posted by JcLusso, 09-08-2010, 10:26 AM
Continue to read over to the right past where you read.
Posted by Faris Aziz, 09-08-2010, 10:31 AM

Quote:

Originally Posted by JcLusso

Continue to read over to the right past where you read.

Opps my bad . Thanks JcLusso
Posted by JcLusso, 09-08-2010, 02:37 PM
If anyone is experiencing any issues please inform us immediately. You can open a ticket at http://www.members.opticgrid.com/
Posted by JcLusso, 09-08-2010, 08:31 PM
Some clients may experience a short downtime of 5 minutes duringthe next 30 minutes.
Posted by tarheelisp, 09-08-2010, 09:18 PM
down again, and for more than 20 minutes that I have been back here.
Posted by dudul, 09-08-2010, 10:55 PM
any news? I have down time too here
Posted by JcLusso, 09-09-2010, 01:12 AM

Quote:

Originally Posted by tarheelisp

down again, and for more than 20 minutes that I have been back here.

Quote:

Originally Posted by dudul

any news? I have down time too here

It appears some DNS was not updating. This issue would have been resolved faster if a ticket was opened. Please use our ticket system for now on. This forum is not monitored nearly as much as our support system.
Posted by madchas, 09-09-2010, 09:18 AM
Hey tarheelisp and dudul, consider yourself lucky...I am still down from the first go round and have yet to come back up.

So, anyone else still down??
Posted by JcLusso, 09-09-2010, 12:01 PM

Quote:

Originally Posted by madchas

Hey tarheelisp and dudul, consider yourself lucky...I am still down from the first go round and have yet to come back up.

So, anyone else still down??

If you are experiencing issues please open a support ticket. We will continue to check this forum but it is not monitored like our support ticket system. Posting in this forum will not expedite your issue since we will not see it as fast as we would see a support ticket.
Posted by tarheelisp, 09-13-2010, 10:46 PM
I am not quite technical enough to understand the details of what happened in this last outage with OpticGrid. What I assume happened, in my eyes, is that the redundancy system that is supposed to keep the system from failing, failed and took the system with it. (I have polished up my shotgun just for Murphy!)

Jarrett has taken a lot of heat from me, as well as others who were sweating it out and are still feeling the repercussions of this failure. He did acknowledge that his lack of communication in the beginning was a major mistake, and it was. However, he has recognized it and has committed to preventing that in the future.

I think a second problem was that his site portal was also on that same server (or whatever the component or group is) that failed. I just signed on with OpticGrid about a month ago and we discussed how I wanted to protect myself from just such a disaster. Maybe this was the law of attraction at work.

However, Jarrett did come back on line and kept us informed and committed to protecting the communication with his customers in case there is another similar event. In spite of our panic and frustration (his customers) he became, and remained very professional in his communication with us.

Since this is the forum in which most of the communication took place, I thought it would be most appropriate that I say "Thank you" here, rather than in a private email or message to him.

Thank you Jarrett, Please pass the thanks on to the rest of your crew as well. I know it was a trying time for you all as well.

I had prepared an escape to move on to another service, but I have not only decided to remain here, I also purchased hosting on one of your customers so I have a location in two different parts of the country so if one system goes down, the other should remain operational.

Thank you,
Hang in there
Buck
Tarheelisp.com
Posted by JcLusso, 09-13-2010, 11:43 PM

Quote:

Originally Posted by tarheelisp

I am not quite technical enough to understand the details of what happened in this last outage with OpticGrid. What I assume happened, in my eyes, is that the redundancy system that is supposed to keep the system from failing, failed and took the system with it. (I have polished up my shotgun just for Murphy!)

Jarrett has taken a lot of heat from me, as well as others who were sweating it out and are still feeling the repercussions of this failure. He did acknowledge that his lack of communication in the beginning was a major mistake, and it was. However, he has recognized it and has committed to preventing that in the future.

I think a second problem was that his site portal was also on that same server (or whatever the component or group is) that failed. I just signed on with OpticGrid about a month ago and we discussed how I wanted to protect myself from just such a disaster. Maybe this was the law of attraction at work.

However, Jarrett did come back on line and kept us informed and committed to protecting the communication with his customers in case there is another similar event. In spite of our panic and frustration (his customers) he became, and remained very professional in his communication with us.

Since this is the forum in which most of the communication took place, I thought it would be most appropriate that I say "Thank you" here, rather than in a private email or message to him.

Thank you Jarrett, Please pass the thanks on to the rest of your crew as well. I know it was a trying time for you all as well.

I had prepared an escape to move on to another service, but I have not only decided to remain here, I also purchased hosting on one of your customers so I have a location in two different parts of the country so if one system goes down, the other should remain operational.

Thank you,
Hang in there
Buck
Tarheelisp.com

Everyone at OpticGrid Networks appreciates your cooperation, patience and understanding. We are very appreciative to the clients who are staying with us even after the downtime. Thank you Buck and to all others!