Is 8 hrs downtime normal? Im with Innohosting.

Portal Home > Knowledgebase > Industry Announcements > Web Hosting Main Forums > Providers and Network Outages and Updates > Is 8 hrs downtime normal? Im with Innohosting.

Posted by Aamnah, 01-03-2012, 02:05 PM
Hi guys,

I have a reseller account with Innohosting.com and currently all my sites are down. I contacted the support and they say 'a tech is working on diagnosing and resolving the issue'.. its been 8 hours.

I switched to Inno a few months back from EZPZ hosting (wonderful support - i loved it, but websites being down every now and then was the problem so i switched), and now Inno is giving me downtime too.

My clients are cursing me and im getting worried now. Is 8 hours normal? Because the highest i have ever had is about 1 hr.
Posted by ezbnc, 01-03-2012, 02:08 PM
8 hours never normal but again depends what the problem is on their end.
Posted by InternetBrothers, 01-03-2012, 02:19 PM
It is not normal but have you been notified before? what was wrong exactly ? does that happen often? It is not acceptable
Posted by Aamnah, 01-03-2012, 02:25 PM

Quote:

Originally Posted by InternetBrothers

It is not normal but have you been notified before? what was wrong exactly ? does that happen often? It is not acceptable

No notification whatsoever. The sites just went down. This is the first time this has happened for so long. I dont know what's wrong, i asked the support and he replied 'a tech is diagnosing'.

Instead of being a pest to the support and whining, i patiently waited another 4 hours before contacting them again and this time he escalated the issue to someone higher-up. That was some 18 minutes ago.

Im just stuck here refreshing my browser, this is not fair.
Posted by Aamnah, 01-03-2012, 02:30 PM
Just received this from Inno:
Greetings
The server dropped a partition.
Our admins are working to restore the partition from our backup server, through a process known as a bear metal restore.
I am sorry, however there is no ETA for how long the restore can take. It depends on too many factors.
We apologize for the inconvenience, However please be aware that our system admin team is working to resolve this issue as quickly as possible.
Regards
NateS

Since there is no ETA.. i'll just go back to refreshing the browser. Lets hope this gets 'restored' soon..
Posted by InternetBrothers, 01-03-2012, 02:31 PM
Well even though it is not acceptable to have such long downtime I think before rushing to a forum like this you have to wait for their final response. We never know ! I hope you have a backup of your data.
Posted by Aamnah, 01-03-2012, 02:36 PM

Quote:

Originally Posted by InternetBrothers

Well even though it is not acceptable to have such long downtime I think before rushing to a forum like this you have to wait for their final response. We never know !

I have not 'rushed' to the forum, i have waited 8 hours. If you have had any websites down for this long you would know it feels a looooong time. Since i am kinda new, im just asking more knowledgeable folks and trying to get an idea of how worried should i be.

Btw, i updated the thread with their response so its not like i'm defaming them or anything.

Thanks for your response
Posted by InternetBrothers, 01-03-2012, 02:41 PM
I see you are right too. I understand. It is not easy.
Posted by XelionOne, 01-03-2012, 02:42 PM
I kindly disagree with InternetBrothers - I believe that by going onto a website such as webhostingtalk, you can sometimes get invaluable information with regards to downtime as so many of the hosting owners and hosting experts hide on this forum. This could have escalated the issue much quicker by getting the attention of the right people online! Sometimes they can give valuable information such as "http://www.downforeveryoneorjustme.com/" to check whether the server is actually going through issues or not!

On another note 8 hours is a rather long time - I know how it feels!! I am sure that they are doing everything they can in order to fix the problem as fast as possible! Full server restores can take a very long time - and it seems that there is actually nothing you can do with the exception of letting all your clients in the know at all times (this is the key to downtime). Its how you are able to weed out the weaker companies from the top companies - they let you know when stuff goes wrong because let's face it, something must go wrong at some point!! Its just a machine after all!

I hope you didnt lose any of your data (or that you were wise enough to make your own recent backup) as sometimes these restores aren't 100%!

Best of luck - in the hope this is resolved soon!
Posted by twhiting9275, 01-03-2012, 03:38 PM
It's absolutely a good idea to post here that your host was down for 8 hours! This kind of downtime is most definitely not 'normal' and quite unprofessional, especially if they haven't notified customers as to what's going on.
Posted by Aamnah, 01-03-2012, 04:24 PM

Quote:

Originally Posted by XelionOne

I believe that by going onto a website such as webhostingtalk, you can sometimes get invaluable information with regards to downtime as so many of the hosting owners and hosting experts hide on this forum.

Exactly!

Quote:

Originally Posted by XelionOne

On another note 8 hours is a rather long time - I know how it feels!! I am sure that they are doing everything they can in order to fix the problem as fast as possible! Full server restores can take a very long time - and it seems that there is actually nothing you can do with the exception of letting all your clients in the know at all times (this is the key to downtime). Its how you are able to weed out the weaker companies from the top companies - they let you know when stuff goes wrong because let's face it, something must go wrong at some point!! Its just a machine after all!

Knowing what is happening actually help big time, sometime even calms down.

Quote:

Originally Posted by XelionOne

I hope you didnt lose any of your data (or that you were wise enough to make your own recent backup) as sometimes these restores aren't 100%!

I should have a better contingency plan i guess. Now i will never forget to back up.

Quote:

Originally Posted by XelionOne

Best of luck - in the hope this is resolved soon!

Thank you, it was resolved 17 minutes ago =)
Posted by rwilki, 01-03-2012, 04:25 PM
I just discovered the downtime too. I only have one resold account setup so far and am planning to add all my resold accounts here. After 8 hours (my monitors tell me 10 hours) I'm now not sure if I'm OK staying here...
Posted by Aamnah, 01-03-2012, 04:30 PM

Quote:

Originally Posted by rwilki

I just discovered the downtime too. I only have one resold account setup so far and am planning to add all my resold accounts here. After 8 hours (my monitors tell me 10 hours) I'm now not sure if I'm OK staying here...

10 hours 45 mins to be precise (that's what my alerts show.) But since the sites are back up im all happy again. Here is to hoping this never happens again..
Posted by rwilki, 01-03-2012, 04:33 PM
wow. that's frightening. I would have preferred an email or notification from them about the issue. It's much more professional to be proactive...
Posted by lionlink-dot-net, 01-03-2012, 04:38 PM
Just throwing in my .02 here.... I understand that sometimes things happen, but 8 hours??? that is beyond unacceptable.

--Adam
Posted by IH-Rameen, 01-03-2012, 04:41 PM

Quote:

Originally Posted by linux-tech

It's absolutely a good idea to post here that your host was down for 8 hours! This kind of downtime is most definitely not 'normal' and quite unprofessional, especially if they haven't notified customers as to what's going on.

What is unprofessional is someone who supposedly calls themselves a linux admin, decides to call another companies issue as unprofessional when they themselves know not all issues are alike, and different issues have different problems & resolution times. It really goes to show Tom that in all these years you remain as incompetent as before and you haven't really progressed much at all.

@OP, the server experienced a fault, all customers were notified on our outages section and updates provided. The fault resulted in full data loss on that server (keep in mind this is also a RAID10 server). We had to test the RAID controller and each drive. As we perform dual backups (standard backups and R1Soft backups) on your server, it was restored to the same state as before it went down from our backups. So everything is as it was before the issue. As the OP said, the most they've ever had with us is about an hour of downtime, major issues like this rarely happen because we use decent quality hardware and have robust builds. Considering the gravity of the issue, that there was full data loss and the fact that we resolved the issue and restored everything perfectly, the time it took is normal.

A few hours to perform a full restore as well as test everything + diagnosis as well as keep customers informed, I call that quite reasonable, nothing unprofessional about it.
Posted by tnwhite, 01-03-2012, 05:45 PM
Sounds like a nasty fsck is going on.. heh
Posted by CrocWeb, 01-03-2012, 06:23 PM
People need to realize that downtime does occur. As long as the host provides a valid reason and keeps the clients notified of their progress, you should be happy.
Posted by layer0, 01-03-2012, 07:47 PM

Quote:

Originally Posted by IH-Rameen

What is unprofessional is someone who supposedly calls themselves a linux admin, decides to call another companies issue as unprofessional when they themselves know not all issues are alike, and different issues have different problems & resolution times. It really goes to show Tom that in all these years you remain as incompetent as before and you haven't really progressed much at all.

@OP, the server experienced a fault, all customers were notified on our outages section and updates provided. The fault resulted in full data loss on that server (keep in mind this is also a RAID10 server). We had to test the RAID controller and each drive. As we perform dual backups (standard backups and R1Soft backups) on your server, it was restored to the same state as before it went down from our backups. So everything is as it was before the issue. As the OP said, the most they've ever had with us is about an hour of downtime, major issues like this rarely happen because we use decent quality hardware and have robust builds. Considering the gravity of the issue, that there was full data loss and the fact that we resolved the issue and restored everything perfectly, the time it took is normal.

A few hours to perform a full restore as well as test everything + diagnosis as well as keep customers informed, I call that quite reasonable, nothing unprofessional about it.

I agree with you. In the case of catastrophic RAID failure like that, that's a reasonable restore time. In many cases, there are companies who take much longer to restore their servers. It depends on the amount of data that needs to be restored, etc. No matter how good of a company you are with, if a whole server needs to be restored, it's a process that can take several hours.
Posted by SecureHost Q&A, 01-03-2012, 08:06 PM

Quote:

Originally Posted by Aamnah

Hi guys,

I have a reseller account with Innohosting.com and currently all my sites are down. I contacted the support and they say 'a tech is working on diagnosing and resolving the issue'.. its been 8 hours.

I switched to Inno a few months back from EZPZ hosting (wonderful support - i loved it, but websites being down every now and then was the problem so i switched), and now Inno is giving me downtime too.

My clients are cursing me and im getting worried now. Is 8 hours normal? Because the highest i have ever had is about 1 hr.

Wow, 8 hours is not normal, even for a maintenance window. Do they offer an guarantee if there is down time? I'd ask about that.

Did they give you any indication of what the problem is, ie. power problem, network attack etc?
Posted by Kingfish85, 01-03-2012, 08:10 PM
Sounds like 99.9% uptime to me. At least they were able to recover all of the data.
Posted by IH-Rameen, 01-03-2012, 08:11 PM

Quote:

Originally Posted by SecureHost Q&A

Wow, 8 hours is not normal, even for a maintenance window. Do they offer an guarantee if there is down time? I'd ask about that.

Did they give you any indication of what the problem is, ie. power problem, network attack etc?

Did you not take the time to read previous responses? The reasoning we gave? It wasn't a maintenance window.

Quote:

Originally Posted by layer0

I agree with you. In the case of catastrophic RAID failure like that, that's a reasonable restore time. In many cases, there are companies who take much longer to restore their servers. It depends on the amount of data that needs to be restored, etc. No matter how good of a company you are with, if a whole server needs to be restored, it's a process that can take several hours.

That was the case here. Unfortunately hardware issues do happen, but I'm very satisfied with the way it was handled. A full baremetal restore in just a few hours.
Posted by Kingfish85, 01-03-2012, 08:24 PM

Quote:

Originally Posted by IH-Rameen

I'm very satisfied with the way it was handled. A full baremetal restore in just a few hours.

Nothing against the OP, but I think a lot, if not 95% of resellers don't have experience with hardware. I think a lot of people get the perception that because they physically don't touch a server that their reseller account is on that there's no underlying layer. If that makes sense.. People that are actually in the IT field understand this a little better.
Posted by hostcabin, 01-03-2012, 09:49 PM
If caused by hardware issue, 8 hours is still normal. Some DC even need more than that time to replace the hardware and of course the hosting owner need to do some task after the hardware replaced.
Posted by Kingfish85, 01-03-2012, 09:58 PM

Quote:

Originally Posted by hostcabin

If caused by hardware issue, 8 hours is still normal. Some DC even need more than that time to replace the hardware and of course the hosting owner need to do some task after the hardware replaced.

I had a power outage once in my building from the tornado the ripped through downtown Raleigh this past year. The building security wouldn't let anyone in. When they finally allowed IT staff back in, I found that my ESX cluster had went down due to the UPS's running out. When I fired them up, I found out the 96GB of RAM had been damaged. Dell said it was due to a temp. change, and it took them 6 hours to get new ram to me.

Luckily Powerchute issued a shutdown to my vm's before everything went down. Replaced the RAM, fired ESX & the NetApp up...everything came right up.
Posted by MikeDVB, 01-03-2012, 10:10 PM

Quote:

Originally Posted by VopaHost

I had a power outage once in my building from the tornado the ripped through downtown Raleigh this past year. The building security wouldn't let anyone in. When they finally allowed IT staff back in, I found that my ESX cluster had went down due to the UPS's running out. When I fired them up, I found out the 96GB of RAM had been damaged. Dell said it was due to a temp. change, and it took them 6 hours to get new ram to me.

Luckily Powerchute issued a shutdown to my vm's before everything went down. Replaced the RAM, fired ESX & the NetApp up...everything came right up.

How is any of this relevant? That's an ENTIRELY different scenario and issue all together.

I've see bare metal restores take 3 or 4 days - 8 to 12 hours is actually quite fast for a BMR. Either way - hardware failure does happen and it looks like Inno did what any good provider would do.

It is easier to bring a server with failed ram back online (ram swap) than it is to replace the primary storage of customer data as the data goes with the failure.

The sheer fact that Inno had backups to restore puts them above many other providers.

Good luck.
Posted by Kingfish85, 01-03-2012, 10:21 PM

Quote:

Originally Posted by MikeDVB

How is any of this relevant? That's an ENTIRELY different scenario and issue all together.

I've see bare metal restores take 3 or 4 days - 8 to 12 hours is actually quite fast for a BMR. Either way - hardware failure does happen and it looks like Inno did what any good provider would do.

It is easier to bring a server with failed ram back online (ram swap) than it is to replace the primary storage of customer data as the data goes with the failure.

The sheer fact that Inno had backups to restore puts them above many other providers.

Good luck.

It's relevant because it's still downtime. Regardless if RAM failed, how easy to replace the RAM or whether tapes had to be pulled for a restore. It's not exactly the same situation, I agree.
Posted by MikeDVB, 01-03-2012, 10:41 PM

Quote:

Originally Posted by VopaHost

It's relevant because it's still downtime. Regardless if RAM failed, how easy to replace the RAM or whether tapes had to be pulled for a restore. It's not exactly the same situation, I agree.

Apples to oranges, totally irrelevant.
Posted by BassHost, 01-04-2012, 12:22 AM
Some people can't read ...

Glad everything is up and running for you.

We have had some thing similar happen to one of our server. One of our Hard Drives failed. Not a big deal ... simply restore the Raid Array. What made it annoying was that this happened 3 times the same day on the same server.