Portal Home > Knowledgebase > Articles Database > Domainmonster.com down?


Domainmonster.com down?




Posted by Mr E Mann, 09-29-2008, 02:04 PM
Anyone else seeing the same issue? Not sure if it belongs in here or the provider outages page - so apologies if it's the wrong place (and if so, can someone move it please!)

Posted by Mr E Mann, 09-29-2008, 02:09 PM
stand at ease Looks like they clicked the switch back on again

Posted by GCM, 09-29-2008, 09:22 PM
The site isn't working for me here in the USA. All the best,

Posted by oldgit, 09-29-2008, 10:21 PM
Is this still down for anyone else? DNS came back for a period yesterday evening (I'm on GMT+3) about 8 hours ago - but seems to have gone again. Anyone know anything?

Posted by matt_domainmonster, 09-29-2008, 11:42 PM
Hi All I can help you with this one. I'm from Domainmonster.com! We are experiencing some issues at present across all our DNS server clusters. This is highly un-usual and the first I've known of in many years. We have issues like anyone else occasionally, but rarely across all resilience layers. Engineers are on site both in UK and Dallas. Just so happens another technical team (with me) are at a conference in Hong Kong, so they are also working on the issue remotely and in all cases have been, throughout the evening/night since the issue began. We are trying to resolve as quickly and effeciently as possible and at this stage expect partial resolution within 6 hours and full resolution shortly after. We have increased our level of support cover, so please contact support@domainmonster.com or call for an update if you require, although this is the current information. Please accept our sincerest apologies and rest assured its our top priority throughout the business right now. Rgds, Matt

Posted by enMatt, 09-30-2008, 03:36 AM
Matt (Domain Monster), Unfortunately we can't email you as the DNS is down so the MX records for your address can't be found. Same it true of course for our email, so you can't email back even if we got through. We can't call either because as your automated phone lines politely tell us, we're out of office hours. Is there any other way we can get an update as to what is happening? Our clients who rely on our website being up are getting very agitated, understandably. M

Posted by tttonyyy, 09-30-2008, 03:45 AM
Ah - glad to see this thread and Matt's response. If this were 123-reg it would've just been left for 48 hours unattended, with no communications. It sounds like domainmonster are at least on the ball when things like this happen (one of the reasons I transferred all my domains in after the now-infamous 123-reg weekend off). I've been meaning to set up a tertiary nameserver for some time now - this comes as a reminder that I really should have got around to it... enMatt - you would be wise to do the same.

Posted by drivecs, 09-30-2008, 04:27 AM
I also moved from 123 reg and thoght at the time that the servers at domainmonster were resilient and presumably backed up My sites have been mainly down since 12.00 yesterday which is nearly 24 hours The servers came on briefly at one point last night then went down again We are losing a lot of money through this outage I cannot understand how three servers can be down at the same time and for so long Is the system at domainmonster fundamentally flawed

Posted by drivecs, 09-30-2008, 04:30 AM
DNS server ns3.domainmonster.com[72.249.134.3] is alive, but not authoritative for domain 'xxxxxxxxx.co.uk' and didn't return SOA DNS server ns2.domainmonster.com[81.144.213.182] is alive and authoritative for domain xxxxxxxxxx.co.uk DNS server ns1.domainmonster.com[81.144.213.181] is alive and authoritative for domain xxxxxxxxxx.co.uk 2 server(s) are alive DNS server ns3.domainmonster.com failed and will be dropped from other tests ns3 has not changed state since yesterday at 1.30 while ns1 and 2 have been going up and down like a brides nightie

Posted by enMatt, 09-30-2008, 04:34 AM
Fortunately, it seems we're back online again. I will look into setting up a tertiary nameserver!

Posted by drivecs, 09-30-2008, 04:36 AM
not being an expert in DNS how do I do set up a tertiary nameserver and with who to avoid this issue again Allthough if there are any more outages today with DM I will be moving all domains away from them even

Posted by drivecs, 09-30-2008, 04:47 AM
DNS server ns3.domainmonster.com[72.249.134.3] is alive, but not authoritative for domain xxxxxxxxxx.co.uk' and didn't return SOA Error fetching SOA from ns2.domainmonster.com [81.144.213.182], request timed out. Probably DNS server is offline. DNS server ns1.domainmonster.com[81.144.213.181] is alive, but not authoritative for domain xxxxxxxxxx.co.uk' and didn't return SOA 0 server(s) are alive No DNS servers alive, tests stopped seems they are gone again

Posted by drivecs, 09-30-2008, 04:49 AM
They are now all back to no SOA as they have been all night I have to wonder if whoever is fixing it knows what exactly is wrong as these server have been going on and offline frequently as if getting rebooted

Posted by drivecs, 09-30-2008, 05:02 AM
I was with 123 reg for 10 years and had one outage I have been here since the 123 reg outage and had this period of downtime...not good

Posted by matt_domainmonster, 09-30-2008, 05:39 AM
The issue is now partially resolved but ongoing. We fully appreciate email to support may not be functioning or is intermittent, but please bear with us. Our telephones are manned and agents happy to help. I can tell you at this stage the problem has been identified and a resolution being applied. You should be seeing a lot more intermittent results now and we envisage full resolution within the hour at least to one of the clusters. I can assure those posting about tertiary servers the network and resilience is incredibly resiliant and is something that is audited regularly by ICANN for exactly this reason. What appears to have happened here is just been an incredibly unlucky combination of Hardware and Software failures, something that as those above have posted already, can happen to everyone, Tucows and 123-Reg both being recent 48 hour + outagages. We do have three entirely different levels of redundancy that have all failed for differnet/related reasons in this case. Regardless, I do hope the comms on the matter is of some help. I will update again in 1 hours time. Matt

Posted by matt_domainmonster, 09-30-2008, 06:28 AM
Two (NS1 and NS3) of of the Three clusters are currently back up and have been for about 40 minutes, so all domains and services should be resolving normally. We don't believe it is going to be necessary to take either one or both of them back down again, but the issue is still ongoing. We hope to add the third cluster (NS2) within the hour. I will get an update again in one hour and post again. Your email should now be reaching support@domainmonster.com and be receiving a response from our team, although this is the very latest information that they have too. Matt

Posted by enMatt, 09-30-2008, 06:34 AM
Matt I appreciate the update. Having used various DNS management solutions in the past, I do generally like DomainMonster. I particularly like how easy it is to control the detail of the DNS records, and view the change log. I would like to setup, in this case a quaternary nameserver for safety's sake on the sites where we potentially lose a lot of money if they're down for several hours. Given that this problem "can happen to everyone" it seems like a reasonable thing to do. Hopefully we can find a way to make this work whilst still keeping Domain Monster as the primary DNS and domain registration provider. Like you, I'm looking forward to it all being fixed. Thanks

Posted by tttonyyy, 09-30-2008, 06:42 AM
Matt, the comms are one of the reasons I will stick with domainmonster - it's nice to know in times of crisis what is going on. There's nothing worse than being in the dark.

Posted by drivecs, 09-30-2008, 06:42 AM
How is a quaternary nameserver setup is it something that you would do in the domainmonster configuration? What dns server would you use?

Posted by drivecs, 09-30-2008, 06:44 AM
ttonyyy it nice having comms but even nicer not losing hundreds of pounds which is why I am interested is a fall back like enmatt

Posted by matt_domainmonster, 09-30-2008, 09:01 AM
Posting back again as promised. There is no further update at this stage and will post again with a conclusion. At present, it seems the issues have now been addressed, but don't want to confirm this until I've had the final word and all three clusters are in place and fully operational again. The two available clusters that are live, have been live and un-interrupted since approx 1100 GMT, with intermittent periods PRIOR to this. Once I'm able to conclude I will respost and provide a more detailed explanation of the problem, conclusions and any steps we have or will take to ensure future stability and resilience. This may be tomorrow. Our engineering teams will remain on the issue until fully resolved, investigated and reported, 24 hours in different shifts, but unfortunatley I need to sleep at some point!

Posted by drivecs, 10-01-2008, 04:29 AM
DNS server ns2.domainmonster.com[81.144.213.182] is alive, but not authoritative for domain xxxxxxxxxxx.co.uk' and didn't return SOA Looks like ns2 is still down but the others seem to have been OK

Posted by matt_domainmonster, 10-01-2008, 06:01 AM
Correct. NS1 and NS3 clusters are stable. Will confirm once I have the final report from the engineering teams, hopefully later today. The NS2 cluster has been waiting on some hardware replacements which I believe are on a bike right now. These will hopefully be replaced within the next 4 hours. NS1 and NS3 clusters will more than suffice in the interim though, so there should be no issues. Hope that provides the latest update. Essentially this has been resolved and stable since 11:00 GMT yesterday, although not back at full resiliance which means it maintains a priority for us. Will post again later with final information for you all.

Posted by matt_domainmonster, 10-02-2008, 01:10 AM
My apologies for the delayed post. Full completion was verified at around 1200 GMT since yesterday. All 3 of our Name Server clusters are fully operational. New domains registered and any failing DNS amendments made during the downtime are still subject to ongoing works, but these should be completed today and hopefully have minimal impact. If you are one of these people and need this addressed urgently, that is possible, so please just drop a line to support@domainmonster.com or call the support lines. They will be able to remedy immediately for you, rather than wait for the bulk resolution, due by end of business, GMT. I'm now in possesion of a a full report of the issue and investigation that followed. Frankly, this was simply an incredibily un-lucky combination of hardware, software and infrastructure failures that had knock on effects between each cluster. We do intend to make some minor improvements as a result, but I stress these are minor. We are happy that our resilience level (3 tiers and I mean complete tiers, Power, Hardware, Software, Clusters, Locations, Infrastructure etc etc) overall continues to be industry leading, hence the relativly brief complete down time when compared to other major outages, that some have mentioned in this post from some of the worlds largest registrars at 48+ hours. Regardless, we accept even a seconds downtime is more than any of our customers would want and please rest assured this will continue to be the subject of much investigation for many months and days to do all we can to ensure there is NEVER a repeat. This is our first complete DNS failure that I know of and I hope our last. I do hope we can retain your custom and faith following such a difficult period. Thanks once again for your patience. Matt

Posted by drivecs, 10-02-2008, 08:15 AM
Well I for one shall be staying anyone who takes the trouble matt has to keep us informed deserves to prosper

Posted by digitalmorphosis, 10-02-2008, 09:58 AM
It would be great if the domain monster would let us specify a 4th nameserver for the domain and allow zone transfers to that server. That way we could set-up a nameserver independent of domain monster that could continue to provide service in the event of an outage such as this. This is something offered by other DNS providers, (although mainly paid for ones). If domain monster offered this as an enhanced DNS service, then this is something I would be prepared to pay a small premium for.

Posted by snozwoz, 10-06-2008, 05:36 AM
Issue was i could not get to this forum to see this (dns error), i could not get to the domainmonster website to get the support phone numbers, i could not email domain monster. Our site was down for a long time and i had no way to find out why. My first thought was oh no, domain monster has collapsed as a business and what do i do now, how do I get control of our domain back and how long will this take. I suggest to Domainmonster that they host their site and DNS with another hosting provider. If it all goes tits up like it did here at least the customers can see this. Also in one of the brief moments when i was able to reach the domainmonster web site it would have been reassuring if a news bulletin had been posted on the home page linking to this forum. Also in one of the brief moments whilst it was up; why did domainmonster not email all their customers and tell them what was going on. Why wait until we have to contact them, or wait until several days have passed before sending an apology. So yes, it is all very unfortunate that so many issues happened all at once. Yes this has happened to other service providers, yes matt did post here (well done matt) but still: 1) why did domainmonster let their systems get in to a state where they could have 3 separate issues? 2) why did domainmonster not contact their customers at the earliest oportunity and reasure them. I however will stay with domainmonster. As mentioned they are not the only ones to have experienced this and I believe they will learn from this. But if they don't and there is just one more hick up well there goes my argument and i will be off.

Posted by Mr E Mann, 10-06-2008, 01:47 PM
How can they email people, if the email server cannot see the dns to send the email to the right place? I agree though, that either an alternative host for the website, or a separate independent host for a status page would be a nice to have ...although that's the joy of hindsight... From my view, I saw that the websites were down, checked through some proxies to make sure it wasn't just me... and then posted here to find out more Was kind of relieved when it wasn't just me, as it meant a larger problem that DomainMonster were aware of - and then to get the updates was a nice touch - and certainly one that was missing during the big 123-reg debacle when their servers decided to leave home All in all, top marks to DomainMonster for (a) keeping us updated, (b) being honest about the situation and (c) resolving it as quickly as possible ...but some room for improvement, which hopefully they will resolve, and become even more resilient in the future



Was this answer helpful?

Add to Favourites Add to Favourites    Print this Article Print this Article

Also Read
WebNX down? (Views: 656)
Managed down again!?! (Views: 630)
Config httpd.conf (Views: 625)

Language: