Better architecture than HA Linux?

Portal Home > Knowledgebase > Articles Database > Better architecture than HA Linux?

Posted by calande, 08-29-2008, 08:37 AM
Hello, I'm looking for a solution to host a web site that needs to be always online. I had a look at existing solutions such as heartbeat and high-availability configurations, but I'm not satisfied with the way it works, as there can always be a problem with Apache for instance, and in this case the web site is offline. What I have in mind is rather having duplicate content on three different servers in three different datacenters. They would be all in sync, like a token ring architecture. If a server receives a query to store information to a database, it also stores the info remotely to the other two servers. Every server would poll each other every second for HTTP content supposed to serve test data taken from the database. When a test page doesn't serve the content as expected, the server is taken down and the remaining servers keep syncing each other, while an admin is fixing the faulty server. When the server is back up, it synces the new data from the two other servers (especially the database). Budget for my client is a few thousand dollars a month. Is there a solution that does that? The web site really needs to be always online. Thanks,
Posted by The3bl, 08-29-2008, 09:04 AM
http://www.coyotepoint.com/products/envoy.php
Posted by datums, 08-29-2008, 09:05 AM
Here's a way to do it using DNS. http://ultradns.com/solutions/sitebacker.html The think the best way to have datacenter redundancy would be using TCP anycast which could be an expensive solution.
Posted by calande, 08-29-2008, 09:21 AM
Thanks for the links. I've looked at them, and although I plan to write to the vendors, it seems that these solutions don't keep databases and file trees in sync accross servers, do they?
Posted by The3bl, 08-29-2008, 09:29 AM
I do not know of anyone that has out of the box software that does that. You have to use rsync and write cron jobs or use mysql sync scripts to do that.
Posted by cristibighea, 08-29-2008, 10:11 AM
I'd go for either sitebacker or DNSmadeeasy's DNS Failover service. Depending on how much SQL activity you have it could be quite hard to keep them all in sync all the time, but if there is no new data entered/modified every few seconds it should prove a bit easier to do. If you set everything up properly and configure Apache and MySQL well, these services should not fail, unless there is some problem with your application going haywire.
Posted by calande, 08-29-2008, 10:52 AM
Yes, the databases must be in perfect sync. Once a server comes back up, it has to find out the database activity that has occured during downtime and sync accordingly.
Posted by datums, 08-29-2008, 12:50 PM
Does it need to be 3 datacenters ? What are the requirements ? You can have a slave db at datacenter 2, auto promoted to master based on some criteria. These are just general recommendations. Without a the full requirements of the project it would be difficult to provide an intelligent solution. Will you be supporting the setup yourself?
Posted by Panopta, 08-29-2008, 04:03 PM
For database synchronization, if you're using MySQL the support for multi-master replication in 5.0 and later is pretty good. You can setup two servers to each be a master and a slave, or setup three in a ring. If you have good network connectivity between each location, the replication is pretty quick, and a machine that is down for a while will catch up quickly once it comes back online. There may still be some complications that really depend on your specific application - depending on how you're writing data you might hit some glitches when a node fails or comes back online. To really sort this out needs more information about the setup you're using.
Posted by Darvil, 08-29-2008, 04:30 PM
Hi Calande If you don't mind can you tell me what the apache problem is? I'm asking because I'm going to experiment with HA linux myself.
Posted by folsom, 08-29-2008, 04:38 PM
When you decide on a solution, make sure that you understand what it does when one or more servers are out of the loop. Say you have the three servers, and you want the database to be in sync all of the time what happens when there is some crazy net split, and you get this situation with servers A, B, and C: A can talk to C B can talk to C C can talk to A and B Which server is offline A or B? In that same net split, if server A updates a record, and tells server B to update the same record. Then Server C trys to update the same record yet again, what will happen? If you add another server and have servers A, B, C, and D: A can talk to B B can talk to A C can talk to D D can talk to C Which pair has the master set of data?
Posted by Nex7, 08-29-2008, 05:11 PM
Any good load balancer or load balancing software can do this. Good LB's will: a) Support being redundant themselves, with N+1 load balancers b) Be able to have multiple configured polls/checks that they run against the hosts behind them, and this includes content checks of HTTP URL's So just get a pair of good load balancers or good servers running good load balancing software, and then it is just a matter of handling the back-end database using some sort of replication or mirroring setup. edit: Oh and as for keeping the front-end web servers in sync in terms of the website files? Dozens of ways to do this. Depends on the circumstances. If they're files that only change due to content changes from your authors, just use rsync. If they have some sort of local non-database file changing going on all the time either have all the web heads use the same back-end SAN, or use some sort of network RAID1 setup across them all (performance won't be great that way but it'll get the job done).
Posted by axe9, 08-29-2008, 05:52 PM
What you want basically is a mirrored remote cluster right? If so, I would suggest adding a fourth machine to the mix that will control routing, syncing, and all other operations aside from actually serving webpages, so you do not tax the webservers too much with extranious operations