Page 1 of 2 12 LastLast
Results 1 to 25 of 30
  1. #1
    Join Date
    Oct 2005
    Posts
    435

    A thought on forum clustering

    This was just an idea I had and probably not viable but I thought I'd post here to see what the pros think.

    Lets say I have 2 or more servers and I want to have a better uptime for the forum and spread the load. Lets say the forum is around 300 simultaneous users but most of mysql it is browsing and not writes to the database. Lets say I have a way to spread the load between the 2 servers.

    I know the master to master mysql replication isn't viable if the servers are quite apart. What about using master to slave replication? If the forum software was modified (I have no idea how complicated this is) so that writes are done remotely to the master server and all the reads are done locally. In the case where the master goes down, the people who are on the slave server (and those that gets redirected to the slave server) won't be able to post anymore but they would still be able to browse the server (I think).

    To me this seems like a viable way. What do you guys think about this? I know it probably would be a bad idea if the forum is a massive forum but what about a small one relatively speaking.

  2. #2
    Join Date
    Feb 2004
    Location
    UK
    Posts
    1,431
    Hi

    Its a good idea, but like you say you need to send writes to one server and reads from both.

    I believe Vbulletin has this built in as a feature.

    Thanks

  3. #3
    Join Date
    Apr 2011
    Posts
    351
    I think, this would be very difficult todo.
    You'd have to create alot of coding between master and slave.

    It does seem possible though, ever better would be all servers in the same rack.

  4. #4
    Join Date
    Oct 2005
    Posts
    435
    abtme,

    Vbulletin has this feature?? Really.. thats quite incredible. I guess I will have to look it up but I feel like it wouldn't be builtin.

    intelliServe,

    Why would you have to create alot of coding between master and slave? You just have to put all the writes of the slave to the master db; things like registrations, forum posts and profile updates. But after thats posted to the master, the master will replicate the data to the slave so that should work ok in theory.

  5. #5
    Join Date
    Apr 2011
    Posts
    351
    It would need additional coding because you seem to be wanting to use it as a secondary with one server still reading if the write db is down or vice-versa.
    This would mean you'd have to split the functions, these aren't done by default, however if vb includes this then it doesn't matter.

  6. #6
    Join Date
    Feb 2004
    Location
    UK
    Posts
    1,431
    Well im fairly sure it does... I now am doubting myself.

    In the config.php it mentions slave... Im 99% sure of it..

  7. #7
    Join Date
    Apr 2003
    Location
    NC
    Posts
    3,093
    Quote Originally Posted by abtme View Post
    Well im fairly sure it does... I now am doubting myself.

    In the config.php it mentions slave... Im 99% sure of it..
    Yeah it does and has for awhile. Very handy feature.

    You can also use memcached to help with things like sessions if you want.

    You can also look at caching some of the content
    John W, CISSP, C|EH
    MS Information Security and Assurance
    ITEagleEye.com - Server Administration and Security
    Yawig.com - Managed VPS and Dedicated Servers with VIP Service

  8. #8
    Join Date
    Oct 2005
    Posts
    435
    abtme,

    I think you might be right.

    http://www.vbulletin.com/docs/html/config.php

    But I don't exactly know how that works according to the config but looks like you can use it that way from looking at that.

    If vbulletin does it, wouldn't invision power board do the same? Anyone know?

  9. #9
    Join Date
    Oct 2005
    Posts
    435
    I literally just found this

    http://community.invisionpower.com/f...-mysql-driver/

    I guess you can do a similar thing with IPB using this mod.. But 75 is spendy.

  10. #10
    Join Date
    Apr 2003
    Location
    NC
    Posts
    3,093
    Quote Originally Posted by Darvil View Post
    I literally just found this

    http://community.invisionpower.com/f...-mysql-driver/

    I guess you can do a similar thing with IPB using this mod.. But 75 is spendy.
    If $75 is a lot do you really need clustering or perhaps some optimization would do the trick for now? Generally when we start load balancing sites for performance they are already on some decently expensive hardware (but not always!)
    John W, CISSP, C|EH
    MS Information Security and Assurance
    ITEagleEye.com - Server Administration and Security
    Yawig.com - Managed VPS and Dedicated Servers with VIP Service

  11. #11
    Join Date
    Oct 2005
    Posts
    435
    eth00,

    I don't need clustering done at all. My server is plenty powerful even if the forum grows 10 times bigger.

    The thing is I have another server and probably one more in the future. I am just wanting to do this as a fun project to do.

    I was reading the info on the 75 dollar mod above, but it looks like its not what I think it is.

    I don't think in a setup like that the forum can survive if the master server dies. Its just distributing the mysql load but not the whole site. Thats actually not what I was thinking.

    That makes me wonder if its the same thing with the vbulletin setup.
    Last edited by Darvil; 04-26-2011 at 03:55 PM.

  12. #12
    Join Date
    Mar 2009
    Posts
    3,816
    Are you already using multiple memcached servers?

  13. #13
    Join Date
    Mar 2010
    Location
    Germany
    Posts
    697
    Quote Originally Posted by Darvil View Post
    This was just an idea I had and probably not viable but I thought I'd post here to see what the pros think.

    Lets say I have 2 or more servers and I want to have a better uptime for the forum and spread the load. Lets say the forum is around 300 simultaneous users but most of mysql it is browsing and not writes to the database. Lets say I have a way to spread the load between the 2 servers.

    I know the master to master mysql replication isn't viable if the servers are quite apart. What about using master to slave replication? If the forum software was modified (I have no idea how complicated this is) so that writes are done remotely to the master server and all the reads are done locally. In the case where the master goes down, the people who are on the slave server (and those that gets redirected to the slave server) won't be able to post anymore but they would still be able to browse the server (I think).

    To me this seems like a viable way. What do you guys think about this? I know it probably would be a bad idea if the forum is a massive forum but what about a small one relatively speaking.
    *snicker*
    I think generally people that can write well-distributed apps are not the ones writing forums :>

    BUT!!! that doesn't make you wrong at all!
    Check out my SSD guides for Samsung, HGST (Hitachi Global Storage) and Intel!

  14. #14
    Join Date
    Oct 2005
    Posts
    435
    quantumphysics, not at all. As I've mentioned, I was thinking more of this as a wide distributed set of servers then a few local servers. I was just thinking about this and see if its viable thats all.

    wartungsfenster,

    Everyone gotta do something I suppose that means you agree it can be done?.. So where can I find one of these elite programmers who can write it up (but I probably can't afford them eh?.

  15. #15
    Join Date
    Mar 2009
    Location
    Austin, TX
    Posts
    935
    Those of you with vB4 (single server). How much memory does your memcached take? I've tweak the memory setting and it still uses around ~90MB. No matter if I set memcached to use 128MB or 256MB.
    SysAdmin.xyz
    Having severs with customer data on it without proper monitoring is like having one night stand without using protections - eventually, there will be an 'oh s**t!' moment.

  16. #16
    Join Date
    Apr 2011
    Location
    Massachusetts
    Posts
    123
    Splitting reads and writes to a mysql server is not all that hard at all. You setup some type of mysql proxy that sends commands that makes writes to the master server and commands that read are sent to the slave. As long as you had a session aware load balancer in front of the two web servers it would work fairly well. It isn't perfect but it would give pretty good performance boosts to something that is more read intensive. A really large site would use custom written software and sharding.
    Last edited by kmonchamp; 04-26-2011 at 07:57 PM. Reason: spelling

  17. #17
    Join Date
    Oct 2005
    Posts
    435
    can you elaborate on "some type of mysql proxy"?

    although remember I'm thinking of 2 different servers located in different datacenters. Basically using a clustering DNS solution (nginx, varnish?) to route to the different servers. I guess that would mean its not session aware. I forgot about that. Perhaps store the sessions in the db?

    Well the coding part will be an issue if everytime you upgrade the forum software you have to redo everything. Plus you would need to hire someone who knows the forum software intimately.

    I like the idea of puting something in the front that can split the read and write so you won't have to really mess with the forum code. Try to keep it minimal work.

  18. #18
    Join Date
    Apr 2011
    Location
    Massachusetts
    Posts
    123
    https://launchpad.net/mysql-proxy Is one of the programs that comes to mind. Mysql replication can work in two different datacenters although it can become out of sync. Semi-synchronous mysql replication was a recently added feature that makes geographic redundancy more viable and fixes out of sync issues.

  19. #19
    Join Date
    Oct 2005
    Posts
    435
    kmonchamp,

    Thanks for that link! Now I looked though that but I am not 100 percent sure. Let me see if I can guess how it would work with mysql-proxy.

    Lets say I have 3 servers in 3 different datacenters. 1 master and 2 slaves. I won't need mysql-proxy infront of the master. I have mysql-proxy in front of the 2 slaves filtering the queries. Whenever there are write queries, the mysql-proxy can then redirect all those queries to the master server and let everything else go through (IE select) to the slaves.

    Would that be how it would work? What about if the master isn't available? Can you script it to store the queries or does it just return a typical error on the forum.

    That would be quite incredible if it works that way so you won't need to recode the forum script. Just have to install mysql-proxy and script it a lil.

  20. #20
    Join Date
    Apr 2011
    Location
    Massachusetts
    Posts
    123
    The basic idea is that the proxy gives you one IP address for the entire mysql system. The mysql proxy then handles the separation. You would use mysql replication to replicate data between servers.

    If you wanted a high availability system(automatic failover) you would setup the servers with multi-master replication(both servers can be read and written to, you would use mysql proxy to write to only one and if one is down it would just start using the other).

    A note on mysql replication. Currently there are two types of mysql replication available, asynchronous and semi-synchronous. In asynchronous data is written to a log and then set to a slave. There is no guarantee that the slave receives the file. In new versions of mysql semi-synchronous replication was added to fix this problem. This guarantees the file was delivered to the slave. This version uses a bit more resources but is the best way to ensure better data redundancy if you want to use automatic fail-over. If you have less bandwidth then use asynchronous because it will very rarely loose data(really it will only loose data in the event the master server crashes right after it finishes processing a transaction). Its best to use a HA or load balanced systems where if some data is missing or delayed nothing bad would happen(example a forum post takes a bit extra time to show up on the other server or is lost during fail over) again semi-synchronous fixes most issues with data loss.

    You can see that this can get really complicated really fast but is a good way to add improved availability/load without having to make any modifications to the software.


    Mysql also has another project called cluster that has full synchronous replication and high redundancy. It also uses ram drives and then offloads data to a hard disk for improved speeds. However it is useless for anything but custom written software and has other limitations that would make it unsuitable for forums.

  21. #21
    Join Date
    Oct 2005
    Posts
    435
    thanks kmonchamp,

    But with master to master, you're talking about 2 local servers right? It wouldn't be viable to do this over large geographical servers (IE one in US, one in europe and one in Asia. If you say the proxy gives 1 IP, then I'm assuming that all reads and writes goes thru it which means the sites would browse slow. I was thinking more along the line if, if you were in europe, you would hit the european server with the dns cluster and then you can browse the forum on that server. When you write then, it goes to the master server instead of the slave.

    I don't think the proxy will work well in this case, if what I am understanding is correct.

  22. #22
    Join Date
    Apr 2011
    Location
    Massachusetts
    Posts
    123
    Yeah its not really the best option for geographic redundancy. Actually mysql is not really all that good for geographic redundancy with out custom written software, like I said custom applications use techniques such as sharding. Only mysql goes through one IP(it might be able to be setup in other ways as I use multi master semi-sync), the real performance benefit would be to increase load on the web servers and spread out php load. It would really help if your load problems were mysql.

    And it really all depends on how perfect the replication needs to be?

    What larger sites might do is use multi master load balanced locally and then use asynchronous replication to tie different geographic areas together.


    EDIT EDIT EDIT EDIT
    I was thinking a little more and seeing as each forum would be running on different servers you could run mysql proxy on all of them (even the master encase mysql goes down on that machine). I think that would work. Each forum just points to its own servers proxy.
    Last edited by kmonchamp; 04-27-2011 at 12:03 AM.

  23. #23
    Join Date
    Oct 2005
    Posts
    435
    kmonchamp,

    Thats what I was thinking initially. All of them have their own mysql proxy but only the writes from the slaves would arrive at the masters. It wouldn't be good if all the reads and writes have to come to the master server as that would be pointless. The thing I found is that, at least on my end, most of the mysql traffic is just select. I was thinking if there was a local server (IE in europe) then the browser would be fast which would be for like 90 percent of the use.

    But in this senario, what would happen if the master fails? Lets just say it got cut off. Those users hitting the master server would also get rerouted to the slaves. These slaves won't be able to update anymore (as the master is now down) but would they still run? at least be able to browse. I was wonder what happens in this senario. Will the forum just throw a regular db error if a user try to write? but the forum will still be alive and viewable.

    Also how would sessions work in this case? I didn't think about it. I'm assuming those that were hitting the master server that just got rerouted to the slaves now, have to relogin to the forum. But would they be able to login as logging in itself needs a write to the database (IE last logged in field). Would that mean they can still browse the site but not login.

  24. #24
    Join Date
    Apr 2011
    Location
    Massachusetts
    Posts
    123
    Mysql proxy is not exactly perfect in splitting read/writes at this time(it works fairly well, just somethings still end up going to the current master) so it would highly depend on how the software is written to determine if it would still work. What I would recommend is setup mysql in a circular replication(multi-master when more then two servers). So that the remaining two servers could still keep each other in sync. Or what ever script you use to watch for failure could connect to the mysql databases and change that servers master. (http://mysql-mmm.org/ that should point you in the right direction)

    Realistically I would imagine that with session information you should have no problem. It gets recorded into the data base and the information will be replicated to all of the servers. If they end up having to connect to another server their same session info should work.

  25. #25
    Join Date
    Mar 2009
    Location
    Austin, TX
    Posts
    935
    I was messing with circular multi-master setup w/ federated tables a while back. Thing just never worked during a failover process.

    Thanks for pointing mysql-mmm out. Have you deployed in production and have good success with it?
    SysAdmin.xyz
    Having severs with customer data on it without proper monitoring is like having one night stand without using protections - eventually, there will be an 'oh s**t!' moment.

Page 1 of 2 12 LastLast

Similar Threads

  1. Clustering help
    By Warpline in forum Systems Management Requests
    Replies: 4
    Last Post: 02-12-2011, 02:50 PM
  2. Clustering.
    By Simwah in forum Hosting Security and Technology
    Replies: 17
    Last Post: 05-07-2006, 12:33 AM
  3. Thought for the day...(insert your thought here)
    By lorandm in forum Web Hosting Lounge
    Replies: 28
    Last Post: 05-19-2004, 07:59 PM
  4. Clustering
    By intigret in forum Web Hosting
    Replies: 44
    Last Post: 12-14-2002, 07:13 AM
  5. i thought there were rules in the requests forum?
    By richy in forum WHT Announcements, Feedback and Questions
    Replies: 31
    Last Post: 11-24-2002, 11:03 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •