Clustering a wowemulator

[kpl]

First of all - hello to everyone

Now to the business:
Recently I've came up with an idea of clustering a wow-server. Not just making the database in a master-slave config, but clustering the world server. I've came with a few concepts and I'm looking for information if they're allready scripted by some crew.

The ideas:
1) One worldserver on few machines. The load would be balanced on the auth server and spread to the machines. Like with HAproxy + memcached or some other different/better solution.
1.1) A simmilar to 1) but each continent might be splited into seperate machines / VM's. I don't have the exact solution, just the idea.
2) A failover cluster - when one machine dies (load, server crash or even hardware fialure) than it's just needs a few secs (if evrything goes well) and switches to the 2nd (backup) machine.

I've came up with some questions:
1) Has any emu actually done that? Documented, working and not abandoned?
Just like here (yeah, I'm answering a part of the question. But maybe there are different branches that I don't know about):

 https://youtu.be/WxvaPZGqtV0

 https://youtu.be/mdZZF2qtZyA

2) How could it be done? To be exact I'm wondering how could be data transferred inbetween the machines. Are all the players stored in the RAM or (this might be stupid, due to efficiency) as files on HDD or maybe stored in a kind of database?
3) Also does any emu have a good documentation for the work they're doing, just in case some young padawan would create the basics for that?

I hope I've posted in the right section of the forum.
Sorry for all mistakes in the text. English is not my native.

Cheers,
K.

 

[Rochet2]

Even if overall it might be "easy" to get clustering working as shown in the videos it might be more difficult to deal with all the problems that come with it.
The current code makes all kinds of assumptions. Originally mangos was single threaded and a lot of the code did not take into account what multithreading would do or was not coded properly to work with multiple threads. TC has multithreading, but not that long ago there has been fixes for crashes due to bad code. Threading erros are hard to detect and same will be with a lot of the synchronization and other errors with the clustering etc.

I dont myself know of any public source with clustering, but I have not looked either. Someone probably has some fork sitting in a folder, but it might not be public.
Documented? Probably not. Probably 90% of the code related to emulators isnt exactly documented from what I can see.
But on a very rough scale everything is pretty clear. Maybe ask directly from the (whatever core) developers instead?

The data is sent around in any way you implement it in. For example you could have TCP connections between all servers.
All players are on TC stored on the hard drive (mysql) and moved to RAM when the player is online. This could be moved to complete ram on start up, but that would make the server startup long(er) and consume a lot of ram on high populated servers, which isnt what the server is really designed for.
Most of the servers are designed as private servers for some people - not some dedicated million player server with assumed tons of resources like 1000GB ram and reliability and blah blah.

Surely having clustering and whatnot would be cool and useful, but implementing them and working through the restrictions and maintenance needs they add can be an unwanted load.
Even as is emulators can hold hundreds or thousand(s) of players as seen - though unsure about how some might have been modified to hold as many as they can hold.


I could be wrong, but that is how I see things are.

If you are up for it, then good luck : )
Also I believe there was /some/ posts about this on TC forums.

 

[kpl]

Hello again,

Thank you Rochet2 for your response.

I've managed to figure out a solution for my 2nd question. It's called Heartbeat. In my free time I might try to create it on some virtual machines. If anyone would be interested in the results I can share them.

I have spoken to people on TC and cMangos IRC channel. If I understanded everything correctly none of the crews are planning to do clustering.
So there are two ways. One is to find those branches mentioned in the video. For cMangos it might be slightly easier due the fact it's a film from this year. But even though, I'm more interested in TC.

The other solution is to try to figure it out on my own. But that won't come anytime soon, as I need to refresh everything and bite into the code.
I know this is a bit of offtopic, but what is a good starting point for the emulator itself? Probably I'll need to go through each part on my own. But if someone has done that in some idiot-proof way it's written down somwhere, than it would be a relief. Any suggestions?

The data is sent around in any way you implement it in. For example you could have TCP connections between all servers.

So if the worldserver would have a modular build and we would use SOCKETS to communicate or RPC, XML-RPC, whatever - we could transfer the data between the nodes. I know it's way more complicated, just an idea.

Most of the servers are designed as private servers for some people - not some dedicated million player server with assumed tons of resources like 1000GB ram and reliability and blah blah.

I totally understand it. My idea and "problem" is just for the sake of fun, education and making WoW a better place. God, that sounded nerdy.
On the other hand, what if someday an enthusiast would like to create a big wow-like realm? Something like ex-molten with their high pop (dunno if the numbers are real).

Cheers,
K.

 

[Rochet2]

Dont know what would be best to go look at, but as an overview of the cores there are usually a list of opcodes and their handlers (opcode.cpp) and then there are a bunch of managers like guildmgr and auctionmgr and mapmgr etc that handle various specific systems.
There is a single loop (world.cpp) that is the program loop and it uses a bunch of threads to update all maps on each loop at one point of the loop. It waits so they are done and continues.
Everything revolves a lot around the time from the main program loop (timers use the diff between the loops).

So to get things going you should probably take a look at what sends and receives stuff to and from the opcode handlers. (worldpacket, worldsession ...)

And sure, I understand doing it for the sake of doing it. Sounds like fun stuff to think about.

 

[Kaev]

Do you have any experience in network programming, services, clustering and all the stuff?
If the answer is no: Start small. Write 2 small servers with little functionality and try to start there.
It's a huge and complicated task which can't be done without pretty good experience.

IMO it would be easier to write a new emulator where everything is already planned to work with clustering instead of modify MaNGOS, TC or whatever core you want to modify.

EDIT: iirc MaNGOS VB has some clustering functionality.

 

[kpl]

@Rochet2

Thanks. I'll look through everything. Although reading the info provided by Sunwell server crew (they're closing the server) didin't ecourage me much.

Do you have any experience in network programming, services, clustering and all the stuff?
If the answer is no: Start small. Write 2 small servers with little functionality and try to start there.
It's a huge and complicated task which can't be done without pretty good experience.

I might just give it a shot. I'm on my last year in computer science engineering. My experience is rather pure-academic. Done courses with C, C++, network programming, databases, software design. So I'll see what I can do.