• This is a read only backup of the old Emudevs forum. If you want to have anything removed, please message me on Discord: KittyKaev

Eluna multithreading support

Status
Not open for further replies.

Rochet2

Moderator / Eluna Dev
NOTE! This is work in progress, not finished nor is it on the master branch.
See progress here https://github.com/ElunaLuaEngine/Eluna/commits/mthread

Intro:
Due to lua being designed for single thread, which forces us to use locks (slow everything else) and no good available systems for multithreading have so far been found, I started working on a similar approach LuaBridge has on arcemu.
The way it works is to have a lua state for each map and since a thread only works on one or more maps at a time, any code that works with just that map will be thread safe.
Now, this should make it possible that there will be no mutex locks or anything, except when a lua state loads or unloads, registering events (usually just when a script is loaded first time) or when registering any timed event.
And even then the implementation for the container using the said locks is a single write, multiple read solution, so as the data is only read, it shouldnt disallow other processes from running. Thus the locking should only affect loading and unloading a lua state. (map create / destory)

Script loding:
Since there are now multiple lua states, the script handling and such needs to change a little.
At the moment I set it up so that there are 4 folders.
Extensions that is loaded on all states before anything else.
Global that is loaded on all states after extensions
world that is loaded only on the world state ( a state that is not linked to any map and will always exist )
map that is loaded for all map specific states.

This will attempt removing redundancy in loaded scripts and it will remove the need for you to always check in the script if the lua state is a map or the global state.

Hooks & methods:
The multithreading will also affect hooks and some methods.
Now, before we didnt exactly need to worry about thread safety, so using something like GetPlayersInWorld was rather ok.
Now you simply cant do that. We would need to implement some kind of lock system for lua and I doubt anyone wants to use mutex locks and similar from lua.
To solve the problem similar methods to GetPlayersInWorld will be changed to for example the DoForPlayersInWorld, which allows you to execute any lua function for all the players.
Some functions and methods that try to access players from other maps than the current map the lua state is bound to will be limited to the world lua state or recreated as similar as DoForPlayersInWorld.
Depending on how we will implement this, some hooks only trigger on the global state, or all states and some will trigger only on the map state.

State communication:
As each map has its own state now, you can store instance variables easily just outside all other code or in the global namespace. (since every instance of a dungeon is its own map and has own lua state)
But then again, if this is the case, what about if I need to access variables or data from other maps or lua states if I cant save them for example in the global table?
For this problem I created a communication function. You can use SendStateMsg(map, inst, val1, val2, ...) to send any amount of basic type (string/nil/number/bool) variables to another map or the global state.
A hook is also created for this so you can receive the message from the other state and process it.

At the moment I cant recall anything other major so.. Ill post if I do.
Go ahead and ask if you have any questions.

Related:
http://emudevs.com/showthread.php/2706-Eluna-and-multithreading

that's actually a good idea, kudos
If that was a good idea, this is a really good idea.
 

efonius

Enthusiast
Hey, how is this coming along? I have been watching the development of it(mthread branch) with great excitement but recently it seems to have stopped, I hope everything is going good and I want to remind you that there are a lot of people who out there who appreciate all the effort you've put into this. Cheers.
 

Rochet2

Moderator / Eluna Dev
Hey, how is this coming along? I have been watching the development of it(mthread branch) with great excitement but recently it seems to have stopped, I hope everything is going good and I want to remind you that there are a lot of people who out there who appreciate all the effort you've put into this. Cheers.

The developement was stopped for now since I was working on AIO for a while as well as IRL problems.
Helping with developement is appreciated : | Not getting much, am I?
The whole point of mthread was to implement multithreading so we could ditch the mutex lock.
The need for the multithread solution was also not that needed atm as we anyways removed the thread lock and just limited the number of map update threads to 1. (which it is by default)
Mangos and TC are not multithreaded by default. (They are, but not in the way that would matter)

The mthread would only be needed if you wanted to use more than one map update thread, and that would ONLY apply to TC.
Using mthread will also change a lot how everything works so it will make a lot of scripts outdated as it is atm.

Atm i have not been working on it. Its nearly finished I guess .. hmm
A lot of functions need some sort of rewrite and the hooks may need some tweaking, but otherwise its good as it is atm.

It is not a top priority for me to get mthread out, as it would only benefit TC and change major part of how things work.
 

efonius

Enthusiast
The developement was stopped for now since I was working on AIO for a while as well as IRL problems.
Helping with developement is appreciated : | Not getting much, am I?
The whole point of mthread was to implement multithreading so we could ditch the mutex lock.
The need for the multithread solution was also not that needed atm as we anyways removed the thread lock and just limited the number of map update threads to 1. (which it is by default)
Mangos and TC are not multithreaded by default. (They are, but not in the way that would matter)

The mthread would only be needed if you wanted to use more than one map update thread, and that would ONLY apply to TC.
Using mthread will also change a lot how everything works so it will make a lot of scripts outdated as it is atm.

Atm i have not been working on it. Its nearly finished I guess .. hmm
A lot of functions need some sort of rewrite and the hooks may need some tweaking, but otherwise its good as it is atm.

It is not a top priority for me to get mthread out, as it would only benefit TC and change major part of how things work.

What about the changes to state communication that you mentioned in the original post? That seems like it would make scripts for instances and battlegrounds a lot more easy to make as well as make it safer for developers to use variables since it can't be accessed by other instanced versions of the same content.
 

Rochet2

Moderator / Eluna Dev
What about the changes to state communication that you mentioned in the original post? That seems like it would make scripts for instances and battlegrounds a lot more easy to make as well as make it safer for developers to use variables since it can't be accessed by other instanced versions of the same content.

That is true and that is one of the major changes it would bring. (and all the other things with it basically, everything here is about that)
However there is an extension on latest Eluna by default that you can use to easily overcome that without coding something yourself:
https://github.com/ElunaLuaEngine/Eluna/blob/master/extensions/ObjectVariables.lua
 
Status
Not open for further replies.
Top