Rochet2
Moderator / Eluna Dev
NOTE! This is work in progress, not finished nor is it on the master branch.
See progress here https://github.com/ElunaLuaEngine/Eluna/commits/mthread
Intro:
Due to lua being designed for single thread, which forces us to use locks (slow everything else) and no good available systems for multithreading have so far been found, I started working on a similar approach LuaBridge has on arcemu.
The way it works is to have a lua state for each map and since a thread only works on one or more maps at a time, any code that works with just that map will be thread safe.
Now, this should make it possible that there will be no mutex locks or anything, except when a lua state loads or unloads, registering events (usually just when a script is loaded first time) or when registering any timed event.
And even then the implementation for the container using the said locks is a single write, multiple read solution, so as the data is only read, it shouldnt disallow other processes from running. Thus the locking should only affect loading and unloading a lua state. (map create / destory)
Script loding:
Since there are now multiple lua states, the script handling and such needs to change a little.
At the moment I set it up so that there are 4 folders.
Extensions that is loaded on all states before anything else.
Global that is loaded on all states after extensions
world that is loaded only on the world state ( a state that is not linked to any map and will always exist )
map that is loaded for all map specific states.
This will attempt removing redundancy in loaded scripts and it will remove the need for you to always check in the script if the lua state is a map or the global state.
Hooks & methods:
The multithreading will also affect hooks and some methods.
Now, before we didnt exactly need to worry about thread safety, so using something like GetPlayersInWorld was rather ok.
Now you simply cant do that. We would need to implement some kind of lock system for lua and I doubt anyone wants to use mutex locks and similar from lua.
To solve the problem similar methods to GetPlayersInWorld will be changed to for example the DoForPlayersInWorld, which allows you to execute any lua function for all the players.
Some functions and methods that try to access players from other maps than the current map the lua state is bound to will be limited to the world lua state or recreated as similar as DoForPlayersInWorld.
Depending on how we will implement this, some hooks only trigger on the global state, or all states and some will trigger only on the map state.
State communication:
As each map has its own state now, you can store instance variables easily just outside all other code or in the global namespace. (since every instance of a dungeon is its own map and has own lua state)
But then again, if this is the case, what about if I need to access variables or data from other maps or lua states if I cant save them for example in the global table?
For this problem I created a communication function. You can use SendStateMsg(map, inst, val1, val2, ...) to send any amount of basic type (string/nil/number/bool) variables to another map or the global state.
A hook is also created for this so you can receive the message from the other state and process it.
At the moment I cant recall anything other major so.. Ill post if I do.
Go ahead and ask if you have any questions.
Related:
http://emudevs.com/showthread.php/2706-Eluna-and-multithreading
See progress here https://github.com/ElunaLuaEngine/Eluna/commits/mthread
Intro:
Due to lua being designed for single thread, which forces us to use locks (slow everything else) and no good available systems for multithreading have so far been found, I started working on a similar approach LuaBridge has on arcemu.
The way it works is to have a lua state for each map and since a thread only works on one or more maps at a time, any code that works with just that map will be thread safe.
Now, this should make it possible that there will be no mutex locks or anything, except when a lua state loads or unloads, registering events (usually just when a script is loaded first time) or when registering any timed event.
And even then the implementation for the container using the said locks is a single write, multiple read solution, so as the data is only read, it shouldnt disallow other processes from running. Thus the locking should only affect loading and unloading a lua state. (map create / destory)
Script loding:
Since there are now multiple lua states, the script handling and such needs to change a little.
At the moment I set it up so that there are 4 folders.
Extensions that is loaded on all states before anything else.
Global that is loaded on all states after extensions
world that is loaded only on the world state ( a state that is not linked to any map and will always exist )
map that is loaded for all map specific states.
This will attempt removing redundancy in loaded scripts and it will remove the need for you to always check in the script if the lua state is a map or the global state.
Hooks & methods:
The multithreading will also affect hooks and some methods.
Now, before we didnt exactly need to worry about thread safety, so using something like GetPlayersInWorld was rather ok.
Now you simply cant do that. We would need to implement some kind of lock system for lua and I doubt anyone wants to use mutex locks and similar from lua.
To solve the problem similar methods to GetPlayersInWorld will be changed to for example the DoForPlayersInWorld, which allows you to execute any lua function for all the players.
Some functions and methods that try to access players from other maps than the current map the lua state is bound to will be limited to the world lua state or recreated as similar as DoForPlayersInWorld.
Depending on how we will implement this, some hooks only trigger on the global state, or all states and some will trigger only on the map state.
State communication:
As each map has its own state now, you can store instance variables easily just outside all other code or in the global namespace. (since every instance of a dungeon is its own map and has own lua state)
But then again, if this is the case, what about if I need to access variables or data from other maps or lua states if I cant save them for example in the global table?
For this problem I created a communication function. You can use SendStateMsg(map, inst, val1, val2, ...) to send any amount of basic type (string/nil/number/bool) variables to another map or the global state.
A hook is also created for this so you can receive the message from the other state and process it.
At the moment I cant recall anything other major so.. Ill post if I do.
Go ahead and ask if you have any questions.
Related:
http://emudevs.com/showthread.php/2706-Eluna-and-multithreading
If that was a good idea, this is a really good idea.that's actually a good idea, kudos