Plugin Lua Lanes

[BlubBlab]

For all those who need real threads in MM this should be the solution.
http://cmr.github.io/lanes/
I compiled it for x86 and Lua 5.2, put the lanes.dll in the plugin folder and the two *.lua files in the lib/mods folder.

I haven't tested yet it full functionality but it integrated without errors. I'm testing it while I try to develop a way to bypass the hang up of the bot when rom crashes.

So beware that those are real threads and their for not thread-safe unlike coroutines you need to exactly know what are you doing !!!
(sry for my bad english)

[BlubBlab]

For Future use:
Okay I testes it what you can do now is use the Lua library , pass data to the thread and get the return value.
Unfortunately you can't access global values like vars or functions that are outside the thread it would need changes on the code from MM , add a functions and change 2th calls .......http://cmr.github.io/lanes/#deep_userdata

[Administrator]

Unfortunately you can't access global values like vars or functions that are outside the thread it would need changes on the code from MM

I would have to assume that is because of thread safeness. Typically, using threads, it would work the exact same way unless you're using some system (such as a mutex) to ensure that only one thread can access a memory section at a time. I'm not entirely sure that using an ID function will resolve all the issues mentioned but I don't know much about how Lanes works.


Just an idea here, but can't you pass a reference to the global space (_G) to a thread, then use that to access global variables and functions?

[BlubBlab]

I will try later but the reason you can't access global datas is it is basically each thread(lane) is a separate instance of the VM each with its own Lua State, so this function I mentioned is mirror to reflect datas across the lua stats at least it is what I did understand.

That doc isn't what it should be Its awful their are plenty of holes of missing information and misguiding informations.

What I didn't mentioned is you can pass information through lanes with the help of lindas which are pipes/queues but doesn't help in that case because if the main thread gets stuck the whole application freeze because that are blocking queues which you can use also as mutex.

The only way I see that can work(with out changing the MM code) is copy&paste all needed functions as local function in the thread(which is a anonymous function in lua syntax) and hope I can at least access the C function from MM ,which is an ugly solution.

[Administrator]

Ah, right. I wasn't thinking clearly when I wrote that last post, I guess. Of course passing _G wouldn't work. I like the idea of real threading, however, I have a feeling it would require a lot more work than just adding that ID function.

Are you able to compile MicroMacro? If so, you can try getting it working then submit a patch file to me.

[BlubBlab]

Okay I read the example implementation(or better the linda version) , seems your were right this only to add a custom function which can share information not a general share function for the global data.

I did find a method to bypass the blocking simply peek on the linda(pipe) but the bottle neck is still their it's part of the design I think nobody can do anything about it not whiteout great changes on the original code.

[BlubBlab]

After a long time I read again the API and came across examples
I didn't find away to write to global vars but access any extern function as a working copy.
The idea with _G was the right one

Code: Select all

	local thread = lanes.gen("*",{globals = _G},func)(args)

I haven't much time at the moment so I couldn't test much but I wrote wrappers which are similar to the original MM function to show how things work. If someone wants to experiment with this have fun.

[MiesterMan]

For Future use:
Okay I testes it what you can do now is use the Lua library , pass data to the thread and get the return value.
Unfortunately you can't access global values like vars or functions that are outside the thread it would need changes on the code from MM , add a functions and change 2th calls .......http://cmr.github.io/lanes/#deep_userdata

I haven't finished reading responses to this so this may change but I'm pretty sure that defeats the purpose of using threads over child proccesses. The biggest advantage to using a thread is the shared memory space (easy access to the parent program) which you lose when creating child proccesses. It sounds like this threading plug-in is just not designed properly.

[MiesterMan]

I will try later but the reason you can't access global datas is it is basically each thread(lane) is a separate instance of the VM each with its own Lua State, so this function I mentioned is mirror to reflect datas across the lua stats at least it is what I did understand.

That doc isn't what it should be Its awful their are plenty of holes of missing information and misguiding informations.

What I didn't mentioned is you can pass information through lanes with the help of lindas which are pipes/queues but doesn't help in that case because if the main thread gets stuck the whole application freeze because that are blocking queues which you can use also as mutex.

The only way I see that can work(with out changing the MM code) is copy&paste all needed functions as local function in the thread(which is a anonymous function in lua syntax) and hope I can at least access the C function from MM ,which is an ugly solution.

Ok, yep, that right there. You just described a fork or child process, not a thread. This is a tedious and undesirable method of working multi-processing I would not reccomend pursuing. It is not the way to go.

[BlubBlab]

That is true I was really disappointed when I had found out it would have been easy when the vars were shared in the heap.
I can only guess they did it this way because it was easy to implement or they thought it would be less errors by programming this way.
Basically they force you to use the http://en.wikipedia.org/wiki/Functional_programming paradigm which is really needs getting used to.

Theoretical there is still a way like I said there is the possible the implement your own shared memory space like the Lindas, so instead a queue it could be a table.
Like:

Code: Select all

Table.WriteObject("coordX",coordX);
and  vice versa
coordX = Table.ReadObject("coordX");

I tried to find other implementations not much luck. I found some mentioned but all have their flaws one replace the COROUTINE and have no looks in it I think, others aren't maintained any more and others need deep change of the LuaVM code.

EDIT:Thier is no need to follow the example in the code 1:1 a dll which allocate the needed memory in the heap for the lua_object combined this with a hash map that should do the job. I think someone with experience in this could do that in an hour unfortunate I'm not, never interfaced C and Lua. Okay tables could be difficult.

[BlubBlab]

I did make a search again after those long time for lua and multithreads but I can now definitely explain why it work the way it does.
Okay first I haven't found any new project to that topic. The reason it work like it does is:
In one word the stack the Lua engine has only one stack, you would need to change the code of the lua vm so each thread has it's own stack.

So to accomplish they start multiple instances of the LuaVM(multiple stacks), what does split the heap.

The other way is overwrite the COROUTINE in a dll because so far I did understand COROUTINE has already it's own stack.
The problem with that is
1.) Incompatible with rombot through the fact it use for the timer and yrest COROUTINE it would simple crash.
2.) The code I had found had no lock method
This code was only about 500 lines of code, unfortunately I swept it from my harddrive.*grumel*

[Administrator]

It might be possible with MicroMacro 2 to run multiple Lua states and have some shared space between them. That shared space would have mutexes in place to prevent multiple threads from accessing the same variable(s) at the same time. As far as trying to get this to work with the current version of MicroMacro, forget it. I wouldn't want to touch that with a 10 foot pole.

[BlubBlab]

No problem but I have another idea to add for MM2 , so far I remember their were some issues with 64-bit pointer/addresses .
If they still their you could use string instead number for the MM function and using a BigInt implementation on the Lua side.^^

[Administrator]

Already have 64-bit integers in; that's no problem. The problem is that it is impossible to thunk a 64-bit process from a 32-bit process in Windows.
So, the only way it could work is if there's a 64-bit version of MicroMacro. That may or may not come about.