Intro To Multi-Threading

M

Mystborn

Guest
GM Version: Definitely GMS1, probably GMS2, possibly earlier versions.
Target Platform: All (I think)
Download: Source
Links:
Summary:
This tutorial goes through the steps to create some basic multi-threaded capabilities in Gamemaker using an extension written in c++. (If it's allowed, I'll be writing an almost exact replica, but instead of c++, I'll show how to do this in c#, so if you prefer that, it'll probably be best to wait a few days for it).

Tutorial:

If you have any questions, suggestions, or fixes please let me know. (Also sorry about the vid length. It's a bit of a hefty topic).

Major thank you to Y(ellow)A(after)L(ife) for helping get on the right track to creating this. Without him this wouldn't have come to fruition.
 
G

g3tinmybelly

Guest
I would love to see the C# implementation of this if you are willing to make a tut about it.

Also, this would work inside GMS:2 right?
 
M

Mystborn

Guest
I'm almost done with the c# vids :)

Both versions should work in GMS:2, but I have no way to test it personally. If anybody could confirm I'd really appreciate it.
 
G

g3tinmybelly

Guest
Because this is a dll, would this not work for any other platform beside windows? Like, if you used .NetCore would this work for both mac and linux?
 
M

Mystborn

Guest
This library was written in c++ using it's cross-platform threading library, so it should work pretty much anywhere. I do have the first two videos for the c# replica up on the channel, (which should be enough to get you started, especially if you watch this one). However, you need a special library in order to use c# in gamemaker, and currently there is no dotnet core equivalent that I've found. Basically you need a way to make c# code look like unmanaged code when it's compiled, so if dotnet core already does that or there's some way to do that, then you can definitely use it to make the c# version cross-platform. Otherwise you're stuck using c++
 

PNelly

Member
Well done! Super cool extension.

Single threading is one of GM's weaknesses for building network servers, could this extension to be used to spawn a thread that handled each client individually in a networked game? Or would the networking tools need to be moved into an external library as well? I guess at that point you might as well ditch GM for the server and write it in another language.
 
M

Mystborn

Guest
@PNelly
Well done! Super cool extension.

Single threading is one of GM's weaknesses for building network servers, could this extension to be used to spawn a thread that handled each client individually in a networked game? Or would the networking tools need to be moved into an external library as well? I guess at that point you might as well ditch GM for the server and write it in another language.
That's definitely the biggest weakness of this extension, you can only execute external code in other threads. That's why I showed how to send callbacks from other threads to gamemaker which allows you to execute gml code asynchronously, but still on the main thread. If you truly wanted to use this for a server, your best bet would be to retrieve and read the buffers into variables in gamemaker, send the data to a function in you c++ library that would run computations on another thread and return the data back to gamemaker to send back to the clients. Definitely not the best way to go about it.

However if you wanted a threaded server library I'd suggest this wonderful library for c# that was made specifically to work with GM's networking functions. Still has to be done in another language, but at least you can skip most of the junk to get it working and use it out of the box. Here's his tutorial on how to use it :)
 
D

damand23

Guest
GM Version: Definitely GMS1, probably GMS2, possibly earlier versions.
Target Platform: All (I think)
Download: Source
Links:
Summary:
This tutorial goes through the steps to create some basic multi-threaded capabilities in Gamemaker using an extension written in c++. (If it's allowed, I'll be writing an almost exact replica, but instead of c++, I'll show how to do this in c#, so if you prefer that, it'll probably be best to wait a few days for it).

Tutorial:

If you have any questions, suggestions, or fixes please let me know. (Also sorry about the vid length. It's a bit of a hefty topic).

Major thank you to Y(ellow)A(after)L(ife) for helping get on the right track to creating this. Without him this wouldn't have come to fruition.
Hi Mystborn,

I followed you tutorial, but when i run it after adding all the function 'thread_create, thread_kill, thread_join and RegisterCallbacks to Extension, and run it crashes at function ds_map_create in Threading.dll. I am sorry I am new to GM and C++, was wondering if you have nay suggestions on that what I am doing wrong.

Thanks
 
M

Mystborn

Guest
Hi Mystborn,

I followed you tutorial, but when i run it after adding all the function 'thread_create, thread_kill, thread_join and RegisterCallbacks to Extension, and run it crashes at function ds_map_create in Threading.dll. I am sorry I am new to GM and C++, was wondering if you have nay suggestions on that what I am doing wrong.

Thanks
That definitely seems like there is an error either in your RegisterCallbacks function in the dll or it's definition in GM. The pointer casts have to be exactly correct or else it won't work correctly. Otherwise the ds_map_create() function is probably the least likely thing to crash :p

Could you paste the code for RegisterCallbacks and preferably a screenshot of the GM implementation window thingy (The place where you define the external functions)? Also paste the code for the initial function pointer definitions.
 
D

damand23

Guest
That definitely seems like there is an error either in your RegisterCallbacks function in the dll or it's definition in GM. The pointer casts have to be exactly correct or else it won't work correctly. Otherwise the ds_map_create() function is probably the least likely thing to crash :p

Could you paste the code for RegisterCallbacks and preferably a screenshot of the GM implementation window thingy (The place where you define the external functions)? Also paste the code for the initial function pointer definitions.
Thanks for quick reply. I was making dumb mistake calling RegisterCallbacks as Init Function in .dll file added to Extension. Its working as expected.

I do have another problem, I am trying to add Microsoft C++ REST SDK (Link: https://msdn.microsoft.com/en-us/library/jj969455.aspx) to my project in order to make REST API calls. But for some reason when I try to put some REST SDK library related code to my project I am getting Error "LoadLibraryA failed with error code 126". Please let me know if I am doing something wrong or if there is any other appropriate way to make REST API Calls from thread running as explained in you tutorial.

Thanks for the help and is much appreciated.
 
M

Mystborn

Guest
Good question. You'd have to do some testing on your part, but I'd think that it might. If it doesn't, you should look at creating a modeless dialog (just something I found after researching the subject for a bit). Here's a place that can get you started on the right track.
 
M

Mystborn

Guest
http://gmc.yoyogames.com/index.php?showtopic=390517
Why doesn't something like this exist for GMS 1.4?
I can't say for sure, since I don't know enough about GM8.1.
Some possible options:
  • GM8.1 was written in delphi so it may exploit some security flaws in that.
  • GM8.1 had some sort of compiler or interpreter shipped with every application that could allow you to dynamically execute code, so maybe it used some exploit in that.
  • Before Studio, there was a way to decompile applications. They may have used the information found in this process to create their dll.
  • Some combination of the above. (After carefully reading the source I believe it is this one.)
What I can say for sure is that they had information that is just not presently known in Gamemaker Studio. Specifically they knew the exact memory addresses of the compiler, object creation factories (Or at least I think that's what they are), and the address of something that would actually execute code.

We have none of the prior knowledge in GMS. One of the major reasons for this is to fix some of the major security problems previous versions were plagued with. Also, depending on how it's implemented, the memory addresses may change on each execution of the program, though I can't say for sure (I'll test it later), so even if you did manage to find some special addresses, you still wouldn't have a way to compile the code, and it might only work for that session.

Literally all we're given regarding the internal workings of GM are the memory addresses of certain functions related to ds_maps. Even worse, you can't use this to use these functions on maps created in GM, so the only use for them is for external libraries performing asynchronous operations.

It's almost impossible to replicate the functionality as well, which is really disappointing. At one point in time I had thought about making a gml interpreter in c++ or c#, but due to the limited information that can be sent to libraries, it just isn't feasible. So right now, this is the best that we've got.

On the brightside, at least the solution that I've provided works on multiple platforms if the source is compiled accordingly. The library that you linked to is written in MAsm, which only works on Windows. (Though I'm still grasping at straws with this "brightside")
 
R

Remixful

Guest
I see. That's unfortunate. Have you made any progress on the C# tutorial? That seems like my only hope.
 

kupo15

Member
Can multi threading be used to animate a loading screen while you load textures in the background and/or load textures in the background as you play the game? I assume if one is possible the other one is as well or both aren't possible
 
Hi. I'm somewhat uninformed as a programmer, so forgive me if you've already answered a similar question, or if this is a bit obvious.

I'm not sure if GMS 1.4 actually uses multiple processor cores itself, or just runs on one. If it just uses one - is there a way to have a machine thinking its doing networking, but actually is just speaking to itself?

The idea would be to have two (or however many cores the machine can support) instances of the game running, and communicating separate aspects like physics / AI / other intensive actions - which it then "collates" into one as if were just treating multiple players in a network.

As I said - I'm not even sure whether it's necessary, as GMS 1.4 may already run on more than one core, but if not - any thoughts on if this is doable? Thanks!
 
M

Mystborn

Guest
Can multi threading be used to animate a loading screen while you load textures in the background and/or load textures in the background as you play the game? I assume if one is possible the other one is as well or both aren't possible
You can in an extremely roundabout way. You'd have to create a buffer of the size of your texture, then load the image data into the buffer inside of an extension that would execute the operation on a separate thread, then use buffer_set_surface to get something drawable.

Hi. I'm somewhat uninformed as a programmer, so forgive me if you've already answered a similar question, or if this is a bit obvious.

I'm not sure if GMS 1.4 actually uses multiple processor cores itself, or just runs on one. If it just uses one - is there a way to have a machine thinking its doing networking, but actually is just speaking to itself?

The idea would be to have two (or however many cores the machine can support) instances of the game running, and communicating separate aspects like physics / AI / other intensive actions - which it then "collates" into one as if were just treating multiple players in a network.

As I said - I'm not even sure whether it's necessary, as GMS 1.4 may already run on more than one core, but if not - any thoughts on if this is doable? Thanks!
All versions of gamemaker use only one thread. I don't completely understand what you mean, but I'm pretty sure what you're describing would be relativley difficult even in an engine that ran on multiple cores. Given your opening statement, I'd personally put this idea on the back burner until you can answer the question yourself as it will involve a lot of low level operations and essentially running the game from an extension, at which point you'd be better off using something like Monogame (c# game library) or sdl/sfml (c++ graphics/game libraries). With those you can build your own architecture based on multiple cores. Of course after all that work, you'd be excluding users who have single core processors, so that's something to keep in mind.
 
S

Sam (Deleted User)

Guest
I've been wanting to know how to properly do this for a very long time! Bookmark'd. :D
 
M

Mystborn

Guest
Can multi threading be used to animate a loading screen while you load textures in the background and/or load textures in the background as you play the game? I assume if one is possible the other one is as well or both aren't possible
Since I figured this was a good example of how this type of functionality, I went ahead and created a simple extensions that allows you to do this! It's free and open source, so anyone can take a look at how it works.
 
R

ramon schepers

Guest
new here, sorry for reviving this, but i am wondering if we can somehow pass a function pointer to a gml script to the DLL to execute?
 

hippyman

Member
new here, sorry for reviving this, but i am wondering if we can somehow pass a function pointer to a gml script to the DLL to execute?
No that's the point of the async example. Basically you would have a function in the DLL that executes the async function. Then on the GM side you can run your script in the async event when the DLL triggers it.
 
S

Sam (Deleted User)

Guest
Hey! Anyone happen to know what ID to use for all the other async events? I can see 70 is the ID used for the social event, but that appears to be the only one Nocturne's article and this tutorial seems to cover. I'm looking to use the dialog event, more specifically, but it would be even more helpful just to know what they all are off-hand.
 
S

Sam (Deleted User)

Guest
Special thanks to @Nocturne and the YoYo Staff for helping me get this info; here are all async events available and their associated ID's for use in async extension creation:

GML:
"ev_web_image_load", 60
"ev_web_sound_load", 61
"ev_web_async", 62
"ev_dialog_async", 63
"ev_web_iap", 66
"ev_web_cloud", 67
"ev_web_networking", 68
"ev_web_steam", 69
"ev_social", 70
"ev_push_notification", 71
"ev_async_save_load", 72
"ev_audio_recording", 73
"ev_audio_playback", 74
"ev_system_event", 75
"ev_broadcast_message", 76
I recommend adding this to the OP for it to be easier to spot. It's very important for those who need something other than the social event in their extension.
 
Last edited by a moderator:
L

leikela

Guest
This is super helpful! I really appreciate having the ability to run code asynchronously. This was also my first time ever making a dll, and your explanation was very clear.

I'm still pretty new to GameMaker and dlls, so please forgive me if this is an obvious question, but I was wondering if it's possible to use this to read in data from an external program asynchronously into GameMaker . For instance, I have an .exe that prints one line to console every second. I want to pipe the output of each line into GameMaker as it comes so I can do something with that value. I'm not sure how to do that though. I usually work with Python and I can pipe into Python easy enough, but I'm not sure how to pipe into GameMaker. I know your extension is a key part of the answer. Do you have any insight?
 
S

Sam (Deleted User)

Guest
This is super helpful! I really appreciate having the ability to run code asynchronously. This was also my first time ever making a dll, and your explanation was very clear.

I'm still pretty new to GameMaker and dlls, so please forgive me if this is an obvious question, but I was wondering if it's possible to use this to read in data from an external program asynchronously into GameMaker . For instance, I have an .exe that prints one line to console every second. I want to pipe the output of each line into GameMaker as it comes so I can do something with that value. I'm not sure how to do that though. I usually work with Python and I can pipe into Python easy enough, but I'm not sure how to pipe into GameMaker. I know your extension is a key part of the answer. Do you have any insight?
This extension is a good starting point, it can get printed output after the program has quit asynchronously, but with some modifications, it can get that output returned in chunks instead:


The source code is included. More specifically, see the function process_execute() which process_execute_async() calls while creating a new thread. Most of the remaining code is pretty irrelevant to what you are trying to do.
 
Last edited by a moderator:
L

leikela

Guest
This extension is a good starting point, it can get printed output after the program has quit asynchronously, but with some modifications, it can get that output returned in chunks instead:
Thank you, I'll look into it!

Ah, this is your extension! I actually had found that a couple days ago, but I couldn't figure out how to use it right. Per your instructions on the extension page, I sent you a message at the time asking some questions. Is it better to put those in a forum instead?
 
S

Sam (Deleted User)

Guest
Yes, the forum i am more active on than email, so that would make things easier.
 

Edgamer63

Member
If this topic is not dead, this should be updated to vs2019, in my case, i've done a coding in vs2019, but it crashes (i've figured that the crash is in return_double, specifically in the part where it does have to add items to ds_map of async event of gml, using for example:
C++:
gml_ds_map_add_double(map, "type", type);
I have this code, but it crashes when i even implemented:
C++:
char* c_type = _strdup("type");
gml_ds_map_add_double(map, c_type, type);
So now i feel like:
What, wtf - Meme Official - YouTube

PD: It Crashed because the callbacks function don't have a name well wrote, it was like: "RegistersCallBacks" not "RegisterCallbacks", so that's why this wasn't working.
 
Last edited:
Top