GMS 2.3+ Sprites asynchronous loading

Ricardo

Member
I’d like to start a discussion about asynchronous loading - especially when it comes to sprites. A lot has changed since the first iterations of GameMaker when all the project resources were loaded and thrown at RAM (and VRAM), but there are still a few key issues that are a limiting factor considering how GMS is quickly moving forward to become a more professional tool.

Preloading everything at startup

This is the most obvious problem which is clearly a heritage of how GameMaker worked in the past. These days a lot has changed, though: sprites aren’t preload to VRAM anymore, sounds can be streamed, sprite_add/sprite_delete and a bunch of other sprite creating functions such as sprite_create_from_surface/sprite_add_from_surface allow developers to take care of their resources without depending on the build-in texture-page system (which loads everything to RAM at startup). However, the implementation of the sprite loading and creation functions is lacking in a fundamental aspect: they are all blocking functions (they don’t run in a separate thread).

In my studio, we heavily feel the weight of this limitation, and considering how the GMS showcase list has been growing lately with amazing games, we believe other developers also feel impacted by this.

Workarounds

At a first glance, it seems like there are workarounds, such as using RAW textures (surfaces dumps) that can be asynchronously loaded using the buffer functions. However, sprite_create_from_surface/sprite_add_from_surface are terribly slow and they block the main thread - defeating the purpose of the file async loading.

It is technically possible to use the surfaces directly without creating sprites by using buffer_get_surface/buffer_set_surface, but those functions are also slow and thread blocking, and we lost the benefits of having the flexibility of a proper sprite. Besides, storing textures in RAW format takes a lot of disk space.

Oversight?

First of all, the fact that all those functions exist indicates to us that we are not the first ones wanting to do manual resource management, but the way the functions were implemented feel like a terrible waste of potential to us because these days games are getting bigger, SSDs are getting faster and accessible and in short you usually don’t want to load/unload assets synchronously blocking the game thread unless there's a really good reason for it. The benefits of being able to manage sprites asynchronously are greater than a nice animated loading screen: they can fundamentally change how big games and open worlds are created. Bigger and more ambitious games could be born.

Hacks?

There’s certainly a reason why those functions are blocking, but we’ve looked into an old multithread hack that is available here at the forum and… surprise! sprite_add seems to work just fine in a separate thread without causing any apparent glitch. In fact, sprite_add is already asynchronous in HTML5, and it's a mystery why it is blocking in all the other exports.

Final thoughts

We’d love to hear how other professionals and studios feel about this limitation. Splashing a loading screen and hiding the mouse (to avoid visible stutter) is in fact doable, but considering how many functions are available to manage resources, it just feels like a waste of potential that there isn't a proper asynchronous workaround to manage sprites. I’d really love to hear @rwkay thoughts on this matter.
 

Mert

Member
  • As a video game developer, you must always assume that the players have the average-performance devices. Performance is a big issue for me since HTML5 / Mobiles are my main exports.
  • sprite_add creates a big whole texture pack for a single sprite, which is a big waste of texture swaps.
  • You should manually create a texture pack and load is as surface. Then draw the required sprites by drawing parts of this surface. I repeat : You draw the parts of this surface, not adding them as sprites!
  • Yoyogames must implement a good offloading system.
 

COWCAT

Member
I would like async sprite loading too.
My game is high resolution (1080p assets but I'll also try to aim at 4K on PS5/PC at one point)
Aside from character animations and small pictures, all my assets are loaded through sprite_add(). It's just easier to manage and doesn't result in a huge assets file (data.win), which is also an issue for patches (especially on consoles, the patch minimum size becomes at least the size of data.win!)
Being able to preload some pictures asynchronously would be really nice.

For example, players enter a room and then it starts to preload the pictures used for interactions. (close up views)
This would also be useful to design loading screens with an animation. Some big assets will always take more than one frame to load with sprite_add().
It needs to remain an option, though. Sometimes you NEED blocking loadings to make sure players don't see a missing picture.

  • As a video game developer, you must always assume that the players have the average-performance devices. Performance is a big issue for me since HTML5 / Mobiles are my main exports.
  • sprite_add creates a big whole texture pack for a single sprite, which is a big waste of texture swaps.
  • You should manually create a texture pack and load is as surface. Then draw the required sprites by drawing parts of this surface. I repeat : You draw the parts of this surface, not adding them as sprites!
  • Yoyogames must implement a good offloading system.
This is another issue that needs to be addressed.
I would REALLY love to be able to create textures on the fly, at runtime. Because managing them with surfaces is a huge hassle. Not all assets are the same size, some are animations etc...
Currently I don't do that and it results in a huge waste of memory/texture swaps, but I really don't have time to optimize further with that method. (My game still runs fine on any decent computer, it's just a shame)

So yes, please let us:
- Load external pictures (like sprite_add() ) synchronously or asynchronously
- Build texture pages with those at runtime. Grouping them automatically with an identifier we would give, for example the name of the room.
- Make them work the same way as the texture pages for sprites
 
Last edited:

gnysek

Member
I would like async texture loading.

How that would work? GM IDE would allow to prepare such pack, containing PNG texture file + json data file about sprites (their positions, origins, etc.). It would then could be loaded by: tex = texture_add("texture.yyt"); and then single sprite id could be accessed by var my_sprite = texture_get_sprite(tex, "some_sprite"), where "some_sprite" is a name given during pack preparation in IDE tool.
That would be optimal.
 

COWCAT

Member
I would like async texture loading.

How that would work? GM IDE would allow to prepare such pack, containing PNG texture file + json data file about sprites (their positions, origins, etc.). It would then could be loaded by: tex = texture_add("texture.yyt"); and then single sprite id could be accessed by var my_sprite = texture_get_sprite(tex, "some_sprite"), where "some_sprite" is a name given during pack preparation in IDE tool.
That would be optimal.
Sounds good but I would really prefer to make textures at runtime. To decide what should be on it when I need it, not prepared in advance through the IDE.
So basically, we would list the sprites we want and the texture would be generated automatically accordingly.
The reason? Well, according to player choices, sprites actually used can vary. For example I display random animated characters in one area, and considering they're randomized I have no way to know which to put on the texture in advance.

Of course I can do without, but it seems the ideal to me.
 

Ricardo

Member
Sounds good but I would really prefer to make textures at runtime. To decide what should be on it when I need it, not prepared in advance through the IDE.
So basically, we would list the sprites we want and the texture would be generated automatically accordingly.
The reason? Well, according to player choices, sprites actually used can vary. For example I display random animated characters in one area, and considering they're randomized I have no way to know which to put on the texture in advance.

Of course I can do without, but it seems the ideal to me.
I don't believe a "create texture pages on the fly" system will ever happen. This has been discussed in the past and I remember Mike explaining why this can't be done (the math to generate them is a bit complex and too slow for runtime).
Besides, sprite_add isn't the villain it used to be. Textures swaps aren't such a big deal in DX11 (and latest platforms).

At least for me, the big deal is that sprite_add is synchronous. That improvement alone would be a game changer for my projects.
They could add an extra optional parameter to the function to avoid breaking compatibility with old code, like this:
Code:
sprite_add(fname, imgnumb, removeback, smooth, xorig, yorig, forceasync);
The async image load event is already there... This improvement shouldn't be a huge task for YoYo.
 

Ricardo

Member
So mainly that would mean porting HTML5 feature to other runners.
Yeah, basically. In HTML5 sprite_add is always async you like it or not so I don't believe that having and async option for the rest of the runners is be a big deal. Is not a unreasonable request for sure. I believe that the lack of this feature is more of an oversight than a real limitation.

If you're interested in async loading, please fill a feature request for it. The more people do it, the more chances we have of getting it eventually. I have the impression YoYo haven't bothered with this so far because not enough people has complained about it.
 

kroart

Member
I wrote an extension for Android that loads sprite asynchronously. It works well but has some cons:
- It works only on Android
- It uses more memory than regular sprites in GMS, I think so. But I haven't checked it :) It should be equivalent to sprite_add() function in memory management.

If anyone is interested I can share my code.
 
Last edited:

Ricardo

Member
I wrote an extension for Android that loads sprite asynchronously. It works well but has some cons:
- It works only on Android
- It uses more memory than regular sprites in GMS, I think so. But I haven't checked it :) It should be equivalent to sprite_add() function in memory management.

If anyone is interested I can share my code.
I’d love to check it out.
 

Ricardo

Member
Thank you very much for taking the time to document and upload this. It looks like what the extension is doing is similar to what I commented in the "Workarounds" section of the original post, but I imagine the advantage is that it decodes other images format? I'll give it a try soon to figure it out.
 

kroart

Member
Thank you very much for taking the time to document and upload this. It looks like what the extension is doing is similar to what I commented in the "Workarounds" section of the original post, but I imagine the advantage is that it decodes other images format? I'll give it a try soon to figure it out.
Yes, it works as you described in "Workarounds", instead that it loads images not from RAW textures but from image files. It uses BitmapFactory.decodeFile() so it support different formats (jpg and png for sure). I use it not for high-resolution images and it works noticeably smoother than just sprite_add() function in GML for me. How it will work on big images it's an open question...

And btw there is one improvement that can be done for this extension. It fills buffer in the main thread, but it can be done in the background thread so it will give some improvement (again it should be tested for how much). I thought about this just yesterday but, as I said, for my needs (loading quite small images) it works just fine. You can check it in my app: https://play.google.com/store/apps/details?id=com.lazybonesgames.ornamentaltheapp If you open it - you will see an image gallery, these images are downloading from the network and loaded using this extension. And when you scroll the gallery - images are dynamically loading and unloading during scrolling so only these images that are on the screen (and some buffer area outside the screen) are held in memory.

And yes, it's an app written in GameMaker) At first, it was a game but then transferred to the app, so maybe this is the first app made in GameMaker 😅
 
Last edited:

Ricardo

Member
Yes, it works as you described in "Workarounds", instead that it loads images not from RAW textures but from image files. It uses BitmapFactory.decodeFile() so it support different formats (jpg and png for sure). I use it not for high-resolution images and it works noticeably smoother than just sprite_add() function in GML for me. How it will work on big images it's an open question...

And btw there is one improvement that can be done for this extension. It fills buffer in the main thread, but it can be done in the background thread so it will give some improvement (again it should be tested for how much). I thought about this just yesterday but, as I said, for my needs (loading quite small images) it works just fine. You can check it in my app: https://play.google.com/store/apps/details?id=com.lazybonesgames.ornamentaltheapp If you open it - you will see an image gallery, these images are downloading from the network and loaded using this extension.

And yes, it's an app written in GameMaker) At first, it was a game but then transferred to the app, so maybe this is the first app made in GameMaker 😅
Thanks for your detailed input! There's something I don't understand, though: you are saying the images in your app are downloaded from the network, so sprite_add should work asynchronous already. Does it still causes a noticeable performance hit on Android?
 

kroart

Member
Thanks for your detailed input! There's something I don't understand, though: you are saying the images in your app are downloaded from the network, so sprite_add should work asynchronous already. Does it still causes a noticeable performance hit on Android?
Images are downloaded only for the first time and then stored locally on the device. So the next time, it will load from the device directly.

Yes, for my case it gave me a noticeable performance. At first I tried to load sprites using sprite_add() and during scrolling it gave me lags in the UI (obviously). When I switched to this solution - lags disappeared.
Again, as I wrote in a previous message, it can be improved by switching buffer filling into background thread but I didn't investigated this because the current solution satisfied me.
 
Top