Ricardo
Member
I’d like to start a discussion about asynchronous loading - especially when it comes to sprites. A lot has changed since the first iterations of GameMaker when all the project resources were loaded and thrown at RAM (and VRAM), but there are still a few key issues that are a limiting factor considering how GMS is quickly moving forward to become a more professional tool.
Preloading everything at startup
This is the most obvious problem which is clearly a heritage of how GameMaker worked in the past. These days a lot has changed, though: sprites aren’t preload to VRAM anymore, sounds can be streamed, sprite_add/sprite_delete and a bunch of other sprite creating functions such as sprite_create_from_surface/sprite_add_from_surface allow developers to take care of their resources without depending on the build-in texture-page system (which loads everything to RAM at startup). However, the implementation of the sprite loading and creation functions is lacking in a fundamental aspect: they are all blocking functions (they don’t run in a separate thread).
In my studio, we heavily feel the weight of this limitation, and considering how the GMS showcase list has been growing lately with amazing games, we believe other developers also feel impacted by this.
Workarounds
At a first glance, it seems like there are workarounds, such as using RAW textures (surfaces dumps) that can be asynchronously loaded using the buffer functions. However, sprite_create_from_surface/sprite_add_from_surface are terribly slow and they block the main thread - defeating the purpose of the file async loading.
It is technically possible to use the surfaces directly without creating sprites by using buffer_get_surface/buffer_set_surface, but those functions are also slow and thread blocking, and we lost the benefits of having the flexibility of a proper sprite. Besides, storing textures in RAW format takes a lot of disk space.
Oversight?
First of all, the fact that all those functions exist indicates to us that we are not the first ones wanting to do manual resource management, but the way the functions were implemented feel like a terrible waste of potential to us because these days games are getting bigger, SSDs are getting faster and accessible and in short you usually don’t want to load/unload assets synchronously blocking the game thread unless there's a really good reason for it. The benefits of being able to manage sprites asynchronously are greater than a nice animated loading screen: they can fundamentally change how big games and open worlds are created. Bigger and more ambitious games could be born.
Hacks?
There’s certainly a reason why those functions are blocking, but we’ve looked into an old multithread hack that is available here at the forum and… surprise! sprite_add seems to work just fine in a separate thread without causing any apparent glitch. In fact, sprite_add is already asynchronous in HTML5, and it's a mystery why it is blocking in all the other exports.
Final thoughts
We’d love to hear how other professionals and studios feel about this limitation. Splashing a loading screen and hiding the mouse (to avoid visible stutter) is in fact doable, but considering how many functions are available to manage resources, it just feels like a waste of potential that there isn't a proper asynchronous workaround to manage sprites. I’d really love to hear @rwkay thoughts on this matter.
Preloading everything at startup
This is the most obvious problem which is clearly a heritage of how GameMaker worked in the past. These days a lot has changed, though: sprites aren’t preload to VRAM anymore, sounds can be streamed, sprite_add/sprite_delete and a bunch of other sprite creating functions such as sprite_create_from_surface/sprite_add_from_surface allow developers to take care of their resources without depending on the build-in texture-page system (which loads everything to RAM at startup). However, the implementation of the sprite loading and creation functions is lacking in a fundamental aspect: they are all blocking functions (they don’t run in a separate thread).
In my studio, we heavily feel the weight of this limitation, and considering how the GMS showcase list has been growing lately with amazing games, we believe other developers also feel impacted by this.
Workarounds
At a first glance, it seems like there are workarounds, such as using RAW textures (surfaces dumps) that can be asynchronously loaded using the buffer functions. However, sprite_create_from_surface/sprite_add_from_surface are terribly slow and they block the main thread - defeating the purpose of the file async loading.
It is technically possible to use the surfaces directly without creating sprites by using buffer_get_surface/buffer_set_surface, but those functions are also slow and thread blocking, and we lost the benefits of having the flexibility of a proper sprite. Besides, storing textures in RAW format takes a lot of disk space.
Oversight?
First of all, the fact that all those functions exist indicates to us that we are not the first ones wanting to do manual resource management, but the way the functions were implemented feel like a terrible waste of potential to us because these days games are getting bigger, SSDs are getting faster and accessible and in short you usually don’t want to load/unload assets synchronously blocking the game thread unless there's a really good reason for it. The benefits of being able to manage sprites asynchronously are greater than a nice animated loading screen: they can fundamentally change how big games and open worlds are created. Bigger and more ambitious games could be born.
Hacks?
There’s certainly a reason why those functions are blocking, but we’ve looked into an old multithread hack that is available here at the forum and… surprise! sprite_add seems to work just fine in a separate thread without causing any apparent glitch. In fact, sprite_add is already asynchronous in HTML5, and it's a mystery why it is blocking in all the other exports.
Final thoughts
We’d love to hear how other professionals and studios feel about this limitation. Splashing a loading screen and hiding the mouse (to avoid visible stutter) is in fact doable, but considering how many functions are available to manage resources, it just feels like a waste of potential that there isn't a proper asynchronous workaround to manage sprites. I’d really love to hear @rwkay thoughts on this matter.