Windows 1GB games? Max game file sizes? I haz Production halp pleez

I

INFO XMiGuPLAY LLC

Guest
Hello all. I have been publishing for sometime and in Summer of 2017 I transitioned from Android Market to Steam PC games. In publishing my first two PC titles, I noticed that ...

Once my application size gets to about 1GB, the games I develop become pretty unstable.

My latest published at 897MB for the EXE file size ( not including audio ) and only my customers running legit graphics cards can play (at least 1GB VRAM). Even the Intel HD graphics cards wont run the games at this size. So, my 1 on 1 fighting game - which includes 14 playable characters with individual intro/ending splash screens & designed for a 1080p output with a total of 46 Sprite sheets at 4096x4096 is suffering due to this limitation. I need to double my sprites to get some nice frame rates.

My only solution during production was to do a massive cut in animation frames on all characters just to reduce game size. My Idle stance originally designed for about 40 frames is cut to 8 and so on. I even reduced many sprites and up scaled them with interpolation to help optimize the sprite sheet's utilization.


Do I need a different engine or does anyone know how to make some big ol games in GM?
I am all ears on any insight and thank you!

version - GameMaker Studio Professional v1.4.1763
HW: i5-4440 CPU 3.10 GHz, 16GB RAM, GPU NVIDIA GTX 1070 8GBVRAM



-M
 

sylvain_l

Member
not sure I understand exactly your explanation, so my 2 cents:

46 Sprite sheets at 4096x4096
4k start to be quite big, not sure every GPU handle that (for the older&cheapest)

and if I'm not wrong, a 4096x4096 ARGB is around 67MB uncompressed in VRAM. *46 that's about 3GB. so not that surprising if your game struggle on GPU with less VRAM than that.

edit:
also be aware that windows GM games being 32bit exe they are surely limited to the 2GB-4GB RAM threshold.
(should go away once we get 64bit games also for windows too)
 
Last edited:
I

INFO XMiGuPLAY LLC

Guest
not sure I understand exactly your explanation, so my 2 cents:


4k start to be quite big, not sure every GPU handle that (for the older&cheapest)

and if I'm not wrong, a 4096x4096 ARGB is around 67MB uncompressed in VRAM. *46 that's about 3GB. so not that surprising if your game struggle on GPU with less VRAM than that.

edit:
also be aware that windows GM games being 32bit exe they are surely limited to the 2GB-4GB RAM threshold.
(should go away once we get 64bit games also for windows too)
Thanks for your response... I think my issue is this, is there a way to prevent GM from loading every sprite sheet into Video Ram at once? Once my game's total size (the actual compiled exe file) is about 1 GB, the game tanks and crashes. I need to be able to control what sprite sheets are called in between levels and such.

My latest title uses 46 4096x4096 sprite sheets which yields a 897MB exe file. The game runs fine but when the exe was over 970MB-ish, all Hell broke loose.

-M
 
M

MishMash

Guest
As sylvain has said, the memory requirements for that many sprites is substantial. You will need to employ some form of manual texture management once you start dealing with such large quantities. Rarely do games load all their resources in one go. Given you specified its a 1v1 fighting game, then its reasonable to assume that you only need to have at most 2 fighters animations loaded at any given time, along with common utilities and the stage background.

These requirements will likely be true for whatever engine you use, however some other engines may provide better automatic support.

Now, there are two pools of memory you need to concern yourself with.
- Application memory, stored in RAM
- Texture memory stored on the GPU in VRAM

Generally speaking, there will be a copy of each sprite kept in both CPU memory and GPU memory. When you need to render a texture, the texture MUST be transferred to GPU memory for rendering. By default, GM will load ALL textures into VRAM so that they are immediately available, however you have a few options to allow you to control VRAM usage.

Managing VRAM
The functions:
sprite_flush and sprite_flush_multi allow you to unload graphics from VRAM. The multi variant allows you to pass in an array of sprites (which you could pre-prepare, i.e. for a particular level or character) and it will unload these sprites from memory.
You can then use sprite_prefetch and sprite_prefetch_multi to reload them into VRAM ready for rendering.

In order to get the most out of these functions, you will want to disable GMs automatic loading of textures into memory until they are used. If you go into global game settings, you can check the "Create textures on demand" flag which will cause textures to only get loaded onto the GPU when they are needed:


Also note that rendering a sprite will also trigger its loading onto the GPU if it is not already loaded, however, ideally you want to use sprite_prefetch as there can be a small stutter when sending a graphic from the CPU to the GPU.

Regarding RAM management
You do have a bit more flexibility with RAM with regard to space, however, if you cannot fit everything you need in, then you may need to consider dynamic creation and destruction of sprite resources using sprite_add and sprite_delete. Obviously, this can become difficult when you have large quantities of animated sprites, and may require manual texture page/sprite sheet creation and management.



A general note on memory management
Another really important thing to understand is that while RAM may be of a certain size, e.g. 2GB, data for things like textures MUST be stored contiguously. That is, all of the memory must appear in one continuous back to back block. This means that based on how our memory is structured, both with regard to other applications using memory, and the size of data we are trying to allocate, while there may be free memory, there may not necessarily be a block of contiguous memory as big as we need. This problem is known as memory fragmentation.
The larger the texture, the more of an issue this becomes. With 4k textures, we are looking at a base of 64MB of contiguous memory, this slot can be increasingly harder to find, as even without memory fragmentation, assuming perfectly clear memory, we could only fit 16 pages in. Therefore, there is an advantage here in using 2k texture pages as they will not suffer the effects of fragmented memory as much as the larger ones, given they would only take 16MB each.

This diagram demonstrates how RAM fragmentation can occur:


As far as keeping memory clear and unfragmented, this is rather hard to do in GM. AAA games will likely allocate regions of memory and keep them protected for specific purposes. I.e. when loading in new textures, they won't just allocate and deallocate memory on the fly, but rather, replace existing memory with the new data. This is something that is often not possible in consumer game engines, because it requires manual management of the internals.

The best you can do is just be aware that the problem exists, and be smart about the sizes of data that you have.
 
I

INFO XMiGuPLAY LLC

Guest
As sylvain has said, the memory requirements for that many sprites is substantial. You will need to employ some form of manual texture management once you start dealing with such large quantities. Rarely do games load all their resources in one go. Given you specified its a 1v1 fighting game, then its reasonable to assume that you only need to have at most 2 fighters animations loaded at any given time, along with common utilities and the stage background.

These requirements will likely be true for whatever engine you use, however some other engines may provide better automatic support.

Now, there are two pools of memory you need to concern yourself with.
- Application memory, stored in RAM
- Texture memory stored on the GPU in VRAM

Generally speaking, there will be a copy of each sprite kept in both CPU memory and GPU memory. When you need to render a texture, the texture MUST be transferred to GPU memory for rendering. By default, GM will load ALL textures into VRAM so that they are immediately available, however you have a few options to allow you to control VRAM usage.

Managing VRAM
The functions:
sprite_flush and sprite_flush_multi allow you to unload graphics from VRAM. The multi variant allows you to pass in an array of sprites (which you could pre-prepare, i.e. for a particular level or character) and it will unload these sprites from memory.
You can then use sprite_prefetch and sprite_prefetch_multi to reload them into VRAM ready for rendering.

In order to get the most out of these functions, you will want to disable GMs automatic loading of textures into memory until they are used. If you go into global game settings, you can check the "Create textures on demand" flag which will cause textures to only get loaded onto the GPU when they are needed:


Also note that rendering a sprite will also trigger its loading onto the GPU if it is not already loaded, however, ideally you want to use sprite_prefetch as there can be a small stutter when sending a graphic from the CPU to the GPU.

Regarding RAM management
You do have a bit more flexibility with RAM with regard to space, however, if you cannot fit everything you need in, then you may need to consider dynamic creation and destruction of sprite resources using sprite_add and sprite_delete. Obviously, this can become difficult when you have large quantities of animated sprites, and may require manual texture page/sprite sheet creation and management.



A general note on memory management
Another really important thing to understand is that while RAM may be of a certain size, e.g. 2GB, data for things like textures MUST be stored contiguously. That is, all of the memory must appear in one continuous back to back block. This means that based on how our memory is structured, both with regard to other applications using memory, and the size of data we are trying to allocate, while there may be free memory, there may not necessarily be a block of contiguous memory as big as we need. This problem is known as memory fragmentation.
The larger the texture, the more of an issue this becomes. With 4k textures, we are looking at a base of 64MB of contiguous memory, this slot can be increasingly harder to find, as even without memory fragmentation, assuming perfectly clear memory, we could only fit 16 pages in. Therefore, there is an advantage here in using 2k texture pages as they will not suffer the effects of fragmented memory as much as the larger ones, given they would only take 16MB each.

This diagram demonstrates how RAM fragmentation can occur:


As far as keeping memory clear and unfragmented, this is rather hard to do in GM. AAA games will likely allocate regions of memory and keep them protected for specific purposes. I.e. when loading in new textures, they won't just allocate and deallocate memory on the fly, but rather, replace existing memory with the new data. This is something that is often not possible in consumer game engines, because it requires manual management of the internals.

The best you can do is just be aware that the problem exists, and be smart about the sizes of data that you have.
Thank you for the spectacular break down and I have enough info here to get some traction in the right direction! As for your assumption, yes, the characters currently run on 2 sprite pages + UI and Fx on one and BG on another. With your recommendations, I may explore breaking into 2x2k on the sprite sheets. My ultimate goal is to dedicate 4 4k sprite sheets per character to get some clean framerates. I can not thank you enouh for This!

-M
 

sylvain_l

Member
@INFO XMiGuPLAY LLC
rereading the thing, I realize you are using sprite sheets for what seems "HD quality" animations. Wouldn't it be much better to use skeletal animation in your case? ( that should drastically reduce your VRAM requirement)

IDK what's your actual workflow to generate your animation and if you could easily adapt it and if your content would fit. And chance you are already advanced in your actual project so doing the switch would be too much of a hassle...
 
I

INFO XMiGuPLAY LLC

Guest
@INFO XMiGuPLAY LLC
rereading the thing, I realize you are using sprite sheets for what seems "HD quality" animations. Wouldn't it be much better to use skeletal animation in your case? ( that should drastically reduce your VRAM requirement)

IDK what's your actual workflow to generate your animation and if you could easily adapt it and if your content would fit. And chance you are already advanced in your actual project so doing the switch would be too much of a hassle...
Regarding work flow, I am basically rendering 3D models into 2D sprites. I looked at doing the 2D style animation but it was not fitting into what I wanted my end product to look like.

I did some brief testing last night and the game's VRAM requirements dropped from 54% utilization to about 15-23%. I need to play with the pre-loading images into ram functions and figure out how I am going to recode my scripts transition from one room to the next.

Even though my focus is on my new title that I am working on from scratch, I am looking into doing a revision on this game and patching it. I would love to lower the system requirements and better optimize it.

Thank you both for your time and help. :)


-M
 

kupo15

Member
A general note on memory management
Another really important thing to understand is that while RAM may be of a certain size, e.g. 2GB, data for things like textures MUST be stored contiguously. That is, all of the memory must appear in one continuous back to back block. This means that based on how our memory is structured, both with regard to other applications using memory, and the size of data we are trying to allocate, while there may be free memory, there may not necessarily be a block of contiguous memory as big as we need. This problem is known as memory fragmentation.
The larger the texture, the more of an issue this becomes. With 4k textures, we are looking at a base of 64MB of contiguous memory, this slot can be increasingly harder to find, as even without memory fragmentation, assuming perfectly clear memory, we could only fit 16 pages in. Therefore, there is an advantage here in using 2k texture pages as they will not suffer the effects of fragmented memory as much as the larger ones, given they would only take 16MB each.

This diagram demonstrates how RAM fragmentation can occur:


As far as keeping memory clear and unfragmented, this is rather hard to do in GM. AAA games will likely allocate regions of memory and keep them protected for specific purposes. I.e. when loading in new textures, they won't just allocate and deallocate memory on the fly, but rather, replace existing memory with the new data. This is something that is often not possible in consumer game engines, because it requires manual management of the internals.

The best you can do is just be aware that the problem exists, and be smart about the sizes of data that you have.
Oh no not another thing to keep in mind! So you are saying that hypothetically if you run your game and there is a section which barely is under the memory threshold but is running fine, then throughout the course of playing the game and all the loading and unloading that occurs (objects getting destroyed as well as their variable memory, surfaces etc...) that you could get to a point where the memory is so fragmented that loading that one memory intensive scene that previously was barely fitting won't fit anymore and the game will run out of memory?
 
M

MishMash

Guest
Oh no not another thing to keep in mind! So you are saying that hypothetically if you run your game and there is a section which barely is under the memory threshold but is running fine, then throughout the course of playing the game and all the loading and unloading that occurs (objects getting destroyed as well as their variable memory, surfaces etc...) that you could get to a point where the memory is so fragmented that loading that one memory intensive scene that previously was barely fitting won't fit anymore and the game will run out of memory?
Yes, this can happen, but in practise, it is reasonable to avoid. Most blocks of allocated memory shouldn't really be that large anyway. If you are working in the order of MB (i.e. sub 16MB or so) you will almost certainly have nothing to worry about. You can also pretty well get around this by just moving where you manage your memory. So for example, if you unload assets for the current level right before loading for the new one, it is very likely that you will simply replace the block that was just deallocated.

So long as you unload stuff, then fragmentation shouldn't really be an issue, given that in most cases, things that are allocated near each other will likely also be de-allocated near each other. I.e. things that you load at the start of the game will likely occupy a block of memory near the start. As you load levels, those will likely load into the middle. Then, small allocations like those for small structures, instances etc; will often be so small that they just fit wherever. They may also not necessarily need an entire contiguous block, as lots of structures can be distributed, that is, they use pointers to their sub-components.

Practical techniques that engines will use to have explicit control over their memory are often in this sort of line:
- Pre-allocate heaps of memory to store your data in. In C++ terms, you could just create a large array to contain any data you needed, then
- Use memory pools: This one used to be really common and still is. Rather than ever re-allocating and de-allocating memory, you simply allocate a fixed number of slots, which act as the limit for a particular resource. I.e. for a specific instance, you know that you may never have more than 100 enemies of a given type at once, so you pre-create 100, and mark most as inactive. Then, you selectively enable/disable them by keeping a list of "free" IDs.
For things like textures, this can be the case that you have a generic "texture" slot, and rather than deallocating and re-allocating textures, you just load new data into the previous slot and re-purpose it. If you do run out of items in the pool, you can do a pool resize, which would then allocate another block of memory for a number of instances.
Again in C++ terms, there are other benefits to doing this as well, such as memory locality, meaning if you are likely updating enemies one after another, having the instances near each other in memory is super beneficial for cache performance.

- As mentioned before, time your allocations so if you are just about to unload a 4096*4096, the very next thing you do should be to load an equally sized texture, because you will know for certain that there will be a free memory slot for it.

Now, you don't really need to worry about this in practise, but its good to start getting a conceptual understanding of how memory works, which is why I am going on about it. In your case, I would wait to see if it becomes an issue first, because it isn't too bad to resolve after the fact. So long as you just selectively load assets, you should be fine in most cases.

While windows memory management is a bit of a black box, I think it's reasonable to assume that it is fully aware of this problem and tries to minimise it as much as possible. That is, it would keep track of free blocks within memory and prioritise packing small allocations into the smallest space possible first, before using the big chunks.
 

kupo15

Member
Awesome, great info @MishMash . Thanks! Yeah, I'm not going to worry about this right now either until/if it becomes a problem. I guess the only way to know if its a problem would be if you know you don't have any memory leaks and the game randomly runs out of memory.
 

Yal

šŸ§ *penguin noises*
GMC Elder
- Use memory pools: This one used to be really common and still is. Rather than ever re-allocating and de-allocating memory, you simply allocate a fixed number of slots, which act as the limit for a particular resource. I.e. for a specific instance, you know that you may never have more than 100 enemies of a given type at once, so you pre-create 100, and mark most as inactive. Then, you selectively enable/disable them by keeping a list of "free" IDs.
All of these points are really good! Just to add a thing here that you sort of skipped over: make all slots for the same type of resource the same size, based on the largest amount of data you'll need to fit in. Allocate a whole slot for each thing even if you aren't gonna use all of it. This makes fragmentation issues practically nonexistent (at the expense of wasted memory) and operations like clearing all the memory or accessing a certain slot gets easier and less error-prone since the sizes are uniform. (And if you waste 90% of the data a lot because some resources are 10% of the size of the biggest, you might wanna rethink the approach to use multiple different pools for small and big things)
 
Top