kupo15
Member
GM Version: Studio 2
Target Platform: Windows
Download: (N/A) code examples below
Links:
Relevant GMC discussion:
https://forum.yoyogames.com/index.php?threads/gms-2-memory-management-inquiry-suggestion.12365/
https://forum.yoyogames.com/index.php?threads/memory-use.36369/
https://forum.yoyogames.com/index.p...ment-101-tricks-and-clarity-sprite_add.35850/
TexturePacker Website
Summary:
This is a tutorial about creating and decoding JSON data structures. This particular example uses texture pages created with a commercial software package called TexturePacker. But the techniques are applicable to other JSON data applications.
In this tutorial I will be demonstrating and showing you how to integrate the program TexturePacker by CodeandWeb into your graphics workflow. Doing so will allow you to take control over your texture pages to optimize texture memory instead of having GM do it for you. This provides you with the freedom to proper memory management natively without dlls or extensions which results in allowing you to create bigger games and use more texture memory at once than you otherwise would be able to.
Who is this system designed for?
This method is not necessary for everyone and every project. Its useful mainly for those who have really large projects and large resolution games where memory tends to be a source of concern. GM does a fine job on its own for smaller games and unless you desire this amount of control and extra work, you are better off using GM's built in IDE capabilities
Brief Introduction to memory
(skip to the tutorial if you know about this)
If anything in the spoiler is inaccurate, please inform me so I can modify it!
The Goal
The goal is to manually perform the same texture packing method GM does with sprites inside the IDE and apply that to external loading via sprite_add. We do this by:
1. Creating our own texture pages in TexturePacker
2. Add that texture page into the IDE
3. Draw_sprite_general only the sprites in that page we want to display in our game
This is what GM does behind the scenes. You think draw_sprite(spr_walk,image_index,x,y) simply displays an isolated sprite strip? Think again. When you ask to draw spr_walk, subimage0, GM finds which messy looking texture page that subimage is on at its location and draw_part() from that texture page only where that frame is located. Draw_sprite secretly is draw_part under the hood.
What is TexturePacker?
Here's a basic overview of the tool and the concept behind what we are going to do. The next section shows how to integrate Texturepacker into a workflow that GM can use!
TexturePacker is an external program dedicated to packing and creating texture pages, just like GM does automatically except better. There is a free trial but its relatively inexpensive to purchase a full license. Its better for these reasons:
1. It gives you more control over the size of texture pages you want created
2. You can easily scale the tex page itself without needing to individually scale every sprite and even has HD scaling
3. Allows for more packing options such as polygon fitting and rotation!
4. Can preview your pages before compiling to see how they fit and how much memory will be used allowing for a greater deal of control and memory management
Does it really pack things better? Check out this simple comparison
The left was generated by GM with a max Tex Size of 4096x4096. Even on its simplest settings, Texture packer packed it in a smaller texture size than GM was able to do resulting in halving the memory usage! If external images weren't placed onto a power of 2 texture page, you could have texturepacker output a texture page size tailored exactly to what is needed for even more memory savings. If only!
However, Texturepack has an additional memory saving feature GM doesn't have that you can take advantage of:
Allow rotation enables individual sprites to be rotated 90 degrees which allows Texture pages to be packed even tighter. GM doesn't do this which results in wasted space by forcing all sprites to be packed in their original rotation instead potentially rotating them to fit optimally.
Guide
this section is only going to show how to extract the data from the files. How to apply it will be in the next section.
Outputting from TexturePacker
When you export from TexturePacker, it spits out two files: A texture page(s) and a data file associated with that texture page. TexturePacker has two file formats for export that is useful for GM
Json Array and Json Hash. There is only a small difference in the formats, the array decodes into a ds_list where all the sprites can easily be looped through and the hash decodes into an all ds_map json format. I've found that the Jason Array is the best method to work with simply because the list makes it easy to loop through but you might find the Hash could work better for what you need.
The first step is to take the exported files created from TexturePacker and through them in the included files. You can simply sprite_add the texture pages from the included files folder to load them into memory. All the magic now happens with decoding that json file and creating a drawing system to display the correct sprites when you want to
Json decoding
Credits
Before I begin I want to recognize the members that helped me out with a previous thread of mine dealing with JSON files here https://forum.yoyogames.com/index.php?threads/json-decode-help-solved.35997/
Also a huge shoutout to @chamaeleon @Tsa05 because their code snippets in that thread together gave me the information needed to put all the pieces together to create this system I'm going to be talking about in this tutorial.
When decoding the json, you can't simply json_decode that file as it will not work. Instead you have to create a string and line by line, read from the json file and add it to the string. Then decode that entire string
resultMap now holds all the json information as nested maps/lists depending on the type of json you are using. The first parsing script is for the Json Array Format.
Each Json file has two sections in it, one for the frames or data for each sprite and one section for meta data of the texture page. The metadata section can be largely ignored unless you need it but I included it anyway. Here is a section of the json Array
Here is the script to pull all the information shown. You can delete from the script the information you don't need but I included everything
Json Array
The purpose of an Array Json is it makes it easy to loop through all the sprite sections because of the format and extract the data for each that you want.
Json Hash
This format is better suited if you only want to extract a particular image data that you know ahead of time instead of pulling all the data at once. I don't find it that useful currently
And don't forget to destroy the resultMap ds_map to avoid a memory leak. This will also destroy all the nested data structures as well
Stay tuned for the next section where I will go over an example of how to use the data you have to draw a single image. I'll get more in detail with what the sections in the json file are for.
Target Platform: Windows
Download: (N/A) code examples below
Links:
Relevant GMC discussion:
https://forum.yoyogames.com/index.php?threads/gms-2-memory-management-inquiry-suggestion.12365/
https://forum.yoyogames.com/index.php?threads/memory-use.36369/
https://forum.yoyogames.com/index.p...ment-101-tricks-and-clarity-sprite_add.35850/
TexturePacker Website
Summary:
This is a tutorial about creating and decoding JSON data structures. This particular example uses texture pages created with a commercial software package called TexturePacker. But the techniques are applicable to other JSON data applications.
In this tutorial I will be demonstrating and showing you how to integrate the program TexturePacker by CodeandWeb into your graphics workflow. Doing so will allow you to take control over your texture pages to optimize texture memory instead of having GM do it for you. This provides you with the freedom to proper memory management natively without dlls or extensions which results in allowing you to create bigger games and use more texture memory at once than you otherwise would be able to.
Who is this system designed for?
This method is not necessary for everyone and every project. Its useful mainly for those who have really large projects and large resolution games where memory tends to be a source of concern. GM does a fine job on its own for smaller games and unless you desire this amount of control and extra work, you are better off using GM's built in IDE capabilities
Brief Introduction to memory
(skip to the tutorial if you know about this)
If anything in the spoiler is inaccurate, please inform me so I can modify it!
The memory "wall" in brief
One of the main problems we have to solve especially for the large projects is running out of memory. Currently in GM, that memory limit is 2GB (but you really only have ~1.7GB) meaning that the game will not run if you use more than this amount. This is because GM1.4 creates a 32bit application when you compile your game, I'm not sure if GM2.0 compiles a 32bit game.
The issue with adding sprites to the IDE
You might be wondering why is this needed? Doesn't GM already pack sprites onto texture pages efficiently to reduce memory usage and don't we have memory management with texture_flushing? Yes this is true however its still rather limited. Currently texture management within the IDE is only for VRAM which is memory loaded into a dedicated graphics card.
This does nothing to address the core concern that simply having resources in the IDE uses up precious system memory and chips away at that 2GB limit even if you aren't using those resources in the game. This means that even if you have an empty room in your game and no graphics loaded, you can still run out of memory simply by having a lot of graphics in the IDE.
This is due to all resources in the IDE being bundled into the same WAD file which is all put into memory. If in the future GM allows the use of multiple WAD files you can swap in/out of memory, this would make this system less useful because one of the goals of this tutorial is to mimic multiple WAD files. This brings us to...
External Files
But hey?! I can simply throw everything as an external file and never have to worry about getting close to that 2GB limit? Not so fast. If you never need to have a large amount of texture memory loaded at one time, then sure the memory limit won't be an issue. You still might run into performance issues associated with simple sprite_add functions which put each resource on its own texture page and requires swaps between each loaded resource.
However, if you are making a game with a large world, high res graphics and super fluid animations the simple sprite_add won't work. This will put each resource on its own texture page which most likely will waste space by requiring a bigger tex page than needed. Essentially you don't have the benefits of smart texture atlas to fit everything efficiently the way GM does with sprites inside the IDE
One of the main problems we have to solve especially for the large projects is running out of memory. Currently in GM, that memory limit is 2GB (but you really only have ~1.7GB) meaning that the game will not run if you use more than this amount. This is because GM1.4 creates a 32bit application when you compile your game, I'm not sure if GM2.0 compiles a 32bit game.
The issue with adding sprites to the IDE
You might be wondering why is this needed? Doesn't GM already pack sprites onto texture pages efficiently to reduce memory usage and don't we have memory management with texture_flushing? Yes this is true however its still rather limited. Currently texture management within the IDE is only for VRAM which is memory loaded into a dedicated graphics card.
This does nothing to address the core concern that simply having resources in the IDE uses up precious system memory and chips away at that 2GB limit even if you aren't using those resources in the game. This means that even if you have an empty room in your game and no graphics loaded, you can still run out of memory simply by having a lot of graphics in the IDE.
This is due to all resources in the IDE being bundled into the same WAD file which is all put into memory. If in the future GM allows the use of multiple WAD files you can swap in/out of memory, this would make this system less useful because one of the goals of this tutorial is to mimic multiple WAD files. This brings us to...
External Files
But hey?! I can simply throw everything as an external file and never have to worry about getting close to that 2GB limit? Not so fast. If you never need to have a large amount of texture memory loaded at one time, then sure the memory limit won't be an issue. You still might run into performance issues associated with simple sprite_add functions which put each resource on its own texture page and requires swaps between each loaded resource.
However, if you are making a game with a large world, high res graphics and super fluid animations the simple sprite_add won't work. This will put each resource on its own texture page which most likely will waste space by requiring a bigger tex page than needed. Essentially you don't have the benefits of smart texture atlas to fit everything efficiently the way GM does with sprites inside the IDE
The Goal
The goal is to manually perform the same texture packing method GM does with sprites inside the IDE and apply that to external loading via sprite_add. We do this by:
1. Creating our own texture pages in TexturePacker
2. Add that texture page into the IDE
3. Draw_sprite_general only the sprites in that page we want to display in our game
This is what GM does behind the scenes. You think draw_sprite(spr_walk,image_index,x,y) simply displays an isolated sprite strip? Think again. When you ask to draw spr_walk, subimage0, GM finds which messy looking texture page that subimage is on at its location and draw_part() from that texture page only where that frame is located. Draw_sprite secretly is draw_part under the hood.
What is TexturePacker?
Here's a basic overview of the tool and the concept behind what we are going to do. The next section shows how to integrate Texturepacker into a workflow that GM can use!
TexturePacker is an external program dedicated to packing and creating texture pages, just like GM does automatically except better. There is a free trial but its relatively inexpensive to purchase a full license. Its better for these reasons:
1. It gives you more control over the size of texture pages you want created
2. You can easily scale the tex page itself without needing to individually scale every sprite and even has HD scaling
3. Allows for more packing options such as polygon fitting and rotation!
4. Can preview your pages before compiling to see how they fit and how much memory will be used allowing for a greater deal of control and memory management
Does it really pack things better? Check out this simple comparison
The left was generated by GM with a max Tex Size of 4096x4096. Even on its simplest settings, Texture packer packed it in a smaller texture size than GM was able to do resulting in halving the memory usage! If external images weren't placed onto a power of 2 texture page, you could have texturepacker output a texture page size tailored exactly to what is needed for even more memory savings. If only!
However, Texturepack has an additional memory saving feature GM doesn't have that you can take advantage of:
Allow rotation enables individual sprites to be rotated 90 degrees which allows Texture pages to be packed even tighter. GM doesn't do this which results in wasted space by forcing all sprites to be packed in their original rotation instead potentially rotating them to fit optimally.
this section is only going to show how to extract the data from the files. How to apply it will be in the next section.
Outputting from TexturePacker
When you export from TexturePacker, it spits out two files: A texture page(s) and a data file associated with that texture page. TexturePacker has two file formats for export that is useful for GM
Json Array and Json Hash. There is only a small difference in the formats, the array decodes into a ds_list where all the sprites can easily be looped through and the hash decodes into an all ds_map json format. I've found that the Jason Array is the best method to work with simply because the list makes it easy to loop through but you might find the Hash could work better for what you need.
The first step is to take the exported files created from TexturePacker and through them in the included files. You can simply sprite_add the texture pages from the included files folder to load them into memory. All the magic now happens with decoding that json file and creating a drawing system to display the correct sprites when you want to
Json decoding
Credits
Before I begin I want to recognize the members that helped me out with a previous thread of mine dealing with JSON files here https://forum.yoyogames.com/index.php?threads/json-decode-help-solved.35997/
Also a huge shoutout to @chamaeleon @Tsa05 because their code snippets in that thread together gave me the information needed to put all the pieces together to create this system I'm going to be talking about in this tutorial.
When decoding the json, you can't simply json_decode that file as it will not work. Instead you have to create a string and line by line, read from the json file and add it to the string. Then decode that entire string
Code:
var filename = [included file path];
var data = "";
var json = file_text_open_read(filename);
while(!file_text_eof(json))
{
data += file_text_read_string(json);
file_text_readln(json);
}
file_text_close(json);
var resultMap = json_decode(data); // decode the json
Each Json file has two sections in it, one for the frames or data for each sprite and one section for meta data of the texture page. The metadata section can be largely ignored unless you need it but I included it anyway. Here is a section of the json Array
Here is the script to pull all the information shown. You can delete from the script the information you don't need but I included everything
Json Array
The purpose of an Array Json is it makes it easy to loop through all the sprite sections because of the format and extract the data for each that you want.
Code:
var framesList = resultMap[? "frames"]; // load all the frame section information
var metaList = resultMap[? "meta"]; // load all meta data
// loop through all sprites
for(var i=0;i<ds_list_size(framesList);i++)
{
var frameROOT = framesList[| i]; // get ALL sprite data
// extract individual data
var spritename = frameROOT[? "filename"];
var rotated = frameROOT[? "rotated"];
var trimmed = frameROOT[? "trimmed"];
// get Sprite Source Size data
var sssMap = frameROOT[? "spriteSourceSize"]; // decode the map
var sss_x = sssMap[? "x"];
var sss_y = sssMap[? "y"];
var sss_w = sssMap[? "w"];
var sss_h = sssMap[? "h"];
// get source size data
var ssMap = frameROOT[? "sourceSize"]; // decode the map
var ss_w = ssMap[? "w"];
var ss_h = ssMap[? "h"];
// get Frame data
var frameMap = frameROOT[? "frame"]; // decode the map
var frame_x = frameMap[? "x"];
var frame_y = frameMap[? "y"];
var frame_w = frameMap[? "w"];
var frame_h = frameMap[? "h"];
// get pivot data
var pivotMap = frameROOT[? "pivot"]; // decode the map
var pivot_x = pivotMap[? "x"];
var pivot_y = pivotMap[? "y"];
}
// Get Metadata
var version = metaList[? "version"]; // get version
var format = metaList[? "format"]; // get format
var scale = metaList[? "scale"]; // get scale
var image = metaList[? "image"]; // get imagename
// Get texture size
var sizeMap = metaList[? "size"];
var size_w = sizeMap[? "w"];
var size_h = sizeMap[? "h"];
This format is better suited if you only want to extract a particular image data that you know ahead of time instead of pulling all the data at once. I don't find it that useful currently
Code:
var filename = "sprite_0"; // the name of the sprite you are looking for
var framesList = resultMap[? "frames"]; // load all the frame section information
var metaList = resultMap[? "meta"]; // load all meta data
// Get Targeted Image Data
var spritename = filename;
var frameROOT = framesList[? string(spritename)]; // decode the sprite data
// extract individual data
var rotated = frameROOT[? "rotated"];
// get Sprite Source Size data
var sssMap = frameROOT[? "spriteSourceSize"]; // decode the map
var sss_x = sssMap[? "x"];
var sss_y = sssMap[? "y"];
var sss_w = sssMap[? "w"];
var sss_h = sssMap[? "h"];
// get source size data
var ssMap = frameROOT[? "sourceSize"]; // decode the map
var ss_w = ssMap[? "w"];
var ss_h = ssMap[? "h"];
// get Frame data
var frameMap = frameROOT[? "frame"]; // decode the map
var frame_x = frameMap[? "x"];
var frame_y = frameMap[? "y"];
var frame_w = frameMap[? "w"];
var frame_h = frameMap[? "h"];
// get pivot data
var pivotMap = frameROOT[? "pivot"]; // decode the map
var pivot_x = pivotMap[? "x"];
var pivot_y = pivotMap[? "y"];
// Get Metadata
var scale = metaList[? "scale"]; // get scale
var image = metaList[? "image"]; // get imagename
// Get texture size
var sizeMap = metaList[? "size"];
var size_w = sizeMap[? "w"];
var size_h = sizeMap[? "h"];
And don't forget to destroy the resultMap ds_map to avoid a memory leak. This will also destroy all the nested data structures as well
Code:
ds_map_destroy(resultMap);
Last edited: