SOLVED Saving ds_grid using JSON

I'm doing some research right now on how to take my inventory system that uses a ds_grid and saving it properly using JSON (as I will be using cross platform eventually so not wanting to use ini). From what I've learned thus far is that I need to convert my ds_grid into a ds_map eventually to then save using json_encode. I'm fairly new to saving with JSON so I just wanted to ask the community more directly about the process.

One question I have about the process is whether I need to directly convert my ds_grid into a ds_map? ds_maps seem disorganized in nature, so should I first put my ds_grid into a ds_list? This way my data is organized in a particular order as my input and so that when I need to create and pull from my saved file I have the order just the same as I put it in.

I've also noticed through my research that I cannot use ds_grid_write as it doesn't support array data which I do have in regards to my inventory item stats that are assigned uniquely to each item.

What I imagine I must do is create a ds_list (17 values) and nest inside of it ds_maps (17 keys) with each map being a key of: (name, amount, description, object, sprite, etc.) then the map has a value of a nested ds_list in the specific order I want for input/output 0 = itemName00, 1 = itemName01, etc. From tutorials I watched it seems that nested ds_maps and ds_lists are appropriate and fine to do.

Thanks for any and all feedback :)
 

samspade

Member
The most straight forward ways would be to convert the ds_grid into a list of lists or array of arrays as both lists and arrays are now json compatible as of 2.3.1. I'd recommend arrays and structs personally over ds_lists and ds_maps.
 
The most straight forward ways would be to convert the ds_grid into a list of lists or array of arrays as both lists and arrays are now json compatible as of 2.3.1. I'd recommend arrays and structs personally over ds_lists and ds_maps.
Oh wow that is cool to know!
 
The most straight forward ways would be to convert the ds_grid into a list of lists or array of arrays as both lists and arrays are now json compatible as of 2.3.1. I'd recommend arrays and structs personally over ds_lists and ds_maps.
It does seem that feature you mentioned is only available with the very latest update so not usable for 2.3 but only for 2.3.1?
json_stringify doesn't seem built in it seems with 2.3 also yoyo doesn't have any forums explaining its use in detail which is unfortunate. I may use json_encode just because I'm building my game with someone else and want to wait till all the bug patches before updating since 2.3.1 just came out like last week, and also there at least seems to be yoyo forums on json_encode.
That is fascinating to know though so thanks for that info still.
 
That's right, stringify and parse (json functions for arrays and structs) is 2.3.1 only. So if you're pre 2.3.1 you would use encode/decode. And these are still good after 2.3.1 as well so it will upgrade just fine.

Here's my tutorial on JSON stuff for encode/decode: https://www.youtube.com/playlist?list=PLwgH1hDD0q1EHVrscm8m_N2kKtQaDAgO8

And here is one for arrays and structs (really its no different though)

Wow thanks you're awesome. I'll be watching your videos now and will subscribe to your channel!
 
So its been a few days since this post. I'm not sure if I should make a new one since this may be buried and not receive responses now. Ideally I would just like to add to this post since it is the same topic specifically ds_grids using JSON. I looked over @samspade source, I've also looked at other forums, but mostly I've built my code around Shaun Spalding's older JSON video that uses 'json_encode' and 'json_decode' that deals with ds_maps and ds_lists. Unfortunately he doesn't go over pulling from a ds_grid and then storing data back into it.

For the sake of convivence I've taken a small portion of my code and condensed it down so instead of me showing all 12 values I'm saving I'm just displaying this example that shows 2 values (Name, and Amount) from my inventory.

I'm hoping someone can best explain to me the order of process. I'll attach to this my save and then my load and break down all that is going on. To begin..

To Save:
I press "P" to save and by doing this I create a dsRootList to organize and store all the column categories from my ds_grid but again here I'm just showing, Name and Amount. So after I've created the dsRootList I set a map inside of it at positon 0 named mapName, and at position 1 I set mapAmount. I then say 'ds_list_mark_as_map' and I'm not sure if I should do this at the beginning or wait until the end since I'm still adding sub lists. I create to small sub lists that are placed according to their maps. Meaning that listName is created then I set inside of it all of the Names being stored from my ds_grid inventory. Lastly I use 'ds_map_add_list' to place listName inside of my mapName under the key "NAME" to pull from later. After I've done this for each category such as Amount, etc. I then place my dsRootList inside of a map called wrapperToSave with the key "ROOT". Finally I save this as a string using 'json_encode'. The script I used from Shaun Spalding is in the next image 'scrSaveStringToFile'. Lastly to prevent memory leaks I destroy the wrapperToSave and everything inside of it.

Next Image will go into detail of me loading what I've saved.
 

Attachments

To Load:
I press "L" to load what I've saved back into the game. Scripts are to the right where I first create a wrapperToLoad using 'scrLoadFromFileJSON' to use 'json_decode'. I then create a variable rootListToLoad and pull from that wrapperToLoad the key "ROOT". Now I go through everyEntry / category and assign it to mapCategoryToLoad. After I create a variable that is loadListName and pull from mapCategoryToLoad the key "NAME" basically wanting to take all the names stored within that list.
I think this is where I get a little lost and issues in my code because when I use 'ds_list_size' the game breaks and it returns that loadListName isn't a list. To finish up and if my code was working I would then set inside my ds_grid inventory in position 0 everyName using the value loadListName. After I've repeated that for all my other categories I then destroy my wrapperToLoad to prevent memory leaks.


I just personally want to understand this process better and there doesn't seem to be much out there regarding saving ds_grids so I'm hoping that this post can also help people in the future who want to do what I'm doing using JSON. Sorry To @samspade to not fully use your tutorials I just found Shaun's helpful as well and just doing what I can with what I understand. I still appreciate your help. I hope this above was written well enough to be understood and make sense.
 

Attachments

samspade

Member
I would do a few things:
  • Step through the save function in the debugger and make sure the data that goes into the wrapper is what you want
  • Review what is actually saved in notepad or some other text editor
  • Review the data structure that is returned by json_decode in the debugger
The great thing about JSON is that it is human readable pretty much at every point of its existence so you should be able to verify, at every step, that it contains the data you want. If you find steps where the data is not going in as you want, and you're not sure why, post that section of the code here along with what you want the data to be and what it actually looks like.
 
I'm not familiar with reading and opening text files. I tried using your example
theFile = file_text_open_write(fileName);
file_text_write_string(theFile, json_encode(mapToRead));
file_text_open_read(fileName);
file_text_close(theFile);


but couldn't get anything to open, nor do I know how to access my file "savedGame.sav" that is being saved.
Here I have a show_debug_message and you can see something is being saved in there. Strangely only the key "STATSARRAY" is appearing when there should be 12 other map keys with their own lists.
 

Attachments

When I check the object and the ds_lists I noticed that it does have 12 spots for every category and its saving my items that are positioned at 6, 7, 8 which is correct the others are blank spaces, but for some reason its saving 1, 3, 1 in every category when specialInGrid should be all booleans being false. Same for nameInGrid its showing 1, 3, 1 not actually the string names.
 

Attachments

Last edited:

samspade

Member
I should have said this before, but you should be posting your actual code in between code tags, not pictures of code as that is harder to read. But having looked at your debug message, and debug window, it look like you're only saving the list indexes and not marking them as lists. So for example you aren't marking your map name map as a map when you add it. Also, you don't really need to do the length - 1 trick when you know what they are. For example, this would be slightly more straightforward:

GML:
var root_list = ds_list_create();
var map_amount = ds_map_create();
var map_name = ds_map_create();
ds_list_add(root_list, map_amount, map_name);
ds_list_mark_as_map(root_list, 0);
ds_list_mark_as_map(root_list, 1);
How are items in your inventory represented?
 
Oh okay that is really helpful! Yeah I don't usually come to the yoyo forum so I'm a little new to the etiquette of it all. 😅

I took your advice and so at the start of the code after I have created all the maps, and set them in the position I want them inside of dsRootList, I then do what you mentioned where I use "ds_list_mark_as_map" with dsRootList as the id and then I go (0 through 12) positions and now it does seem to be saving the map key name where before it was only saving the index value, and also now it is taking in all the lists that belong to their respective keys. 😁

Now onto pulling that data back in and storing it inside of my ds_grid inventory. For reference this is what my ds_grid looks like although it is bigger in width by 17 but the first 13 columns are what I'm most concerned with saving since the other values update when the inventory is open and don't really need to be saved.

Sorry to respond so late. I've been without electricity all day.
 

Attachments

As I try to import my data back in and read the json string using "json_decode" I am running into this error: 'Data structure with index does not exist'. I'm trying to retrieve the map key "NAME" from inside of the map "ROOT". It seems that retrieving the "ROOT" doesn't give me an error.

GML:
var wrapperToLoad = scrLoadFromFileJSON("savedGame.sav");

var rootListToLoad = wrapperToLoad[? "ROOT"];

var findNameMap = rootListToLoad[? "NAME"];
for(everyName = 0; everyName < ds_list_size(findNameMap); ++everyName){
   var loadListName = findNameMap[| everyName];
   ds_grid_set(global.playerInventory, 0, everyName, loadListName); 
}
the line that is giving me the error is
var findNameMap = rootListToLoad[? "NAME"];

I do see the map key NAME when I debug and look at the string and the list of values inside of it is all there. I know there may be an issue with ds_list_size or ds_map_size whatever I need to use but the first error is my current road block.
I apologize having to rely on you so heavily for advice but so much of json is new to me and so I do very much appreciate your feedback.
 

FrostyCat

Redemption Seeker
GML:
function grid_to_map(grid) {
    var map = ds_map_create();
    var width = ds_grid_width(grid);
    var height = ds_grid_height(grid);
    map[? "width"] = width;
    var data = ds_list_create();
    for (var yy = 0; yy < height; ++yy) {
        for (var xx = 0; xx < width; ++xx) {
            ds_list_add(data, grid[# xx, yy]);
        }
    }
    ds_map_add_list(map, "data", data);
    return map;
}

function map_to_grid(map) {
    var data = map[? "data"];
    var size = ds_list_size(data);
    var width = map[? "width"];
    var height = size div width;
    var grid = ds_grid_create(width, height);
    for (var i = 0; i < size; ++i) {
        grid[# i mod width, i div width] = data[| i];
    }
    return grid;
}
GML:
ds_map_add_map(wrapperToSave, "inventory", grid_to_map(global.playerInventory));
GML:
ds_grid_destroy(global.playerInventory);
global.playerInventory = map_to_grid(wrapperToLoad[? "inventory"]);
 
GML:
function grid_to_map(grid) {
    var map = ds_map_create();
    var width = ds_grid_width(grid);
    var height = ds_grid_height(grid);
    map[? "width"] = width;
    var data = ds_list_create();
    for (var yy = 0; yy < height; ++yy) {
        for (var xx = 0; xx < width; ++xx) {
            ds_list_add(data, grid[# xx, yy]);
        }
    }
    ds_map_add_list(map, "data", data);
    return map;
}

function map_to_grid(map) {
    var data = map[? "data"];
    var size = ds_list_size(data);
    var width = map[? "width"];
    var height = size div width;
    var grid = ds_grid_create(width, height);
    for (var i = 0; i < size; ++i) {
        grid[# i mod width, i div width] = data[| i];
    }
    return grid;
}
GML:
ds_map_add_map(wrapperToSave, "inventory", grid_to_map(global.playerInventory));
GML:
ds_grid_destroy(global.playerInventory);
global.playerInventory = map_to_grid(wrapperToLoad[? "inventory"]);

I don't know what more to say except for.. wow.. and thank you. @FrostyCat your mind is beautiful and this is a work of art. You just took what was more than 200 lines of code and made it into less than 40 lines.. I'm literally speechless at what you've given me and everything works wonderfully.

Thank you to @samspade and to @FrostyCat for your help. You two are absolutely fantastic! 😍🤩🥳
 
Last edited:
I have noticed that it is saving everything including my stats that are arrays.

However when I load my grid back in the stats that are arrays are now instead a single integer value as seen in the image attached instead of it being all arrays. As seen slot [12] in my grid is where all my stats go and currently slots [6], [7], and [8] have items. None should be displaying a single integer and even if there isn't an item it should display a blank array of nine 0s. I will keep messing with it and see if I'm able to properly extract from what has been successfully saved. From me looking at the json string that is saved everything including the arrays are there and besides slot [12] everything is properly positioned and placed. I'll keep messing with it.. 😅
 

Attachments

Last edited:
@FrostyCat Would I need to save a ds_list for all of my 13 arrays inside of my grid in order to properly load from it?
I'm contemplating on how I go about this building on top of what you've given me. Initially I thought what I must do is create one ds_list of stats separately and store it inside of my map in addition to the data list inside of your function grid_to_map. Then using the other function map_to_grid I load in the map with the key stats and then I overwrite that column [12] before returning my grid. It seems despite all of that it is still being treated as a single value rather than an array.

Now I'm wondering if I need to have a ds_list for every array in order to properly load it through. Meaning 13 lists inside of grid_to_map with 13 keys that I can save to my return map and then load in each key individually as an array through the function map_to_grid. It seems excessive but that seems to be the only thing I've not yet tried at this point.

As mentioned before the stat arrays do seem to be properly saved when I debug its just the loading that is returning a single value instead of an array.
 
Last edited:
Depends on what version of GMS you are using. I haven't looked into it fully, but I'm pretty sure you can stringify arrays for storage in JSON (json_stringify()) in 2.3 (maybe 2.3.1, not too sure). However, if you're using a previous version, you'll have to convert your arrays into lists before json'ing them. It's possible to do this without converting every array in your project like this (using frosty's example):
Code:
   for (var yy = 0; yy < height; ++yy) {
        for (var xx = 0; xx < width; ++xx) {
            if (is_array(grid[# xx,yy])) {
               var _arr = grid[# xx,yy];
               var _list = ds_list_create();
               for (var i=0;i<array_length_1d(_arr);i++) {
                  ds_list_add(_list,_arr[i]);
               }
               ds_list_add(data,_list);
               ds_list_mark_as_list(data,ds_list_size(data)-1);
            }
            else {
               ds_list_add(data, grid[# xx, yy]);
            }
        }
    }
Pressed send before I'd finished, but in order to load the arrays back in, you'll have to manually know where each array is meant to be stored, and loop through the list saved in the data structure there, storing each entry from it inside the appropriate array, this'll be a bit more tedious and specific to your project. But depending on how much work converting every array to a list is, it might be worth it.
 
I'm using GM2 version that doesn't have json_stringfy. I am strongly considering doing so as exporting the array seems to be of great difficulty. I just wanted to use json_encode/json_decode because since the 2.3.1 is relatively new I'm not sure what issues might arise from it before bugs are fixed. That said I also wonder if json_encode/decode will be obsolete in the future maybe it is better to update.. Also since I'm working with another programmer I'm trying not to update if I don't have to. I did try your example above but had no success it still returns a single value. I even tried to write out each list with their own key and try and create arrays inside the function map_to_grid but still getting the same result everytime.
 
Last edited:

FrostyCat

Redemption Seeker
My JSON Structs library was designed to do what json_parse and json_stringify are for in 2.3.0.
GML:
function grid_to_struct(grid) {
    var width = ds_grid_width(grid);
    var height = ds_grid_height(grid);
    var arrays = array_create(height);
    for (var yy = height-1; yy >= 0; --yy) {
        var currentRow = array_create(width);
        for (var xx = width-1; xx >= 0; --xx) {
            currentRow[xx] = grid[# xx, yy];
        }
        arrays[yy] = currentRow;
    }
    return {
        width: width,
        height: height,
        data: arrays
    };
}

function struct_to_grid(strc) {
    var height = strc.height;
    var width = strc.width;
    var arrays = strc.data;
    var grid = ds_grid_create(width, height);
    for (var yy = height-1; yy >= 0; --yy) {
        var currentRow = arrays[yy];
        for (var xx = width-1; xx >= 0; --xx) {
            grid[# xx, yy] = currentRow[xx];
        }
    }
    return grid;
}
GML:
jsons_save(working_directory + "savedGame.sav", {
    inventory: grid_to_struct(global.playerInventory)
});
GML:
var data = jsons_load(working_directory + "savedGame.sav");
ds_grid_destroy(global.playerInventory);
global.playerInventory = struct_to_grid(data.inventory);
Going forward, if you want easy cross-platform portability in exported data, DO NOT mix arrays/structs with DS data structures, or different kinds of DS data structures other than maps with lists. DS data structures are so incompetent in this regard that I've spent a month writing this library to replace it in daily use.
 
My JSON Structs library was designed to do what json_parse and json_stringify are for in 2.3.0.
GML:
function grid_to_struct(grid) {
    var width = ds_grid_width(grid);
    var height = ds_grid_height(grid);
    var arrays = array_create(height);
    for (var yy = height-1; yy >= 0; --yy) {
        var currentRow = array_create(width);
        for (var xx = width-1; xx >= 0; --xx) {
            currentRow[xx] = grid[# xx, yy];
        }
        arrays[yy] = currentRow;
    }
    return {
        width: width,
        height: height,
        data: arrays
    };
}

function struct_to_grid(strc) {
    var height = strc.height;
    var width = strc.width;
    var arrays = strc.data;
    var grid = ds_grid_create(width, height);
    for (var yy = height-1; yy >= 0; --yy) {
        var currentRow = arrays[yy];
        for (var xx = width-1; xx >= 0; --xx) {
            grid[# xx, yy] = currentRow[xx];
        }
    }
    return grid;
}
GML:
jsons_save(working_directory + "savedGame.sav", {
    inventory: grid_to_struct(global.playerInventory)
});
GML:
var data = jsons_load(working_directory + "savedGame.sav");
ds_grid_destroy(global.playerInventory);
global.playerInventory = struct_to_grid(data.inventory);
Going forward, if you want easy cross-platform portability in exported data, DO NOT mix arrays/structs with DS data structures, or different kinds of DS data structures other than maps with lists. DS data structures are so incompetent in this regard that I've spent a month writing this library to replace it in daily use.
Just to understand what I'm looking at, the code you just posted is for JSON saving using arrays and structs for the updated GM2.3.1? That code wouldn't be useable for GM2?

Also you're saying that for cross platform I should avoid storing my arrays of stats entirely within my ds_grid because cross platform wont work well when reading arrays from ds_grids? Would you recommend ds_grids at all for inventory systems for cross platform? I've heard mixed things. I feel comfortable using ds_grids when I created my room generator and saw some tutorials of people using them for examples on inventories hence why I decided to opt for it (also it is nice having my data in a convenient place to retrieve and add to). I'm curious about how you would develop an inventory for cross platform, if you would use ds_grid at all or strictly have objects carry their data within themselves and save that way?

Thanks for the advice.
 
I'm thinking of just creating a hybrid at this point. Since I save my object, sprite, and have it placed in a specific order that maybe I save the stats in the object and then load them back into the object. This has been quite the learning experience at least. 🤔 I don't mind rebuilding what I have if it means I have a more efficient inventory, and if anything I think I can still reuse your grid_to_map and map_to_grid to do other things later involving my room generator.
 
Last edited:

FrostyCat

Redemption Seeker
Just to understand what I'm looking at, the code you just posted is for JSON saving using arrays and structs for the updated GM2.3.1? That code wouldn't be useable for GM2?
That code was meant to be used with my JSON Structs library, which works on 2.3.0 to provide functionality similar to json_parse and json_stringify in 2.3.1.
Also you're saying that for cross platform I should avoid storing my arrays of stats entirely within my ds_grid because cross platform wont work well when reading arrays from ds_grids? Would you recommend ds_grids at all for inventory systems for cross platform?
Yes, you should avoid putting arrays in grids unless HTML5 is entirely ruled out, or you are willing to man-handle the entire saving process yourself.

No, I do not recommend using DS data structures for ANYTHING post 2.3.0, especially not for saving data. You've seen for yourself why.
I've heard mixed things. I feel comfortable using ds_grids when I created my room generator and saw some tutorials of people using them for examples on inventories hence why I decided to opt for it (also it is nice having my data in a convenient place to retrieve and add to).
You're getting mixed messages because you're mixing pre-2.3 advice with post-2.3 advice. There are much better ways to handle inventories using new 2.3-exclusive syntax.
I'm curious about how you would develop an inventory for cross platform, if you would use ds_grid at all or strictly have objects carry their data within themselves and save that way?
I would develop the inventory entirely in structs-and-arrays space. Like this, for example:
GML:
function Inventory() constructor {
    items = array_create(16, undefined);   

    static addItem = function(item) {
        ...
    };
    static dropItem = function(pos) {
        ...
    };

    static reduceToData = function() {
        ...
    };

    static expandFromData = function(data) {
        ...
    };

    ...
}
Then all access to the data (e.g. UI elements, object instances, save/load) are mediated through the fields and supplied methods.

As for whether I'd use DS grids for any of this, absolutely not. I have completely stopped using DS data structures in my own work, for many reasons:
  • They require manual cleanup.
  • They serialize in an opaque, developer-unserviceable manner, and this problem publicly blew up as of 2.3.0.
  • They do not serialize the same way in native exports as on HTML5.
  • They require error-prone manual work to serialize when nested in non-JSON ways, for example grids-inside-grids or stacks-inside-queues.
  • Even when they nest in JSON ways (i.e. maps and lists inside each other), the syntax is no less verbose and equally error-prone.
  • json_encode encodes both lists and arrays to [ ... ], but json_decode decodes [ ... ] only to lists (hence the single numbers you saw when loading, those were list IDs)
The arrays-and-structs replacements that I wrote for them have had NONE of those problems since the day it was born.
 
Last edited:
That code was meant to be used with my JSON Structs library, which works on 2.3.0 to provide functionality similar to json_parse and json_stringify in 2.3.1.

Yes, you should avoid putting arrays in grids unless HTML5 is entirely ruled out, or you are willing to man-handle the entire saving process yourself.

No, I do not recommend using DS data structures for ANYTHING post 2.3.0, especially not for saving data. You've seen for yourself why.

You're getting mixed messages because you're mixing pre-2.3 advice with post-2.3 advice. There are much better ways to handle inventories using new 2.3-exclusive syntax.

I would develop the inventory entirely in structs-and-arrays space. Like this, for example:
GML:
function Inventory() constructor {
    items = array_create(16, undefined); 

    static addItem = function(item) {
        ...
    };
    static dropItem = function(pos) {
        ...
    };

    static reduceToData = function() {
        ...
    };

    static expandFromData = function(data) {
        ...
    };

    ...
}
Then all access to the data (e.g. UI elements, object instances, save/load) are mediated through the fields and supplied methods.

As for whether I'd use DS grids for any of this, absolutely not. I have completely stopped using DS data structures in my own work, for many reasons:
  • They require manual cleanup.
  • They serialize in an opaque, developer-unserviceable manner, and this problem publicly blew up as of 2.3.0.
  • They do not serialize the same way in native exports as on HTML5.
  • They require error-prone manual work to serialize when nested in non-JSON ways, for example grids-inside-grids or stacks-inside-queues.
  • Even when they nest in JSON ways (i.e. maps and lists inside each other), the syntax is no less verbose and equally error-prone.
  • json_encode encodes both lists and arrays to [ ... ], but json_decode decodes [ ... ] only to lists (hence the single numbers you saw when loading, those were list IDs)
The arrays-and-structs replacements that I wrote for them have had NONE of those problems since the day it was born.
I appreciate your advice and educating me on JSON. I do think I will update my engine after all and try to do some more research and review the newer tutorials out there that use structs and arrays and json_stringfy. Thank you @FrostyCat and to all above for your time. I've learned a lot from this forum and from all those that contributed. I will go ahead and put 'solved' on this and end it here. 🙂
 

Tewwww

Member
GML:
function grid_to_map(grid) {
    var map = ds_map_create();
    var width = ds_grid_width(grid);
    var height = ds_grid_height(grid);
    map[? "width"] = width;
    var data = ds_list_create();
    for (var yy = 0; yy < height; ++yy) {
        for (var xx = 0; xx < width; ++xx) {
            ds_list_add(data, grid[# xx, yy]);
        }
    }
    ds_map_add_list(map, "data", data);
    return map;
}

function map_to_grid(map) {
    var data = map[? "data"];
    var size = ds_list_size(data);
    var width = map[? "width"];
    var height = size div width;
    var grid = ds_grid_create(width, height);
    for (var i = 0; i < size; ++i) {
        grid[# i mod width, i div width] = data[| i];
    }
    return grid;
}
GML:
ds_map_add_map(wrapperToSave, "inventory", grid_to_map(global.playerInventory));
GML:
ds_grid_destroy(global.playerInventory);
global.playerInventory = map_to_grid(wrapperToLoad[? "inventory"]);
thanks for this, works like a charm
 
Top