Problem saving and loading data structures with json

Toulhane

Member
Hi,

I followed this Shaun Spalding tutorial to try to implement a saving system in my game while adapting it a bit :
My game being a card game, I wanted to save my card inventory which is a ds map (I should probably have made it as a ds list instead but I don't think that's the problem here).
Here is the code I use :

In my obj_master "press s" event I have this to save my cards inventory
GML:
// create root list
 var _root_list = ds_list_create();

// save deck into root list
//ds_list_add(_root_list, obj_deck.deck);
var deck_copy = ds_map_create();
ds_map_copy(deck_copy, obj_deck.deck);
ds_list_add(_root_list, deck_copy);
ds_list_mark_as_map(_root_list, ds_list_size(_root_list)-1);

// wrap root list in a map
var wrapper = ds_map_create();
ds_map_add_list(wrapper, "root", _root_list);

// save in a file as string
var _string = json_encode(wrapper);
scr_save_string_to_file("savegame.sav", _string);
And in my obj_master "press l" event I have this to load what I saved
GML:
if (file_exists("savegame.sav") )
{

    
    var wrapper = scr_load_json_from_file("savegame.sav");
    var list = ds_map_find_value(wrapper, "root");
    
    
     var _deck = ds_list_find_value(list, 0);
    
    ds_map_destroy(obj_deck.deck);
    
    obj_deck.deck = ds_map_create();
    ds_map_copy(obj_deck.deck, _deck);
    
    
    
}
Here is the script used in the save event to encode all of this with JSON :
GML:
function scr_save_string_to_file(argument0, argument1){
    
    var filename = argument0;
    var _string = argument1;
    
    var buffer = buffer_create(string_byte_length( _string)+1, buffer_fixed, 1);
    buffer_write(buffer, buffer_string, _string)
    buffer_save(buffer, filename);
    buffer_delete(buffer);
}
And here is the script used in the load event to decode this file :
GML:
function scr_load_json_from_file(argument0){
    
    var filename = argument0;
    
    var buffer = buffer_load(filename);
    var _string = buffer_read(buffer, buffer_string);
    buffer_delete(buffer);
    
    var json = json_decode(_string);
    return json;

}
The problem is that everything runs just fine, except that I have the correct number of entries in my loaded "deck" ds map (if I saved while having 6 cards in the deck, there indeed 6 entries in the ds map once loaded), but all the entries are undefined.
I've done some testing and everything is good until the JSON script is ran, I have the correct number of entries and each entry contains a card name. But once loaded, all entries are suddenly undefined.
I have to say that I have pretty much no idea what those JSON script taken from the tutorial actually do, so I'm pretty helpless at figuring out what is going wrong.

This is really frustrating and I would really appreciate any help.

And just in case : I know I should destroy my data structure once it is saved, but I didn't do it for practicality reasons during testing and even if I actually destroy it, the problem remains.
 

Nocturne

Friendly Tyrant
Forum Staff
Admin
Before Doing anything, check that the encoded JSON is correct. You can use "show_debug_messge" on the JSON string before saving and then copy the output into a JSON prettifier to see what it actually contains. Also, I would suggest calling JSON encode on the deck DS map directly and avoid ALL THE REST of that code. It's needlessly jumping through hoops to create extra lists and maps that aren't required and that are over-complicating everything. Basically you only need to do the following:
GML:
// SAVE
json_encode on the deck DS map
Write to buffer
Save buffer
Destroy buffer

// LOAD
read buffer
jason_decode on the string from the buffer
destroy the deck DS map
assign the decoded DS map to the deck variable
destroy the buffer
No need to create ANY other DS maps or lists or anything... and you'll find the code far easier to maintain and also far easier to understand if this is all a bit new to you. :)
 

Toulhane

Member
Before Doing anything, check that the encoded JSON is correct. You can use "show_debug_messge" on the JSON string before saving and then copy the output into a JSON prettifier to see what it actually contains. Also, I would suggest calling JSON encode on the deck DS map directly and avoid ALL THE REST of that code. It's needlessly jumping through hoops to create extra lists and maps that aren't required and that are over-complicating everything. Basically you only need to do the following:
GML:
// SAVE
json_encode on the deck DS map
Write to buffer
Save buffer
Destroy buffer

// LOAD
read buffer
jason_decode on the string from the buffer
destroy the deck DS map
assign the decoded DS map to the deck variable
destroy the buffer
No need to create ANY other DS maps or lists or anything... and you'll find the code far easier to maintain and also far easier to understand if this is all a bit new to you. :)
Thanks for the help! Unfortunatly, sipmplyfing the code didn't solve the problem.

To keep things as simple as possible, I used this code to save in the "deck" object :
GML:
// save in a file as string
var _string = json_encode(deck);
show_debug_message(_string);
scr_save_string_to_file("savegame.sav", _string);
and this code to load :
GML:
if (file_exists("savegame.sav") )
{
    
    var _deck = scr_load_json_from_file("savegame.sav");
    show_debug_message(_deck);
    
    
    ds_map_destroy(deck);
    
    deck = ds_map_create();
    ds_map_copy(obj_deck.deck, _deck);
        
}
As for the debug messages here is what I got when saving after putting 3 attack card and 3 armor cards in the deck ds map:
Code:
{
   "2":"strenght",
   "3":"armor",
   "1":"strenght",
   "5":"armor",
   "4":"armor",
   "0":"strenght"
}{
   "2":"strenght",
   "3":"armor",
   "1":"strenght",
   "5":"armor",
   "4":"armor",
   "0":"strenght"
}{
   "2":"strenght",
   "3":"armor",
   "1":"strenght",
   "5":"armor",
   "4":"armor",
   "0":"strenght"
}{
   "2":"strenght",
   "3":"armor",
   "1":"strenght",
   "5":"armor",
   "4":"armor",
   "0":"strenght"
}{
   "2":"strenght",
   "3":"armor",
   "1":"strenght",
   "5":"armor",
   "4":"armor",
   "0":"strenght"
}{
   "2":"strenght",
   "3":"armor",
   "1":"strenght",
   "5":"armor",
   "4":"armor",
   "0":"strenght"
}{
   "2":"strenght",
   "3":"armor",
   "1":"strenght",
   "5":"armor",
   "4":"armor",
   "0":"strenght"
}
I don't really know what to say about this since there seems to be the right number of cards inside of each brackets but I don't understand why there are so many brackets.
And the json formater tells me that there are the following errors:
Code:
Error:
[Code 22, Structure 25]
Error:
[Code 22, Structure 50]
Error:
[Code 22, Structure 75]
Error:
[Code 22, Structure 100]
Error:
[Code 22, Structure 125]
Error:
[Code 22, Structure 150]
And here is what I got in the debug message when I loaded:
Code:
2
3
4
5
6
7
8
I started trying to understand how JSON encoding works, but honestly I'm a bit lost.
 

Nocturne

Friendly Tyrant
Forum Staff
Admin
Okay, that's interesting! The JSON contains seven copies of the DS map information. So, the first thing I'd do is check and see what the DS map actually contains before it's converted to JSON. If it contains 7x the number of entries then part of the issue is that you're DS map is wrong, so I'd fix that first then come back to the saving/loading bit. You can check the contents by adding a breakpoint to the line where you call JSON encode, then in the debugger, when the code stops, you can check the contents of the DS map in the Output window.
 

Toulhane

Member
Okay, that's interesting! The JSON contains seven copies of the DS map information. So, the first thing I'd do is check and see what the DS map actually contains before it's converted to JSON. If it contains 7x the number of entries then part of the issue is that you're DS map is wrong, so I'd fix that first then come back to the saving/loading bit. You can check the contents by adding a breakpoint to the line where you call JSON encode, then in the debugger, when the code stops, you can check the contents of the DS map in the Output window.
Ok, I "solved" the problem regarding the seven copies: I put the code in a "keyboard" event instead of a "keyboard press" event. So the event was getting triggered everything frame until I released the key. And I did the same thing with the load event...
So now there is the correct amount of entries in the ds map and the correct names, but still every entries return "undefined" once loaded.
And the debug message from this code:
GML:
var _deck = scr_load_json_from_file("savegame.sav");
show_debug_message(_deck);
now returns "2".

Edit:
I looked at this code:
GML:
function scr_load_json_from_file(argument0){
   
    var filename = argument0;
   
    var buffer = buffer_load(filename);
    var _string = buffer_read(buffer, buffer_string);
    buffer_delete(buffer);
   
    show_debug_message(_string);
   
    var json = json_decode(_string);
    return json;

}
And everything is fine until the json_decode.
The debug code returns this:
Code:
{ "2": "strenght", "3": "armor", "1": "strenght", "5": "armor", "4": "armor", "0": "strenght" }
Which are the same values found in the string value saved in the save event. And the json var simply returns "2".
So, my guess is there is either a problem with the json_decode, or with the way I handle the decoded values (but I can't figure out what I'm doing wrong).
 
Last edited:

4i4in

Member
I put the code in a "keyboard" event instead of a "keyboard press" event.
Try:
io_clear();

How long is the list You are trying to save?
Try first to write it in .ini and see what You get there. Its much easier (and shorter code) to use ds on .ini files.
 

samspade

Member
Returning 2 may not be a mistake. If you are using json_decode you'll get a data structure and a variable that holds a data structure is just holds an int which is a reference to that data structure show calling show_debug_message will just give you a number. Instead, you should review this in the actual debug and right click the variable then choose view as map/list (as relevant) to see if the data is correct.

That said, if there's still an issue, I would strongly suggest removing the buffer line while debugging. That way you can review the text file that is saved out. In general my steps would be:
  1. Review the data in the debugger before using json_encode or json_stringify
  2. Review the string you get from using those functions
  3. Review the external file (which is why you should save it out without the buffer line)
  4. Review the string you get from loading in the file
  5. Review the data structure you get after using json_decode or json_parse
Steps 1 and 5 should show the same results and steps 2-4 should also show the same results. You can copy the text to something like jsoneditor online if you want to view it with line breaks instead of all one big string.

Also, use the debugger, not the show debug message. You need to actually be able to review and compare.
 

Toulhane

Member
Returning 2 may not be a mistake. If you are using json_decode you'll get a data structure and a variable that holds a data structure is just holds an int which is a reference to that data structure show calling show_debug_message will just give you a number. Instead, you should review this in the actual debug and right click the variable then choose view as map/list (as relevant) to see if the data is correct.

That said, if there's still an issue, I would strongly suggest removing the buffer line while debugging. That way you can review the text file that is saved out. In general my steps would be:
  1. Review the data in the debugger before using json_encode or json_stringify
  2. Review the string you get from using those functions
  3. Review the external file (which is why you should save it out without the buffer line)
  4. Review the string you get from loading in the file
  5. Review the data structure you get after using json_decode or json_parse
Steps 1 and 5 should show the same results and steps 2-4 should also show the same results. You can copy the text to something like jsoneditor online if you want to view it with line breaks instead of all one big string.

Also, use the debugger, not the show debug message. You need to actually be able to review and compare.
Ok, first of all, thank you very much for telling me to check the debug mode, I actually did it but didn't look at the right thing. I should have looked at the content of the loaded ds map in there.
I have found the issue this way. What is happening is, while the keys in my original ds map are integer number like, 1, 2, 3, etc, the json decode converts these integer into strings, like "1", 2", "3". So obviously the code that was supposed to show me the content of the ds map and was looking for integer was not able to access the content of the new ds map.
I guess that's a normal behavior for the json decode, I should have paid attention to that, and I shouldn't have used a ds map to store things as I would with a ds list. Now that I know what the issue is, I should be able to convert back those strings into integer and everything should work fine.

Edit:
So, instead of having this in the load event to load the map:
GML:
ds_map_copy(deck, _deck);
I used this:
GML:
for(var i = 0; i < ds_map_size(_deck); i++)
    {
        var val = ds_map_find_value(_deck, string(i));
        ds_map_replace(deck, i, val);
    }
And everything seems to be working fine. Thank you very much to all of you for your help, I will probably be more cautious with my debugging methods in the future.
 
Last edited:

samspade

Member
Ok, first of all, thank you very much for telling me to check the debug mode, I actually did it but didn't look at the right thing. I should have looked at the content of the loaded ds map in there.
I have found the issue this way. What is happening is, while the keys in my original ds map are integer number like, 1, 2, 3, etc, the json decode converts these integer into strings, like "1", 2", "3". So obviously the code that was supposed to show me the content of the ds map and was looking for integer was not able to access the content of the new ds map.
I guess that's a normal behavior for the json decode, I should have paid attention to that, and I shouldn't have used a ds map to store things as I would with a ds list. Now that I know what the issue is, I should be able to convert back those strings into integer and everything should work fine.
Maybe, but it actually sounds like maps and lists aren't getting added as maps and lists. So if you have a map, and you add a map or a list to it, but you don't mark it as a map or a list, then when it is saved out, it will only save out the reference to the map or the list not the data contained in it. Once you load it back in, it will still just be a number and won't be usable as a data structure.

There are two good ways to tell if internal data structures are being added correctly. The first is in the debugger you only need to tell GM to view the top level one as a map or list and all nested ones are immediately viewable as nests and lists. The second way of course is what is getting saved out. What should be saved out in the case of a nested data structure is all of the data in that data structure, not the reference number.

GML:
map = ds_map_create();

list_a = ds_list_create();
list_b = ds_list_create();
ds_list_add(list_a, 0, 1, 2, "Hello World");
ds_list_add(list_b, 0, 1, 2, "Hello World");

ds_map_add(map, "list_a", list_a);
ds_map_add_list(map, "list_b", list_b);

//if you save this out you will get
{
    "list_a": 0,
    "list_b": [
        0,
        1,
        2,
        "Hello World"
    ]
}
You need the second form.
 

Toulhane

Member
Maybe, but it actually sounds like maps and lists aren't getting added as maps and lists. So if you have a map, and you add a map or a list to it, but you don't mark it as a map or a list, then when it is saved out, it will only save out the reference to the map or the list not the data contained in it. Once you load it back in, it will still just be a number and won't be usable as a data structure.

There are two good ways to tell if internal data structures are being added correctly. The first is in the debugger you only need to tell GM to view the top level one as a map or list and all nested ones are immediately viewable as nests and lists. The second way of course is what is getting saved out. What should be saved out in the case of a nested data structure is all of the data in that data structure, not the reference number.

GML:
map = ds_map_create();

list_a = ds_list_create();
list_b = ds_list_create();
ds_list_add(list_a, 0, 1, 2, "Hello World");
ds_list_add(list_b, 0, 1, 2, "Hello World");

ds_map_add(map, "list_a", list_a);
ds_map_add_list(map, "list_b", list_b);

//if you save this out you will get
{
    "list_a": 0,
    "list_b": [
        0,
        1,
        2,
        "Hello World"
    ]
}
You need the second form.
I was actually directly encoding the ds map conaining the cards name without putting it in another map using this code:
GML:
// save in a file as string
var _string = json_encode(deck);

scr_save_string_to_file("savegame.sav", _string);
So there should not have been a problem with ds map not being stored as ds map.

I tried this anyway just to be sure:
GML:
var save_map = ds_map_create();

ds_map_add_map(save_map, "deck", deck);

// save in a file as string
var _string = json_encode(save_map);

scr_save_string_to_file("savegame.sav", _string);
The save_map content is correct, there is my deck ds map with the correct values inside of it, but the output once loaded is exactly the same.
When I acess the loaded map with this:
GML:
var save_file = scr_load_json_from_file("savegame.sav");
   
var _deck = ds_map_find_value(save_file, "deck");
quotations marks have been added to all the key numbers.

And I was still able to process everything using this code:
GML:
ds_map_destroy(deck);
   
deck = ds_map_create();
for(var i = 0; i < ds_map_size(_deck); i++)
{
    var val = ds_map_find_value(_deck, string(i));
    ds_map_replace(deck, i, val);
}
 
Top