SOLVED Saving / loading maps using JSON; getting the error "Data structure with index does not exist"

Owly

Member
Hey all,

I'm experiencing some difficulties creating a save game system, specifically with saving / loading maps that store lists inside them.

I have a map oData.map_oIntObj_indexes which stores lists with data about certain objects.

In my save script I have tried both this (where _map is the map to be saved using JSON):

_str = ds_map_write(oData.map_oIntObj_indexes);
ds_map_add(_map,"map_oIntObj_indexes", _str);

And later on this:

_temp_map = ds_map_create();
ds_map_copy(_temp_map, oData.map_oIntObj_indexes);
_str = ds_map_write(_temp_map);
ds_map_add(_map,"map_oIntObj_indexes", _str);


And respectively I've tried in my load script both:

ds_map_read(oData.map_oIntObj_indexes,_map[? "map_oIntObj_indexes"]);

And:

_load_map =ds_map_create();
_str = _map[? "map_oIntObj_indexes"];
ds_map_read(_load_map, _str);
ds_map_copy(oData.map_oIntObj_indexes, _load_map);

And in both instances when I load the game I get the error "Data structure with index does not exist" which refers to a line where lists I have stored in the map (map_oIntObj_indexes) are referred to, which makes me believe that while the map is 'saved' the lists inside it are not.

I am experiencing no difficulties in saving and loading lists, arrays, etc, only with type of data structure.

I'm using Windows 10 and my GM version is IDE 2.2.5.481, Runtime 2.2.5.378.

Please help if you can :) Thanks!
 

samspade

Member
It's been awhile since I've used maps regularly, but does ds_map_copy deep copy nested data structures (you can test this by viewing the copy in the debugger)? If not, copying the map won't work. You also don't need to copy the map before using ds_map_write because that function doesn't touch the underlying map.

Also, just as a note this is not a JSON structure. This is using GM's built-in functions to save maps to strings. Useful, but honestly using the JSON structure while a little more complicated gives you human readable save files which are an incredible help when debugging. For example in this case, you could inspect the string and see what is actually getting saved into it.
 

Owly

Member
It's been awhile since I've used maps regularly, but does ds_map_copy deep copy nested data structures (you can test this by viewing the copy in the debugger)? If not, copying the map won't work. You also don't need to copy the map before using ds_map_write because that function doesn't touch the underlying map.

Also, just as a note this is not a JSON structure. This is using GM's built-in functions to save maps to strings. Useful, but honestly using the JSON structure while a little more complicated gives you human readable save files which are an incredible help when debugging. For example in this case, you could inspect the string and see what is actually getting saved into it.
Hey samspade, thanks for replying! The rest of the code does save it using JSON, I've basically used Shaun Spalding's template as he showcased it in a video of his. Everything works with the exception of saving / loading maps that store lists and I'm at a loss why.

I don't know if it does a deep copy, I do know that it saves the number that refers to the map (the debugger shows the same number), however I have no idea why the map structure or its stores queues is not saved / loaded properly, or gets destroyed somehow.... really frustrating :(
 
Last edited:

chamaeleon

Member
For JSON serialization I'd recommend using structs and arrays instead of ds_maps and ds_lists. No hassles dealing with allocations and deallocations. No keeping track of whether a particular entry in a map or list needs to be stored or marked as a map or list.

Edit: You haven't shown how you store your lists in the map. I assume you have used ds_map_add_list(), or GMS won't know there is a list for that key.
Edit 2: Using the mark/add functions implies the container takes ownership of the stored maps or lists, and destruction of the container will destroy the stored maps and lists as well, meaning you can't still refer to them elsewhere as part of your game state, or whatever, they need to be copies in such a case.
 
Last edited:

Owly

Member
For JSON serialization I'd recommend using structs and arrays instead of ds_maps and ds_lists. No hassles dealing with allocations and deallocations. No keeping track of whether a particular entry in a map or list needs to be stored or marked as a map or list.

Edit: You haven't shown how you store your lists in the map. I assume you have used ds_map_add_list(), or GMS won't know there is a list for that key.
Edit 2: Using the mark/add functions implies the container takes ownership of the stored maps or lists, and destruction of the container will destroy the stored maps and lists as well, meaning you can't still refer to them elsewhere as part of your game state, or whatever, they need to be copies in such a case.
Hey Chameleon! Thanks for replying!

1) I will learn how to use structs in the future, but since my current code for the game doesn't use it I'd rather not employ it at the moment.

2) You may be onto something! I just tried saving / loading another map entirely and it worked fine. I suppose the problem here is that with this specific map I do use ds_map_add_list in order to store the lists inside the map (I use this command to prevent memory leaks, to ensure that when the map gets destroyed upon exiting the game, the lists get destroyed too.)

Specifically this is how I add the lists into the map:

ds_map_add_list(oData.map_oIntObj_indexes, _name, ds_list_create());
ds_list_copy(oData.map_oIntObj_indexes[? _name], _indexes);

(the map is oData.map_oIntObj_indexes and the list is _indexes)

If ds_map_add_list is the culprit, how else can I write this piece of code to prevent unwanted data destruction with regards to saving?
 

chamaeleon

Member
Does _indexes contain ds_lists or ds_maps, or is it just numbers and strings? If it is ds_lists or ds_maps, you can't just copy _indexes, you'd need to recursively copy any stored ds_lists or ds_maps (and use the mark as map or list functions to make sure your toplevel data structure destruction destroys them as well, or manual cleanup).

Compare all of that with
GML:
var _indexes = [foo, bar, baz]; // or something converting your current _indexes ds_list to an array
oData.map_oIntObj_indexes = {};
oData.map_oIntObj_indexes[$ _name] = $_indexes;
json = json_stringify(oData.map_oIntObj_indexes);
...
oData.map_oIntObj_indexes = json_parse(json);
var _indexes = oData.map_oIntObj_indexes[$ _name];
// convert _indexes array to your ds_list version
Not having to use ds_lists and ds_maps in the first place for any of this would be much nicer of course. No worrying about whether a particular structure or array is referred to anywhere else.
If ds_map_add_list is the culprit, how else can I write this piece of code to prevent unwanted data destruction with regards to saving?
You cannot prevent the destruction without losing the ability to use the ds_map json serialization function. If you want to retain the data structure it implies you have not marked it as a map or list, which mean GMS won't know it needs to serialize it.

Oh, also, the mark as list and map stuff may not be heeded when using the read and write functions instead of json functions (don't quote me, it's just a thought without solid evidence), which may mean you end up with only numbers being stored without the creation of the nested structures when parsing json into a ds_map. If this is the case, you're out of luck using those functions by themselves.

Personally I stay far away from the read/write functions. Serialization I want to either be a format I control, or a well-defined format like JSON. Black box strings without definition that could change between GMS releases I don't want to rely on.
 

Owly

Member
Does _indexes contain ds_lists or ds_maps, or is it just numbers and strings? If it is ds_lists or ds_maps, you can't just copy _indexes, you'd need to recursively copy any stored ds_lists or ds_maps (and use the mark as map or list functions to make sure your toplevel data structure destruction destroys them as well, or manual cleanup).

Compare all of that with
GML:
var _indexes = [foo, bar, baz]; // or something converting your current _indexes ds_list to an array
oData.map_oIntObj_indexes = {};
oData.map_oIntObj_indexes[$ _name] = $_indexes;
json = json_stringify(oData.map_oIntObj_indexes);
...
oData.map_oIntObj_indexes = json_parse(json);
var _indexes = oData.map_oIntObj_indexes[$ _name];
// convert _indexes array to your ds_list version
Not having to use ds_lists and ds_maps in the first place for any of this would be much nicer of course. No worrying about whether a particular structure or array is referred to anywhere else.

You cannot prevent the destruction without losing the ability to use the ds_map json serialization function. If you want to retain the data structure it implies you have not marked it as a map or list, which mean GMS won't know it needs to serialize it.

Oh, also, the mark as list and map stuff may not be heeded when using the read and write functions instead of json functions (don't quote me, it's just a thought without solid evidence), which may mean you end up with only numbers being stored without the creation of the nested structures when parsing json into a ds_map. If this is the case, you're out of luck using those functions by themselves.

Personally I stay far away from the read/write functions. Serialization I want to either be a format I control, or a well-defined format like JSON. Black box strings without definition that could change between GMS releases I don't want to rely on.
Oh this problem was doing my head in :) I will certainly learn structs soon...

Anyway I found this solution. I'll just turn the lists into strings and then add those strings into the map. So now I have:


var _str = ds_list_write(_indexes);
ds_map_add(oData.map_oIntObj_indexes, _name, _str);

And this way no data seems to be destroyed when loading / saving (at least I just tested it and it seems to work....!) I sure hope I won't have to mess with it more.


Thanks for all your input. <3 You are all awesome. If it breaks again I'll make an update :x If it still works when I test it again tomorrow I'll update the thread to 'resolved.'
 

kupo15

Member
Oh god, I do not miss grid list nested marked as type save system at all. It's a mess. Please do yourself a favor as stated and switch to struct right away lol saving is literally 4 lines of code. Json stringify, write to a buffer, save file. Done!
 

Owly

Member
Thanks all. I will learn structs as you all suggest.

Meanwhile as said previously I think I got it to work with strings.

Cheers! Your input is so much appreciated.

I shall resolve this topic now.
 
Top