GameMaker [SOLVED] ds_list_mark_as_ behavior question

[hughrock18]

Hello fellow devs!

I am building a complicated save system (will be so much fun when done), and part of my system relies on the use of ds_list_copy (ex. copy the "char_data_list" to "var _temp_list". Grab one of the many ds_maps held by the list. Change values of said map, and then save to "_temp_list" and replace "char_data_list" with "_temp_list" to "update" the saved information. This uses "ds_list_copy" and "ds_list_replace" functions to clarify. I don't know if there's a difference, but I tend to avoid using the shorthand "blahblah[? "key"] = value" stuff.

Now the questions (hoping I've made sense so far):

1. When i set up "char_data_list" and the inclusive ds_maps, i used "ds_list_mark_as_map" function on each of the list positions. Now that the list has been marked (fully) as having a bunch of ds_maps, do I ever need to use the "ds_list_mark_as" function when "ds_list_replace"-ing anything within said list?

2. I may have missed this one in the manual to be honest. If i were to define the 3rd position (ds_list[| 2]) how does gamemaker deal with the 1st and 2nd positions? I imagine they are also made, but filled with a default value. Since we're already talking about ds_lists, what is that default value, and (in that same example) if I were to grab the value of the 4th position (hasn't been previously defined) would I get an error, or would gms add the 4th position and fill it with a default value (like when opening a file that doesn't exist yet).

I really appreciate any information.

 

[nacho_chicken]

1. When i set up "char_data_list" and the inclusive ds_maps, i used "ds_list_mark_as_map" function on each of the list positions. Now that the list has been marked (fully) as having a bunch of ds_maps, do I ever need to use the "ds_list_mark_as" function when "ds_list_replace"-ing anything within said list?

Forgive me if I'm wrong, but it sounds like you're a bit mistaken in the use of those functions. The ds_list_mark_as_* functions are only for use when encoding JSON.

This functionality is designed only for use when encoding JSON strings (which you can create using json_encode) and the complimentary ds map functions.

https://docs2.yoyogames.com/source/_build/3_scripting/4_gml_reference/data_structures/ds lists/ds_list_mark_as_map.html

Dealing with nested data structures is a lot easier than that. GM handles data structure types internally, so you don't have to mark them by type. You just specify the variable pointing to the ds_* and GM will do the rest.
Here's an example of what I'm talking about:

var map = ds_map_create(),
    list = ds_list_create();
list[| 2] = map;
map[? "test"] = 3;
show_message(ds_map_find_value(list[| 2], "test"));
// Shows "3"

// Bonus answering question 2
show_message(list[| 1]);
// "0" is the default
show_message(list[| 3]);
// "undefined" when reading outside of a ds_list's bounds

 

[hughrock18]

Forgive me if I'm wrong, but it sounds like you're a bit mistaken in the use of those functions. The ds_list_mark_as_* functions are only for use when encoding JSON.

https://docs2.yoyogames.com/source/_build/3_scripting/4_gml_reference/data_structures/ds lists/ds_list_mark_as_map.html

Dealing with nested data structures is a lot easier than that. GM handles data structure types internally, so you don't have to mark them by type. You just specify the variable pointing to the ds_* and GM will do the rest.
Here's an example of what I'm talking about:

var map = ds_map_create(),
    list = ds_list_create();
list[| 2] = map;
map[? "test"] = 3;
show_message(ds_map_find_value(list[| 2], "test"));
// Shows "3"

I AM using json encoding to save the massive ds_map (called, "main") into a file. The ds_map, "main", is holding the ds_list, "char_data_list" (which is then holding a ds_map for each char and npc). In order for the saving process to work correctly, every ds_map/list needs to be marked as such within a list.

The "main" ds_map is built and adjusted regularly throughout the game (as it also holds all account_data, item_data, settings_data, world_data, etc.). Saving will be manual, and only when the player presses the "save" button will the "main" ds_map be saved into a file.

I'm still (relatively) new to json_encoding/decoding. Is there a problem with marking the maps within a list when not immediately saving afterwards? I like to mark my list positions immediately after writing to them to keep everything organised and ready. Are you saying that I need to, instead, use a for loop (or the like) to mark the positions within lists ONLY when saving said lists (using json) to a file in the same code block?

I thought that as long as i kept up with what parts of the list are marked as what, then everything SHOULD run smoothly. The "should" part is what I'm unsure of and the reason I made this thread; as I just don't know how i can expect ds_list_mark -ing to behave (especially when interacting with, or just plain replacing, the value(s) stored at the marked position).

For all i know, I could be making a huge mistake if the "ds_list_mark_as_*" functions act like surfaces on mobile (unstable and can't be expected to still exists at the beginning of the next step).

My assumption is that marking a position is telling gamemaker that that list position holds a ds_map/list. Even if said map/list were removed/replaced the position is what I think gm is watching. If THAT is the case, then it would make sense that I could mark a position and leave it as is (regardless of how the value(s) at the marked positions are changed afterwards).

If you have a "better" (more like an easier) way that will still work with json file saving, I'm open to suggestions. Although, i would still want my question answered.

EDIT: in my assumption, i would still need to make sure that the value stored at a marked position was still what I marked it as.

 

[nacho_chicken]

I AM using json encoding to save the massive ds_map (called, "main") into a file. The ds_map, "main", is holding the ds_list, "char_data_list" (which is then holding a ds_map for each char and npc). In order for the saving process to work correctly, every ds_map/list needs to be marked as such within a list.

My apologies. It wasn't really clear from the first post and a LOT of users new to data structures make the error of using ds_list_mark_as_* unnecessarily. I'm afraid I can't be of any more help since I mostly use JSON for loading external data and I'm not too familiar with using JSON for saving data structures.

 

[the_dude_abides]

@nacho_chicken
Encoding to JSON is the means to write the data of the nested structure to a text file, and so save it for later use. Flagging a list or a map is part of that JSON function, but is not the only way it can be used.

The other function of adding a list to a map, or vice versa, in this fashion, is that deleting the "container" structure ensures deletion of all of those nestled structures.

This topic is marked as GMS 2, and I think it does "garbage collection" automatically. Whereas as I believe GMS 1.4 does not.

A 1.4 user would very much need to do this process, or end up with lots of memory leaks from data structures not being destroyed. If that is correct then it really needs to be checked which version a thread states they have (if they've tagged it with such info), otherwise the wrong advice will be given.

 

[hughrock18]

@nacho_chicken
Encoding to JSON is the means to write the data of the nested structure to a text file, and so save it for later use. Flagging a list or a map is part of that JSON function, but is not the only way it can be used.

The other function of adding a list to a map, or vice versa, in this fashion, is that deleting the "container" structure ensures deletion of all of those nestled structures.

This topic is marked as GMS 2, and I think it does "garbage collection" automatically. Whereas as I believe GMS 1.4 does not.

A 1.4 user would very much need to do this process, or end up with lots of memory leaks from data structures not being destroyed. If that is correct then it really needs to be checked which version a thread states they have (if they've tagged it with such info), otherwise the wrong advice will be given.

i'm using gms2. I wasn't even aware of the existence of ds_list_mark_as_* when i used gms1.*

I actually already knew about how marking list positions turns it into a nested structure, and destroying the "wrapper" (as it was called in the save file system i read into) it cascades through all ds_maps/list/grids and destroys them as well.

My issue is the BEHAVIOR of how gamemaker studio: 2 treats the ds_list_mark_as_* functions. When a list position is marked as a list/map, what is gamemaker studio: 2 actually doing to "mark" that position?

 

[Homunculus]

The first question is very valid, a while ago I tested the behavior of deleting, inserting, replacing etc... on ds_maps and ds_lists in relation to nested DS. Found out that replacing or deleting a list or map entry holding a nested DS deletes the "marker" as well (but not the referenced DS). In ds_lists, as you may expect, what gets marked is not the list position itself but the list item, meaning that operations changing the nested item position in the list (like inserting or deleting before it) will move the "marker" along with its marked item.

I suggest to test these cases yourself though, it has been quite some time and GMS2 versions since I tested and, while improbable, you never know if there are differences between platforms.

I think that the manual should go a bit more in-depth and clearly state what happens for those specific use cases, and while you may reasonably expect a specific behaviour when dealing with nested DS, it's really not obvious imho, and should be clearly stated in the docs.

EDIT: also, isn't it time to get rid of the "only use for JSON purposes" in favor of something more useful? These functions are being used (and suggested) also as a way to ensure deletion when getting rid of the main DS, and I see no reason to consider this use case invalid.
Moreover, the warning doesn't really make sense in scenarios where the data is frequently serialized to JSON as a save mechanism but the structure is being kept and modified throughout the gameplay as well. If I really had to follow the manual on this, I should - copy the map > mark whatever needs to be marked > save as JSON > delete the map - every time instead of saving the data directly, since those functions are "to be used for JSON only".

 

[the_dude_abides]

@Brennan Duncan
I've done this a specific way, to show you something. When you @ a user it tags them into the conversation, and replies to them directly. I did it with nacho_chicken because there was something they said that I felt needed addressing.

My apologies if you were unaware of this as a new(ish) user. Those comments weren't to you, and should not be taken as a response to your thread - just something said within

Unfortunately I am not familiar enough with the process to actually answer your question

 

[hughrock18]

@the_dude_abides
My bad. As you guessed, I didn't know. I'll keep that in mind. Thanks.

Unfortunately I am not familiar enough with the process to actually answer your question

I appreciate it anyway.

 

[hughrock18]

@Catan: I was actually figuring that gm would mark the position as appose to the structure. I WILL have to do some in depth testing. If the "marker" is moved with the structure (in the event of replacing the nested ds), would that mean that deleting the "wrapper" (no one has corrected me on whether this is the correct jargon) would STILL delete the "marked" structure as well, regardless of where it ended up? OR is that the part I have to test?

EDIT: I apologize for double posting. I pressed post by accident, and forgot i could edit until a second ago.

 

[Homunculus]

If you replace a nested ds that has been marked, the marker is deleted as well. Deleting the wrapper will not delete the nested ds that has been removed from the list at this point. Maybe an example is more clear:

inner = ds_list_create();
inner[| 0] = "foo";

wrapper = ds_list_create();
wrapper[| 0] = inner;
ds_list_mark_as_list(wrapper, 0);

/* destroying wrapper here would delete inner as well */

wrapper[| 0] = "something";
ds_list_destroy(wrapper);

show_debug_message(inner[| 0]); //inner still exists, and will show "foo" in the console

About question 2:
If you set an index in a list that's greater than the current size, all non initialized indexes will be set to 0. See the docs for ds_list_set.
If instead you try to read from outside the bounds, you may get 0 or undefined. See ds_list_find_value for details about this.

 

[hughrock18]

@Catan: I noticed you first said that it would be deleted (the marker), but then you said that it was "moved" in the following sentence. Thanks for clarifying. I would actually be worried if the removed nested ds would still be deleted after being "un-nested". Since that would mean that gm is doing shiz behind the scenes in a way i wouldn't like (just like the "solid" gm option). While not entirely needed at this point, the illustration is also very clear, and serves to show that maybe I'm NOT wrong when i call the "head" of the nested ds' hierarchy a "wrapper". Also, thanks for the answer to my second question. It was already answered, but I failed to mark it as such. My bad.

Side question: A "wrapper" ds_map has a "root" ds_list that everything is saved to. For a reason I don't know, it's somehow instrumental that the saved data is stored in a root and that root in a wrapper before saving to a file via json. Does that root HAVE to be a ds_list? What's the difference between "ds_map_add_map" (which i thought automatically nested the structure - the original reason I thought a "marked" list position could exist for extended periods of time, since the ds_map_add_* functions "mark" the added structures and keep them that way) and marking a list position with the ds_list_mark_as_* functions? Do they "mark" differently?

 

[Homunculus]

I don't think there's a correct term for the "main" structure, I tend to use the same terms you do interchangeably.

You can't save a ds_list directly to JSON, since json_encode takes a ds_map, that's why you have to create a "wrapper" map. What's inside the map doesn't really matter, you could have anything in there. In you case it happens to be a single ds_list.

ds_map_add_* and ds_list_mark_as_* have the same purpose, nesting data structures. For some reason, in the case of ds_maps, nested structures are added and marked in a single call, while in the case of lists you first have to add, and then mark. The purpose is the same though.
You could totally make your own script ds_list_add_list and ds_list_add_map to make things more consistent.

 

[hughrock18]

@Catan: Nice. That makes much more sense now. It's a json specific thing, not a gm or gml thing. You'll have to forgive my ignorance. Like i said previously, I'm still relatively new to json. It really helps to actually understand how the functions function. At least, i think it isn't enough to know that "it works". Moreover, since ds_map_add_* and ds_list_mark_as_* are (more or less) the same with an extra step, it makes sense to assume that a ds_list full of nested structures is perfectly OK to exist "loosely" (for the sake of adjusting the nest before actually saving it) throughout the game. The only way I can see this NOT working is if there was a specific reason that gm didn't just make a "ds_list_add_list/map". Said reason might be an issue. Of course, human error/oversight is enough to make things look out of place even intended otherwise. So... either i look for someone who at least helped MAKE gamemaker (as it is today), or i do some extensive testing... Looks like the debugger is going to earn its keep today. Anything i said that sounds off, or doesn't make sense? I want a second opinion (since mine is likely to come from limited knowledge).

 

[hughrock18]

Does this (especially the "result checking" part) make for a working "add and mark" script for ds_lists and ds_maps (obviously a second script will be made to accommodate adding lists to lists)?

/// @func ds_list_add_map();
/// @desc combines (ds_list_add) and (ds_list_mark_as_*) for ease of use
/// @arg list_to_add_to
/// @arg map_to_add
/// @arg [list_position] OPTIONAL (leave blank to target the END of the list)

var _list = argument[0];
var _map = argument[1];
var _pos = ( argument_count > 2 ) ? ( argument[2] ) : ( ds_list_size(_list) );

var _result = false; // result is verified before being set to true

ds_list_set( _list, _pos, _map );
ds_list_mark_as_map( _list, _pos );

if( _list[| _pos] == _map ) _result = true; // result checking

return( _result );

 

[Homunculus]

That makes much more sense now. It's a json specific thing, not a gm or gml thing.

It is a GM specific thing, but related almost exclusively to JSON serialization.

:it makes sense to assume that a ds_list full of nested structures is perfectly OK to exist "loosely" (for the sake of adjusting the nest before actually saving it) throughout the game.

It can definitely exist "loosely" until you need to encode it to JSON. Even then, you can address the list directly to change it, no need to go through the wrapper ds_map to get a reference if you already have a reference to the list in a variable.

Anything i said that sounds off, or doesn't make sense? I want a second opinion (since mine is likely to come from limited knowledge).

Makes sense to me, but all this stuff being undocumented behavior could definitely benefit from someone else opinion in case I said something out of place.

Your script looks ok, although return does not require parentheses and the verification is a bit pointless. You can avoid creating the _result variable though by just returning this:

return _list[| _pos] == _map;

 

[hughrock18]

@Homunculus: I appreciate the feedback. I had to teach myself how to program (for the most part). My programming style is a little unrefined as a result. All i can really aim to do is maintain organization. I understand the lack of a need for the "result checking". It was more to figure out if comparing the map's index to a nested ds (same map) would be the same thing. I figured it would be, but wasn't sure. Now i know how gm nests data structures (at least enough to make efficient use of it).

Thanks for putting up with me. I suppose that that makes my issue resolved. Thanks again.