1. Hey! Guest! The 36th GMC Jam will take place between February 27th, 12:00 UTC - March 2nd, 12:00 UTC. Why not join in! Click here to find out more!
    Dismiss Notice
  2. NOTICE: We will be applying a Xenforo update on Tuesday 25th of February. This means that from approximately 10:00 to 14:00 BST the forums will be offline (or possibly longer). Sorry for the inconvenience! Official Announcement here.

Behavior of ds_list_clear() with Nested Lists

Discussion in 'Programming' started by TheMagician, Feb 11, 2020.

  1. TheMagician

    TheMagician Member

    Joined:
    Jun 20, 2016
    Posts:
    66
    I read a JSON string into a ds_map using json_decode().

    Inside there is a list of lists. Here is an overview over the hierarchy and the indices that GMS assigns to the data structures:

    {map id 0
    ..[list id 3
    ....[list id 0]
    ....[list id 1]
    ....[list id 2]
    ..]
    }


    So far so good.

    Now I copy list 3 into a new list using

    ds_list_copy(list_new, 3);

    This works as intended. The original list (id 3) gets copied into a new list (id 4) which now also contains the references/indices of lists 0,1,2.

    However, when I now do

    ds_list_clear(list_new);

    GMS will not just remove the values 0,1,2 from the new list but will also destroy/free the corresponding lists!

    Is that intended behavior?

    I would have expected such a behavior from ds_list_destroy() because it also destroys any nested data structures but from my understanding ds_list_clear() should only empty the slots in the list?
     
    Last edited: Feb 11, 2020
    Ido-f likes this.
  2. samspade

    samspade Member

    Joined:
    Feb 26, 2017
    Posts:
    2,236
    This sentence can't be true unless you have a typo: "This works as intended. The original list (id 3) gets copied into a new list (id 4) which now also contains the references/indices of lists 0,1,2." If you did it as you wrote, then you would be copying, what is presumably a blank list, into list 3. Instead, it should be:

    Code:
    
    ds_list_copy(list_new, 3);
    
    
    See the manual. That sentence aside, if you created a new blank list and then copied the contents (nothing) of the blank list into list three and then cleared the presumably already empty list you would get two blank lists.

    Since list ids are just numbers you could verify whether or not those lists that disappear still exist by using ds_exists(1, ds_type_list) right after the copy and clear.
     
  3. TheMagician

    TheMagician Member

    Joined:
    Jun 20, 2016
    Posts:
    66
    That was indeed a typo here in my pseudo-code. I've corrected my original post.

    And I know for a fact that the referenced lists get destroyed/freed because I checked using ds_exists(). My question is: is this a bug or intended behavior?

    Also, what is the workaround if I want to free the new list (id 4) from memory without freeing the original nested lists?
     
  4. Ido-f

    Ido-f Member

    Joined:
    Feb 19, 2018
    Posts:
    146
    Maybe you destroyed the upper level ds_map somewhere in between? That also destroys every list and map nested within it.

    Alternatively, maybe the resulting lists after json_decode aren't actually id #0,1,2? Even if they were when you Json_encoded them, Json_decode assigns new ds id's for them (as Json_encode doesn't save the original id's, only the ds' contents)
     
  5. kupo15

    kupo15 Member

    Joined:
    Jun 20, 2016
    Posts:
    926
    Yes! I ran into this too and it doesn't make any sense to me either. I think clear should simply remove the pointer references from the list and not destroy the nested lists. I filled a bug report weeks ago and haven't heard back yet
     
  6. TheMagician

    TheMagician Member

    Joined:
    Jun 20, 2016
    Posts:
    66
    @Ido-f: I've done my research and narrowed it down to ds_list_clear() being the culprit as described in the original post.

    @kupo15: Glad to hear I'm not the only one with that problem. I think I will also file a bug report to back you up on this. There's nothing in the manual that indicates this behavior.

    Did you come up with a walkaround? What seems to work for me is that I first iterate through the list and set all of the entries to undefined and only after that I call ds_list_destroy().
     
  7. kupo15

    kupo15 Member

    Joined:
    Jun 20, 2016
    Posts:
    926
    I'm happy to hear I'm not alone as well. No I haven't gotten far enough to attempt a workaround, I just went a different direction entirely. I like your workaround for the issue for the moment.
     
  8. samspade

    samspade Member

    Joined:
    Feb 26, 2017
    Posts:
    2,236
    It does seem that ds_list_clear will destroy nested lists, but only if they are marked as nested lists. You can show this with the following code in the debugger:

    Code:
    
    main_list = ds_list_create();
    list_a = ds_list_create();
    list_b = ds_list_create();
    list_c = ds_list_create();
    
    ds_list_add(main_list, list_a, list_b, list_c);
    ds_list_mark_as_list(main_list, 0);
    ds_list_mark_as_list(main_list, 1);
    ds_list_mark_as_list(main_list, 2);
    
    copy_of_main_list = ds_list_create();
    ds_list_copy(copy_of_main_list, main_list);
    
    ds_list_clear(copy_of_main_list);
    list_exists = false;
    list_exists = ds_exists(list_a, ds_type_list);
    list_exists = ds_exists(list_b, ds_type_list);
    list_exists = ds_exists(list_c, ds_type_list);
    
    
    If you remove the three ds_list_mark_as_list lines, then the three sub lists will still exists. You can also just do:

    Code:
    
    main_list = ds_list_create();
    list_a = ds_list_create();
    list_b = ds_list_create();
    list_c = ds_list_create();
    
    ds_list_add(main_list, list_a, list_b, list_c);
    ds_list_mark_as_list(main_list, 0);
    ds_list_mark_as_list(main_list, 1);
    ds_list_mark_as_list(main_list, 2);
    
    ds_list_clear(main_list);
    list_exists = false;
    list_exists = ds_exists(list_a, ds_type_list);
    list_exists = ds_exists(list_b, ds_type_list);
    list_exists = ds_exists(list_c, ds_type_list);
    
    
    And get the same results. In other words, ds_list_clear on nested lists marked as lists will delete the list. This does seem like it should be considered a bug or at least a failure in documentation (I can half understand why you might want this behavior as clearing a list with sub lists might forever lose you the reference to the list and therefore cause a memory leak, but this seems like a bad way to solve that problem since it prevents you from doing things you might actually want or need to do).

    When you copy a list with sub lists marked as lists, the marked as list data does carry over, which is why ds_list_clear on the copy list also destroys the sub lists.
     
  9. kupo15

    kupo15 Member

    Joined:
    Jun 20, 2016
    Posts:
    926
    I got a reply from my bug filing of this

    Also is there no way to "unmark" marked lists? It seems like that would be a helpful feature
     
  10. TheMagician

    TheMagician Member

    Joined:
    Jun 20, 2016
    Posts:
    66
    Here's my answer from the bug filing:

    I don't think that "others on that forum said that they would expect the clear to remove the sub-lists in order to avoid memory leaks" is really the take-away from this thread. Anyway, if the manual gets updated with the information on how to correctly clear a list without freeing the nested lists then I guess I'm happy.
     
    Ido-f likes this.
  11. kupo15

    kupo15 Member

    Joined:
    Jun 20, 2016
    Posts:
    926
    ouch sounds like you murdered a kitten or something! Yeah i don't see where memory leaks came from either. What should be the expected outcome of this function? I originally thought that only the target list should be cleared with the nested in tack, but it sounds like perhaps this should be clearing everything that is nested as well? In which case, it would seem like your undefined workaround is the way to go
     
    TheMagician likes this.

Share This Page