Properly nesting JSON (Juju's tut) [SOLVED]

Discussion in 'Programming' started by kupo15, Dec 14, 2019.

  1. kupo15

    kupo15 Member

    Joined:
    Jun 20, 2016
    Posts:
    882
    Juju Adam's talk on JSON saving was amazing and Sean's breakdown tut of it really helped out for most of the saving you need to do in a game. How do we continue that and do more branches if we need to? Here is what I did by keeping the structure the same without branching. Is this okay or should I actually create another level for the grid instead of saving the grid_write string?

    Code:
    var root_list = argument0;
    
    // save every wishlist
    var wishlist_size = ds_grid_height(wishlist_grid);
    for(var i=0;i<wishlist_size;i++) // loop through every wishlist in the grid
        {
        var _map = ds_map_create();
        ds_list_add(root_list,_map);
        ds_list_mark_as_map(root_list,ds_list_size(root_list)-1);
    
        // add wishlists
        var _name = wishlist_grid[# 0,i]; // wishlist name
        var _item_grid_str = ds_grid_write(wishlist_grid[# 1,i]); // id of item grid THIS VARIABLE
        var _owner = wishlist_grid[# 2,i]; // owner id
        var _date = wishlist_grid[# 3,i]; // event date
        var _unique_id = wishlist_grid[# 4,i]; // unique id
        var _private = wishlist_grid[# 5,i]; // private
       
        ds_map_add(_map,"name",_name); // save wishlist name
        ds_map_add(_map,"itemlist",_item_grid_str); // save ids of item lists THIS VARIABLE
        ds_map_add(_map,"owner",_owner); // save owner id of list
        ds_map_add(_map,"date",_date); // save event date
        ds_map_add(_map,"unique_id",_unique_id); // save unique list id (-1 custom or preset id)
        ds_map_add(_map,"private",_private); // save private or not
        }
    What I essentially have is a grid inside a grid. Each JSON treebranch saves the contents of the main grid, however column 2 contains the ID of another grid. I simply did a ds_write here and saved that string in the JSON but should I be branching this out and looping through that grid to save the values directly like how I did here?

    If so how would I go about that? I feel like its essentially nesting a copy of the code above inside the above but am not sure how to go about that
     
    SilentxxBunny likes this.
  2. SoVes

    SoVes Member

    Joined:
    May 17, 2017
    Posts:
    158
    I think just using nested lists instead of a grid would be easier to encode
     
    kupo15 likes this.
  3. samspade

    samspade Member

    Joined:
    Feb 26, 2017
    Posts:
    2,191
    I didn't think you could save a grid using JSON - all the data will be lost once you save - only maps and lists are legal. Edit: looks like you asked about using grid write - I personally would just use nested lists or maps and would find that easier. Conceptually I find it easier, but it would also allow you to read the data outside if you wanted to, and makes the structure itself easier to modify.

    I'd also just recommend using this: https://marketplace.yoyogames.com/assets/8066/json-toolkit.
     
    Last edited: Dec 14, 2019
  4. kupo15

    kupo15 Member

    Joined:
    Jun 20, 2016
    Posts:
    882
    How would you do that? Similar to how I converted the main wishlist into a list? Here is a diagram of the data I'm storing to help visual
    [​IMG]

    How would I split that itemlist up within the JSON structure and how would I go about coding it?

    I think I'm missing something in this diagram

    upload_2019-12-14_14-0-44.png
     
  5. samspade

    samspade Member

    Joined:
    Feb 26, 2017
    Posts:
    2,191
    A grid can be considered as an array of arrays (or list of lists in this case). Just decide whether you want it to be a be by column or row. Based on your diagrams it seems like it should be by row, and probably the rows should be maps. so you would have a map that holds everything, and then you would have a list (wishlist), where each list holds a map that has name, itemlist, etc. and then item listitself is a list, where each list holds a map that has name, price, etc.
     
  6. SoVes

    SoVes Member

    Joined:
    May 17, 2017
    Posts:
    158
    Yeah so you can think of every x coordinate holding a list of the column. I'm not at my pc, but I do have a script of turning nested lists into a grid. You'll have to figure it out in reverse. It shouldn't be much more difficult
    Code:
    ///@arg list
    
    var _list = argument0;
    
    var _grid = ds_grid_create(ds_list_size(_list),ds_list_size(_list[|0]));
    
    for (var _xx = 0; _xx < ds_list_size(_list); _xx++){
        
        var _xlist = _list[| _xx];
            
        for (var _yy = 0; _yy < ds_list_size(_xlist); _yy++){
            
            _grid[# _xx,_yy] = _xlist[| _yy];
            
        }   
    }
    ds_list_destroy(_list);
    
    return(_grid);
     
    kupo15 likes this.
  7. samspade

    samspade Member

    Joined:
    Feb 26, 2017
    Posts:
    2,191
    While admittedly a personal preference, in general, I don't think you should use a grid unless there are connections in both the x and y (e.g. position in 2d). If you don't need that type of relationship then it is an artificial limitation that requires more work and gets you nothing except the ability to reference natively in a single line, something you can easily get through a custom script (such as provided by the JSON utility I linked) and will have natively anyway once 2.3 comes out. (also sort, but you can code that pretty easily as well).
     
  8. kupo15

    kupo15 Member

    Joined:
    Jun 20, 2016
    Posts:
    882
    Yep, from my original code it appears as if I've already have the first part down which is separating out the original wishlist by row. The issue that I'm not clear on doing is how to split that initial map into another I think map? Because I have the main wishlist with its headers, and linked to that is another list which contains all the items inside it. So those lists are connected to each other yet completely separate lists. So I think the json format needs to be:

    ROOT > list > [map > map] - brackets represents one branch of the list

    ******in detail****

    ROOT : root directory
    ..
    list : contains a list of all the wishlists created 0-n (each list entry holds a map)
    ...
    map : contains keys "wishlist" and "itemlist" and the values are maps
    ...
    map: contains the data for either the wishlist or the itemlist

    Yeah, I guess its personal preference. I don't understand how using a grid is more limited? Would you prefer yourself making a bunch of lists for each wishlist instead of combining it all as a grid in the form of rows? That seems more convoluted to me actually. And if you wanted to order the wishlists by priority or date for display, it seems much easier to put them all in a grid so you can sort them by a column. Also if you wanted to move the order around on the fly it seems easier. So in that sense I guess all the wishlists and items inside the wishlists are connected?
     
    Last edited: Dec 16, 2019
  9. kupo15

    kupo15 Member

    Joined:
    Jun 20, 2016
    Posts:
    882
    Ok I think I'm getting a bit closer. I'm just having trouble figuring out how to wrap and mark things properly. Here's my updated code. I treated each grid (wishlist and itemlist) as creating its own map and now its a matter of wrapping it correctly, what am I doing wrong here? I'm pretty sure I'm close but not quite right. the code marked with the ************** is where I think the mistake is. @samspade @GMWolf

    This is the structure I think I need to make
    [​IMG]


    Code:
    var root_list = ds_list_create(); // contains map keys "wishlist" and "itemlist"
    
    // save every wishlist
    var wishlist_size = ds_grid_height(wishlist_grid);
    for(var i=0;i<wishlist_size;i++) // loop through every wishlist in the grid
        {
          // create submenu map ********************
        var _map = ds_map_create(); // each list contains a map of "wishlist" and "itemlist"
        ds_map_add_list(_map,"Wishlist",_root_list);
        ds_map_add_list(_map,"Itemlist",_root_list);
     
        // create wishlist map
          var _map_wish = ds_map_create();
          ds_list_add(root_list,_map_wish );
          ds_list_mark_as_map(root_list,ds_list_size(root_list)-1);
    
          // get wishlist values
          var _name = wishlist_grid[# 0,i]; // wishlist name
          var _owner = wishlist_grid[# 2,i]; // owner id
          var _date = wishlist_grid[# 3,i]; // event date
          var _unique_id = wishlist_grid[# 4,i]; // unique id
          var _private = wishlist_grid[# 5,i]; // private
    
          // add wishlists to a map
          ds_map_add(_map_wish ,"name",_name); // save wishlist name
          ds_map_add(_map_wish ,"owner",_owner); // save owner id of list
          ds_map_add(_map_wish ,"date",_date); // save event date
          ds_map_add(_map_wish ,"unique_id",_unique_id); // save unique list id (-1 custom or preset id)
          ds_map_add(_map_wish ,"private",_private); // save private or not
    
          // add to wishlist "key" *************************
          ds_map_add(_map,"Wishlist",_map_wish); // add above map under the key "wishlist"
    
        // create ltemlist map
          var _map_item = ds_map_create();
          ds_list_add(root_list,_map_item );
          ds_list_mark_as_map(root_list,ds_list_size(root_list)-1);
    
          // get item values
          var item_grid = wishlist_grid[# 1,i]; // get the id of item grid
    
          var _name = item_grid [# 0,i]; // item name
          var _price = item_grid [# 2,i]; // item price
          var _qty = item_grid [# 3,i]; // item qty
          var _url = item_grid [# 4,i]; // item url
          var _notes = item_grid [# 5,i]; // item notes
    
          // add itemlists to a map
          ds_map_add(_map_item ,"name",_name); // save item name
          ds_map_add(_map_item ,"price",_price); // save item price
          ds_map_add(_map_item ,"qty",_qty); // save item quantity
          ds_map_add(_map_item ,"url",_url); // save url
          ds_map_add(_map_item ,"notes",_notes); // save notes
    
          // add to itemlist map *****************
          ds_map_add(_map,"Itemlist",_map_item); // add above map under the key "itemlist"
        }
     
    // wrap the data up
    var _wrapper = ds_map_create();
    ds_map_add_list(_wrapper,"ROOT",_root_list)
    
    // save as string
    var _string = json_encode(_wrapper);
    
     
    Last edited: Dec 16, 2019
  10. samspade

    samspade Member

    Joined:
    Feb 26, 2017
    Posts:
    2,191
    Regarding the above, I would run the debugger to check. So long as each nested map and list is marked as such (which they need to be to be saved) you can view the entire structure in the debugger.

    Pros of using nested lists over grid:
    • easier to move internally, to swap rows of a grid you have to turn each row into something else, such as an array, save one, then swap. To swap nested lists you just swap the 'pointers' skipping the copy to array step and all data moves automatically
    • easier to move and change order. For example, to move a row in a grid to the middle of another grid you need to copy the entire row to something like an array, resize the second grid, manually shift all rows and then insert. To move a nested list from the middle of one list to another you simply move the 'pointer' and use the built in ds_list_insert.
    • easier to view and debug: with the built in mark as list and mark as map functions you can automatically view the entire nested structure in the debugger.
    • easier to avoid memory leaks: with the built in mark as list and mark as map functions deleting the wrapper automatically deletes all nested maps and lists.
    • easier to save: with the built in json structure you can save without having to write your grid to something else.
    • are not limited to same length of rows or consistency between rows
    Pros of using a grid over nested lists:
    • built in one line referencing (barely a pro as this is coming in 2.3 and you can use the asset linked above to do it already)
    • built in sort (barely a pro as you can copy a bubble sort algorithm and give it a custom sort check based upon whatever internal value you want)
    • built in functions where there is a relaltionship between x/y such as mean, max, set region, set disk etc.
    The only pro for grids that I think is truly worth while is this last one. Everything else is such a loss of functionality and extra work. While I'm hardly an expert programmer I would say the strongest argument for nested lists over grids is that argument (presumably) has already been had by expert programmers and resolved in favor of nested lists - hence the JSON format (which is not GML based). Also, as someone who used grids for awhile and then switched to lists, it's just much easier.
     
    kupo15 likes this.
  11. chamaeleon

    chamaeleon Member

    Joined:
    Jun 21, 2016
    Posts:
    1,072
    Looks like you're assigning the same _map_wish map to two different map entries. Presumably you'd want to allocate two different maps. I guess maybe you have the _map_item map in mind really. However, both _map_wish and _map_item are already owned by the root list.. By using mark as map and add map you're creating a conflict in who owns it when you at some later point either delete the containing map or the root list. Whoever goes second would end up trying to delete a map that has already been deleted.
     
  12. kupo15

    kupo15 Member

    Joined:
    Jun 20, 2016
    Posts:
    882
    Oh oops, yeah. I meant to make that _map_itemlist.

    I see, so how do I create the proper structure?
    [​IMG]


    Or is this is a wrong structure? I have figured out the ends of this JSON. Creating the map_wish and map_item is easy and creating the ROOT containing a list is done as all of those were contained in the tutorial. What I have a problem wrapping my head around is how to insert that map inbetween the list the ending map. The only change between the tutorial and what I'm trying to do is inserting the Map after the list and having that contain a map of the data
     
  13. chamaeleon

    chamaeleon Member

    Joined:
    Jun 21, 2016
    Posts:
    1,072
    Given your image I have no idea why you are doing
    Code:
    ds_list_add(root_list,_map_wish );
    ds_list_mark_as_map(root_list,ds_list_size(root_list)-1);
    ds_list_add(root_list,_map_item );
    ds_list_mark_as_map(root_list,ds_list_size(root_list)-1);
    
    _map_wish_ and _map_item are not owned by root_list in your image. They are owned by _map, while _map should be owned by root_list.
     
  14. kupo15

    kupo15 Member

    Joined:
    Jun 20, 2016
    Posts:
    882
    Oh I see. Not only am I assigning those items to the wrong thing I'm also using the wrong functions as well. Those two items should be added to a map not a list. I finally figured out how to properly represent the structure I'm going for and realized my initial drawing was wrong. I need to represent the items in the itemlist grid as a list first THEN a map for the values just like I did with the wishlist
    upload_2019-12-16_20-48-57.png

    I did some more studying from your comment and all the knowledge from when you helped last time I tried to learn how to decode JSONs for texturepacker is flooding back to me. I think I figured it out how to correctly complete this. Here's my updated code, I reduced the number of entries to shorten the code...looking better?

    https://pastebin.com/wcMvpKSx

    Code:
    var _root_list = ds_list_create(); // list containing each wishlist created
    
    var wishlist_size = ds_grid_height(wishlist_grid); // get the number of wishlists created
    for(var i=0;i<wishlist_size;i++) // loop through every wishlist in the grid
        {    
        // create submap that contains "wishlist" and "itemlist"
        var _submap = ds_map_create();
     
        // create map for the wishlist
        var _map_wish = ds_map_create();
     
        // get values
        var _name = wishlist_grid[# 0,i]; // wishlist name
        var _owner = wishlist_grid[# 1,i]; // owner id
    
        // add key/values to wishlist map
        ds_map_add(_map_wish ,"name",_name); // save wishlist name
        ds_map_add(_map_wish ,"owner",_owner); // save owner id of list
     
        // add wishlist map to the submap
        ds_map_add_map(_submap,"wishlist",_map_wish);
    
        // save every item in the item list
        var item_list = ds_list_create(); // contains the itemlist
     
        var item_grid = wishlist_grid[# 2,i]; // get the pointer of item grid associated with the wishlist
        var itemlist_size = ds_grid_height(item_grid); // number of items in the grid
        for(var n=0;n<itemlist_size;n++) // loop through the grid
            {
            // create the map for the item
            var _map_item = ds_map_create();
            ds_list_add(item_list,_map_item); // add the map to the item_list
            ds_list_mark_as_map(item_list,ds_list_size(item_list)-1); // mark it as a map
     
            // get values
            var _name = item_grid [# 0,n]; // item name
            var _price = item_grid [# 1,n]; // item price
    
            // add key/values to itemlist map
            ds_map_add(_map_item ,"name",_name); // save item name
            ds_map_add(_map_item ,"price",_price); // save item price
            }
       
        // add item_list to the submap
        ds_map_add_list(_submap,"itemlist",item_list); // added to submap
    
        ds_list_add(root_list,_submap); // add the submap to the root_list
        ds_list_mark_as_map(root_list,ds_list_size(_root_list)-1); // mark as map
        }
    
    // wrap the data up
    var _wrapper = ds_map_create();
    ds_map_add_list(_wrapper,"ROOT",_root_list);
    
    // save as string
    var _string = json_encode(_wrapper);
    

    EDIT: This appears to be working. Thanks for everyone who helped guide me to figuring it out! And to complete this thread so its not half completed. Here is the json_load script for this particular example for future reference
    Code:
    var _wrapper = LoadJSONFromFile(_filename);
    var _root_list = _wrapper[? "ROOT"];
    
    var size = ds_list_size(_root_list); // number of wishlist entries
    for(var i=0;i<size;i++) // loop through the wishlists
        {
        var wishlistROOT = _root_list[| i]; // get ALL wishlist's data  
          
        // get "Wishlist" data
        var wlMap = wishlistROOT[? "wishlist"]; // decode the map
        var wl_name = wlMap[? "name"]; // get wishlist name
        var wl_owner = wlMap[? "owner"]; // get wishlist's owner
      
        // set the loaded variables
        var itemgrid_id = ds_grid_create(itemlist_header_num,0); // create itemlist grid and get the id
        var row = funct_grid_row_add(wishlist_grid); // add a row
        wishlist_grid[# 0,row] = wl_name; // add name to the grid
        wishlist_grid[# 1,row] = itemgrid_id; // add id of itemlist id to the grid      
    
        // get "Itemlist" data
        var ilList = wishlistROOT[? "itemlist"]; // decode the map
      
        var itlist_size = ds_list_size(ilList); // get the number of items in the itemlist
        for(var n=0;n<itlist_size;n++) // loop through items
            {
            var ilMap = ilList[| n]; // get all the item's data
          
            // get "Itemlist" data
            var il_name = ilMap[? "name"]; // get item's name
            var il_price = ilMap[? "price"]; // get item's price
    
            // set the loaded variables
            funct_grid_row_add(itemgrid_id); // add a row
            itemgrid_id[# 0,n] = il_name; // add item's name
            itemgrid_id[# 1,n] = il_price; // add item's price
            }
        }
    
    ds_map_destroy(_wrapper);
    
     
    Last edited: Dec 17, 2019
  15. kupo15

    kupo15 Member

    Joined:
    Jun 20, 2016
    Posts:
    882
    I know this topic has been solved, but I wanted to comment and say...I think I finally understand why you prefer nested lists over grids especially with your comment about referencing pointers of lists to easily move things around. Currently, my friendslist is held in a grid because it seemed easier laid out that way like an excel spreadsheet. I also created Groups in which you can add friends too which currently works as well through grids as well but in order to do that I essentially have to copy the row in the friendslist and paste a duplicate of it in the group. If I delete the contact, I have to loop through all the groups and compare to the friendslist grid to see if that friend name is still there otherwise delete it from the grids. I can see how this is somewhat convoluted and cumbersome the more different features require referencing the same friends or data. It would require more and more copies of it to work which sounds incredibly messy and unwieldy the bigger it becomes.

    So what you are saying is the way to do this much better is with lists nested inside lists all the time. What I would do is create a MASTER friend's list which holds pointers to individual lists that make up each friend (name, id, age etc....). Each Group is its own list that contains pointers to member list IDs as well. If I want to add a friend to a group, all I have to do is simply create a list for the group and add the pointer to the friend's list into it. This way, if for example, the friend decides to change their name, it only needs to be updated in that individual member list. Then all other lists (friends list, groups list) are simply referencing the pointer to that list all automatically update. With the grid method, once the name is changed I would have to loop through ALL groups and grids all members are on, compare if their name, age etc... has changed and if it has, overwrite it in all those grids.

    Yeah, lists nested in lists are much more versatile now that you put it that way simply because you can reference and swap out pointers to individual lists which is the same as swapping rows/columns. Do I understand your point about this correctly?
     
  16. samspade

    samspade Member

    Joined:
    Feb 26, 2017
    Posts:
    2,191
    Yes. Although I would also think about when it is best to use a map and when to use a list. For example, a 'friend' in this case would probably make more sense as a map than a list. If I understood you correctly, I would have a map of all people, where the key is the person's name and the data is itself a map. That map would hold all of the person's data. One element of that data would be a list of friends. The list of friends would be an actual list, containing the people's names.

    For example, writing this out in the normal JSON structure would give you something like this:

    Code:
    
    {
    
      "John": {
        "Friends": [
          "Mary",
          "Sue"
        ]
      },
    
      "Mary": {
        "Friends": [
          "John"
        ]
      },
    
      "Sue": {
        "Friends": [
          "John",
          "James"
        ]
      },
    
      "James": {
        "Friends": [
          "Sue"
        ]
      }
    
    }
    
    
    Note the main level is a map where John, Mary, Sue, and James are also maps, holding a list to their friends. If you wanted to visualize this in another way, copy this code into an online json editor such as: https://jsoneditoronline.org/ (which it looks like you're familiar with)
     
  17. kupo15

    kupo15 Member

    Joined:
    Jun 20, 2016
    Posts:
    882
    Hmm, well my post was strictly referring to operations within the app itself and how I realized how lists are more flexible to store data instead of grids, for example, when creating and displaying groups in which a member is part of several. I wasn't referring to saving things in JSON because I figured out a structure how to do that:
    https://jsoneditoronline.org/?id=7863e1c7180b40dfa7655fc57a069d3b

    Note: the above is currently with my grids system where on load I actually loop through each group and compare against the friends list. I think I need to figure out a different way to store the group data if I'm going to be creating groups as a list full of pointers to other lists instead of each group containing copies of user data

    Also, when it comes down to a single users app experience, is it really necessary to store the connections of all your friends and their friends? I think its only necessary and practical to simply save who your friends are and what groups you created they are part of, right?

    But what you essentially was saying that lists in lists are better than grids because they are more flexible because you have pointers of each "row". In the context of my app where you have a list of friends and their details (username, user id etc..) in which they can become members of multiple groups, its better to build those groups as a list of pointers to a member's lists than to make a bunch of copies of those friends in each of those groups. With grids, you are basically forced to make hard copies every time you create a new group to assign them, whereas with lists, you can simply add their pointer to any number of groups you want and only one hardcopy is needed

    For clarification, the way I have these groups and lists set up in the json file is not how they are built in the app. They've mostly been converted from grids to lists for the JSON. I definitely need to convert all the grids into lists to make it easier to handle in the app
     
    Last edited: Jan 13, 2020
  18. kupo15

    kupo15 Member

    Joined:
    Jun 20, 2016
    Posts:
    882
    Can we and is it practical to use this for memory management outside of JSON saving/loading as the help file says otherwise?

    upload_2020-1-14_0-12-12.png

    For example, if I'm creating a:
    -list of wishlists that nests
    -an itemlist list with pointers to items that nests
    -a item list which nests
    -a list that contains urls

    as I'm creating these lists in and nesting lists inside of lists, would it be practical to create a separate ds list called list_wrapper in which it basically is a copy the above list but it marks each list as a list (ds_list_mark_as_list). This way if I wanted to just delete an item within the itemlist, I simply delete the that list and it auto destroys the nested list of urls. If I wanted to destroy the whole wishlist, I simply destroy the wishlist and it auto destroys everything below it?

    I do this for JSON stuff and its much more intuitive and easier since we are creating and destroying a temporary nested ds_structure, its a little more involved when dealing with data structures that are persistent and that are constantly referenced. I'm really enjoying transitioning my ds_grids into nested lists, however, I have to triple check that when I delete a ds_list that I'm also deleting any lists nested inside of it as well to avoid a memory leak which is a bit of a pain.
     

Share This Page

  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice