Legacy GM Detect if ds_map value is a ds_map

B

Big_Macca_101

Guest
Hi guys, I'm trying to make a way to detect if a ds_map's value is actually a ds_list or ds_map as I use JSON a lot im just trying to work out a way to simply detect it.

I thought ds_exists would work but it returned true if the value was a map or not :/

Hopefully this was able to describe what I mean!
 

Tthecreator

Your Creator!
This is a problem with the way game maker stores ds_maps in variables.
It work by using id's.
The first ds_map you create will have id 0.
The second ds_map you create will have id 1....
etc..........
Now your problem is that these id's which are just simple numbers starting from 0 and counting are also stored in a variable.
For example you could have the code:
Code:
map1=ds_map_create()
map2=ds_map_create()
show_message(map1+map2)
This code is completely valid.
And if map1 would get an id of 0 and map2 and id of 1, show message would return you 1 since 0+1=1. the variables associated with maps are also normal numbers.


How to get around this?
You could have another ds_list containing whether something is a map or not.
Or you could reserve some information inside the map:
What i mean by that is that if you aren't using negative numbers for example, you can use the "negativity" to mark ds_maps and other structures.
You could also look into binary shift functions. Just type "<<" and ">>" into the manual.
 
B

Big_Macca_101

Guest
Damn, sadly not what i was after, hopefully a way to do so is added in the future!
 

JimmyBG

Member
There is a way, for some unknown reason
Code:
ds_exists(my_var,ds_type_map)
Always returns true whether its a list or a map

However,
Code:
ds_exists(my_var,ds_type_list)
Works correctly and will return 0 for a map and 1 for a list
 
B

Big_Macca_101

Guest
That is a step in the right direction but that means it will treat everything as either a list or a map even its just a normal value such as map[? "random"] = "Random string", that would return either a map or list.
 

JimmyBG

Member
Righto, I've a come up with some code that works, and also worked out which order the checks must go in to ensure it gives the correct results. ds_exists has major issues...
Code:
//create a map
map1 = ds_map_create();

//list to add
var tmp_list = ds_list_create();
ds_list_add(tmp_list,"Asd");

//map to add
var tmp_map = ds_map_create();
ds_map_add(tmp_map,"first",22);

//add to main map
ds_map_add_list(map1,"mylist",tmp_list);
ds_map_add_map(map1,"mymap",tmp_map);
ds_map_add(map1,"myvar","random string");


//find what type is in each key
var key = ds_map_find_first(map1);
for(i = 0; i < ds_map_size(map1); i++) {
    if is_string(map1[? key]) { //check if string
        show_debug_message(key + " is a string");
    }
    else {
        show_debug_message(key + " is NOT a string")
        if ds_exists(map1[? key],ds_type_list) { //check if list
            show_debug_message(key + " is a list");
        }
        else {
            show_debug_message(key + " is NOT a list");
            if ds_exists(map1[? key],ds_type_map) { //check if map
                show_debug_message(key + " is a map");
            }
            else { //must be a variable or other ds structure
                show_debug_message(key + " is NOT a map");
                show_debug_message(key + " is a variable");
            }
        }
    }
    key = ds_map_find_next(map1,key);
}
String must go first as it will also return true as a list
List goes second because they will return true as a map but maps won't return true as a list
Maps go third and whatever fails that is a variable of some other kind
 

YellowAfterlife

ᴏɴʟɪɴᴇ ᴍᴜʟᴛɪᴘʟᴀʏᴇʀ
Forum Staff
Moderator
Righto, I've a come up with some code that works, and also worked out which order the checks must go in to ensure it gives the correct results. ds_exists has major issues...
Code:
//create a map
map1 = ds_map_create();

//list to add
var tmp_list = ds_list_create();
ds_list_add(tmp_list,"Asd");

//map to add
var tmp_map = ds_map_create();
ds_map_add(tmp_map,"first",22);

//add to main map
ds_map_add_list(map1,"mylist",tmp_list);
ds_map_add_map(map1,"mymap",tmp_map);
ds_map_add(map1,"myvar","random string");


//find what type is in each key
var key = ds_map_find_first(map1);
for(i = 0; i < ds_map_size(map1); i++) {
    if is_string(map1[? key]) { //check if string
        show_debug_message(key + " is a string");
    }
    else {
        show_debug_message(key + " is NOT a string")
        if ds_exists(map1[? key],ds_type_list) { //check if list
            show_debug_message(key + " is a list");
        }
        else {
            show_debug_message(key + " is NOT a list");
            if ds_exists(map1[? key],ds_type_map) { //check if map
                show_debug_message(key + " is a map");
            }
            else { //must be a variable or other ds structure
                show_debug_message(key + " is NOT a map");
                show_debug_message(key + " is a variable");
            }
        }
    }
    key = ds_map_find_next(map1,key);
}
String must go first as it will also return true as a list
List goes second because they will return true as a map but maps won't return true as a list
Maps go third and whatever fails that is a variable of some other kind
This is not a reliable method and it will only work in a blank project.
Consider the following:
Code:
var some_list = ds_list_create();
var json = json_decode('{ "map": { } }');
show_debug_message(ds_exists(json[?"map"], ds_type_list));
With shown approach this would detect "map" as a list, because map and list indexes are independent of each other (there can be a map #0 and a list #0).

Without ability to access internal data, the only way to detect maps in decoded JSON is implementing a custom decoder (there are a few on marketplace, if I'm not mistaken).[/code]
 

JimmyBG

Member
Ah I see, just tested it. Lists and maps return true as both a list and a map :/

Shame its not setup to decode to the right type properly, is there any reason for this?
 
The reason is because the returned IDs for all data structures are just numbers, and they all start at index 0. So say you had a bunch of fruits, apples, oranges, and bananas. Each type of fruit is given a number to ID it, but only a number. Say then, I come up to you and hand you a ticket with the number '0' written on it, cause I was asked to come pick up the item on the ticket. If I don't know what I'm picking up, how would you know which fruit to give me, based on just the number? Sure, you could look at the fruits, but if you had all three types of fruits before you, you'd have three different fruits all with the number '0' attached to them.

Unfortunately, until Studio starts utilizing pointers for dynamic resources, there's not going to be a way to determine these kinds of things, unless you program it yourself to keep track. In keeping with the fruit analogy, if the tickets were color coded, red for apple, orange for orange, and yellow for banana, then there'd be no problem.
 

TheouAegis

Member
Are you using ds_map_add_map or just ds_map_addI? I held off from answering last night because I assumed you were using ds_map_add_map.

Regardless, the key in the map can be a string, so you could always just set the key to a string like "map"+string(map_id) where map_id is the index of the map you're saving. Same with lists.

You could set aside particular key ranges for maps and lists. So maps would take up keys 0 through 19 and lists would take up 20 through 39. In that way, you can use a loop to loop through the whole map retrieving values and know right away if they're maps or lists based on which iteration you're on.
 

YellowAfterlife

ᴏɴʟɪɴᴇ ᴍᴜʟᴛɪᴘʟᴀʏᴇʀ
Forum Staff
Moderator
(there are a few on marketplace, if I'm not mistaken).
Correction: I was thinking XML - there aren't any assets that deal with JSON loading.
However, with some effort I was able to implement a custom set of scripts to handle JSON loading together while attaching type information. Therefore, with this, it is possible to check what type any of decoded values is.

Web demo: https://dl.dropboxusercontent.com/u/3594143/yal.cc/16/08/xm/tjson/index.html

If all is well (does not fail on any JSON snippets), I can put it up on marketplace shortly.
 

YellowAfterlife

ᴏɴʟɪɴᴇ ᴍᴜʟᴛɪᴘʟᴀʏᴇʀ
Forum Staff
Moderator
My JSOnion library has built-in support for this.
Oh, I think I've even seen that before. That's nice.
Doing some comparisons on large snippets, it seems like GM8 compatibility takes it's toll on JSO performance though.

Measuring via the following,
http://pastebin.com/5nqXzP1G
(real-life use case - Nuclear Throne's save file)

Results on VM are:
Code:
json_decode: 1497
JSOnion: 643564
TJSON: 10531
Kind of not grand, wouldn't you agree? A 643ms stall is visible to a naked eye.
Though, admittedly, my initial implementation would take 320ms, which isn't much better.

YYC, as per usual, is helpful:
Code:
json_decode: 1413
JSOnion: 595871
TJSON: 2757
While for JSO this wins some 10% performance, for my implementation this brings up performance to half of the speed of the built-in function, which is not something you see often.
I have looked into further optimizations, but it doesn't seem like it's possible to speed this up further without doing something particularly strange.

Overall, I think this'll have it's uses.
 

FrostyCat

Member
Doing some comparisons on large snippets, it seems like GM8 compatibility takes it's toll on JSO performance though.
It takes a huge toll and I know it, some of it comes from GM8's limitations and some come from JSOnion's focus on correctness over speed.
  • JSOnion has an entire script for rectifying what string() and string_format() do poorly or can't do at all (e.g. trimming 1.000000000000 to 1, scientific notation for very large or very small numbers, etc.). Combining clean output with correctness comes at a price.
  • JSOnion checks for errors at every turn at the character level and doesn't expect correct input. This rules out a large number of optimizations that can arise from assuming correct input.
  • GM8 doesn't have buffers. With buffers, I can repeat string concatenations at over 120x the speed of traditional concatenations (source).
  • GM8 can't pass arrays. After decoding a particular sub-element, I have to return its content in full fidelity as well as its ending character position. The only way to do it at the time without creating data structures is with GMTuple, which uses the same rectifying routine plus a lot of string operations.
Now that you've reminded me to look back on it, there are some things that I can do to reduce the amount of string concatenations. Fortunately I have a full test suite on JSOnion, so I can safely experiment with optimizations and refactors all I want.
 

YellowAfterlife

ᴏɴʟɪɴᴇ ᴍᴜʟᴛɪᴘʟᴀʏᴇʀ
Forum Staff
Moderator
It takes a huge toll and I know it, some of it comes from GM8's limitations and some come from JSOnion's focus on correctness over speed.
  • JSOnion has an entire script for rectifying what string() and string_format() do poorly or can't do at all (e.g. trimming 1.000000000000 to 1, scientific notation for very large or very small numbers, etc.). Combining clean output with correctness comes at a price.
  • JSOnion checks for errors at every turn at the character level and doesn't expect correct input. This rules out a large number of optimizations that can arise from assuming correct input.
  • GM8 doesn't have buffers. With buffers, I can repeat string concatenations at over 120x the speed of traditional concatenations (source).
  • GM8 can't pass arrays. After decoding a particular sub-element, I have to return its content in full fidelity as well as its ending character position. The only way to do it at the time without creating data structures is with GMTuple, which uses the same rectifying routine plus a lot of string operations.
Now that you've reminded me to look back on it, there are some things that I can do to reduce the amount of string concatenations. Fortunately I have a full test suite on JSOnion, so I can safely experiment with optimizations and refactors all I want.
- Encoding with exponent notation support is an interesting subject. GMS itself does not make use exponent notation in json_encode, and there's generally little use of extraordinarily small or large numbers. I currently have this store up to 15 decimal spaces when encoding.
- Checking for errors barely costs anything from my experience. Not checking for errors would have made this particular thing a little useless, since a lot of the point is dealing with possibly-wrong input.
- Using a buffer for string building is standard practice. If you understand how UTF-8 encoding works, storing the source string in a buffer also helps, and permits for faster substring copying via buffer_copy.
- Using ds_grids as containers would probably have been the nearest possible way in GM8. Probably still not too fast though. Maybe using GMAPI so that you could write the entire decoder in C++ while allocating GM data structures.
- I've currently ported a handful of unit tests from this suite. Doesn't have too many samples, but covers edge cases. Also found some GMS bugs while doing so (1, 2).

Updated the demo: HTML5; Windows YYC.
- Since buffers aren't too fast on HTML5, that version now uses a browser-provided parser where appropriate.
- Native version is now a tiny bit faster and downloadable. As far as I've tested, there are no cases on which it would crash or incorrectly decode something.
- Demo in general now looks and feels better (scrollbar is now usable; nested structures are indented as one could expect).
I should probably make a topic for the thing instead of bumping this one.
 
Top