1. Hey! Guest! The 35th GMC Jam will take place between November 28th, 12:00 UTC - December 2nd, 12:00 UTC. Why not join in! Click here to find out more!
    Dismiss Notice

GML Utility Array Functions (now available)

Discussion in 'Programming' started by samspade, Nov 9, 2019.

  1. samspade

    samspade Member

    Feb 26, 2017
    There are a number of utility array functions I use and I've recently been taking some javascript courses so I wanted to formalize and add to those functions and make it a free to use utility package. I'll upload it to the marketplace at some point.

    If anyone wants to use them feel free, but right now I'd love feedback on what is there. Specifically:
    • Any missing functions that should be added?
    • Have I made any mistakes?
    • Are there better ways to do any of these?
    The current list of scripts are:

    • array_swap
    • array_add
    • array_insert
    • array_delete
    • array_splice
    • array_find_value
    • array_shuffle
    • array_sort
    • array_copy_shallow
    • array_copy_deep
    • array_combine
    • array_filter
    • array_map
    • array_for_each
    • array_flatten
    • array_accumulate
    • array_to_list
    • list_to_array

    A couple general notes. I haven't tried for optimization yet. I think my ultimate goal will be to have a set of scripts that are a little more verbose, extra assignments, lines and so on, for ease of understanding and debugging, and then make an extension that as the most compact versions of them.

    There's also a couple scripts I made that I'm not sure are worth having. For example, splice could be easily accomplished in two lines with delete then insert and that would probably be more understandable. Also, map and for_each will be significantly less optimized and require you to write special callback scripts which doesn't really seem worthwhile for the gain. So I'm tempted to remove those three scripts.

    I also might want to add an array_to_list and list_to_array deep copy.

    Also, if I did my research right, there's no way to make an existing array shorter is there? Seems like the only option is to create a new shorter array and reassign.

    You can view the repository here.

    Any and all feedback welcome here or directly to the repository.
    KPJ and FrostyCat like this.
  2. CloseRange

    CloseRange Member

    Jul 2, 2016
    These are vary useful but I will say there does come a point where it's just better to create your own data structure.
    Or for the individual user make one based on your needs. At least if you care about optimization.

    Other than that you might want to consider some debugging functions.
    for instance a deep/shallow log of all entries
    log the index of all entries that are equal to some input
    log the total size of the array (adding all sub arrays as well)

    maybe create a safegard functon for setting and getting entries.
    for instance:
    array[3] = 0;
    var temp = array[5];
    will give me an error for going out of bounds but:
    array[3] = 0;
    var temp = array_get(array, 5);
    would return noone or 0 and just logs the error so the game still runs

    Also a custom function sort would be nice as well.
    Let's say me, the user of these functions made this script:
    /// myCustomSort(a, b);
    var a = argument[0];
    var b = argument[1];
    return a+5 < b;
    I could pass this into one of your functions:
    array_custom_sort(array, myCustomSort);
    so the user creates a function that takes in 2 inputs (2 values to compare) and returns true or false based on what value should come first.

    I also noticed you didn't have a replace function (unless that's what you mean by swap?)
    Honestly some of these functions are probably what I already described but I'm not super used to javascripts array utility functions because I my own structures more than I use arrays, usually.
    Last edited: Nov 9, 2019
  3. FrostyCat

    FrostyCat Member

    Jun 26, 2016
    I'd love to see and regularly use a library like this if its variable scoping is up to scratch.

    Some pointers:
    • If array_find_value() is for finding the position of a value in an array, I would rename it to array_find_index() to be consistent with the naming of list functions.
    • It would be nice if there is a strict and a non-strict version of array_find_value(). The strict version would use == (which for arrays will be true only if it's the exact same array, not just two different arrays with the same content), while the non-strict version would multiplex between == for non-array values and array_equals() for array values.
    • array_sort() should optionally take an argument for which direction to sort and/or a callback for how to compare two adjacent values.
    • It would be nice if there is an array_equals_deep() function that checks the value equivalence of two entire nested array structures.
    • array_to_list() should be more clear as to whether only its topmost layer will be converted to lists, or if it recursively converts all sublayers to lists as well. In the same vein, there is currently no way of telling an integer in a list from a list nested in a list, so you'll need a disclaimer too or somehow work around that.
    • There should be an in-place version of array_for_each() and/or array_map().
    • The library would be easier to import if it's in extension form rather than a collection of scripts.
    • The library should be tested by automated scripts instead of visual inspections using print statements. At the very least, use array_equals() to help with that.
    As for callbacks and extra assignments, I'd say your concern about them will be pointless given the upcoming GML 2020 updates. Anonymous functions and named parameters will both become norms soon, and this library should be built facing forward. I wouldn't find it useful to bend over backwards to accommodate legacy dinosaurs at this point.
    KPJ likes this.
  4. samspade

    samspade Member

    Feb 26, 2017
    Thank you both for the inputs. Those are helpful.


    Both the debugger and show_debug_message already do this, so I didn't feel it was necessary.

    This is probably worth adding.

    This is already in there in the form of flatten and array_length_1d although a custom function would be more optimized.

    This might be worth while. As a note, GML already has an array_get function. It will throw an error if you try to access outside of an array. I go back and forth on this. Personally, 9 out of 10 times I'd probably rather have the error. But a safe function might be nice for those times where I don't.

    I'm not sure how far you looked into it, but the sort function is already custom. It takes two arguments, the array and evaluation script I have a couple basic ones included, but you could write and pass any that you wanted.

    I didn't create functions that are easier to do in code so for example:

    //array destroy
    array = -1;
    //array clear
    array = [];
    //array replace
    array[index] = value;
    array_set(array, value); //this one is a built in GML function 
    As a side note, I don't know how to do array_clear in a script because while you can use the @ accessor to prevent the copy upon write behavior when writing to an array inside a script, if you do:

    argument0 = [];
    That copies the array instead of modifying the existing one.


    I thought I had, but I always get these mixed up, so I'll fix that.

    Will add it to my list.

    I think it already does this if I understand what you mean.

    Will add it to my list.

    Is says it in the function description, but it would probably be good to rename both versions and make deep copy versions as well.

    This one I don't understand. What do you mean?

    That's the plan.

    On my list of general programming goals is to learn how to implement actual testing but I'll start with the array_equals as I understand that currently.
  5. FrostyCat

    FrostyCat Member

    Jun 26, 2016
    I mean versions of these scripts that actually work on the given array instead of returning a modified copy. Example:
    /// @function array_map_overwrite(array, script, ...script_arguments)
    /// @param {array} array
    /// @param {scripts} script_id
    /// @param {array} arguments
    /// @description Performs the script on each element of the array and sets each to the script's return value. Note - returns the array given.
    //rename arguments for ease of use
    var _array, _script, _script_argument_array;
    _array = argument0;
    _script = argument1;
    _script_argument_array = argument2;
    //loop through given array and execute the script on each element
    for (var i = 0; i < array_length_1d(_array); i++) {
       _array[@ i] = script_execute(_script, _array[i], _script_argument_array);
    //return given array
    return _array;
  6. samspade

    samspade Member

    Feb 26, 2017
    Repository updated.

    • array_valid_index - takes an array and index and checks to see whether it is a valid index for the array. I think I tested all of the possibilities.
    • array_get_safe and array_set_safe - only allow getting and setting an array if the index is valid. I might remove these functions and just leave the array_valid_index. In general, I think this is the type of error you want to immediately crash your game rather than live on unnoticed. And these functions are very easy to write so people could add them.
    • array_find_index_all - returns an array of all indexes that contain the value
    • array_find_index_non_strict - can find equivalent arrays. Does not do deep searches.
    • array_equals_deep - same as array equals but works on nested arrays
    • array_to_list_deep - same as the shallow version, but works on nested arrays.
    I attempted to write a list_to_array_deep however it is either not possible in GML or I'm missing something. The issue comes in testing whether a a list contains another list. I'm not sure how you can do this as lists are just numbers so I can test a number against available lists but there isn't anyway that I know of to tell if it is supposed to be a number or a list. Maybe there's away that I didn't think of (e.g. parse a JSON file into an array?).

    @FrostyCat in the prior version, for each operated directly on the array and map operated on a copy, but that was a little hidden by the way I structured the callback scripts. I modified them and renamed array_map to array_for_each_copy to the way you suggested. I think it is a little cleaner this way but I haven't written any complicated callback scripts.
    Last edited: Nov 10, 2019 at 6:27 PM
  7. YellowAfterlife

    YellowAfterlife ᴏɴʟɪɴᴇ ᴍᴜʟᴛɪᴘʟᴀʏᴇʀ Forum Staff Moderator

    Apr 21, 2016
    A handful of these could be making use of array_copy/array_create/helper structures - take a look at my implementation of Haxe's standard array library [that compiles to GML] for some hints.
    samspade likes this.
  8. samspade

    samspade Member

    Feb 26, 2017
    I added GMAssert to the project and added some automated testing. Fixed one problem. I'll work on making it an extension next and then probably upload it to the marketplace for ease of use.
  9. Gavolot

    Gavolot Member

    Aug 23, 2019
    I would be glad if someone had decided for me earlier all possible ways of sorting arrays, dynamic lists, and two-dimensional arrays.
  10. samspade

    samspade Member

    Feb 26, 2017
    I looked through all the functions, and, while I don't understand Haxe, I'm not exactly sure where I could add in the gml array_copy and array_create functions that I'm not already using them. Let me know if I'm missing them. However, it did help me do a couple things better. For example, array_insert now operates on the array instead of creating a copy.

    I'm not exactly sure what this means, but it does include a customizable sort. I don't want to deal with two-dimensional arrays for a couple reasons. The biggest reason is that GML's array handling is going to change soon. The second being that most of the functions I'm trying to create don't have an equivalent in the two-dimensional version (what does filter or for each mean in a grid?) and to the extent that there are some useful things you want to do with grids there are ds_grids and GMLineaer already.


    I've turned it into an extension. Everything still seems to work. For the moment, I'm leaving the scripts in this project, prefixed now with src_.

    I'm making another pass through for consistency and here are my questions:

    Array Extension Questions:
    • I'm planning on removing array_get_safe and array_set_safe but leaving array_valid_index. I think they do more harm than good and in general conflict with most other GML assumptions (like errors crash your game).
    • Should array_valid_index accepts true and false? This seems like it should stay that way given how the rest of GML accepts true and false for 0 and 1.
    • Is it worth having both array_copy_shallow and array_copy_deep?
    • For scripts that accept variable arguments, it seems like you must assign the array to a variable in order to use it?
    • There's a bunch of find index type scripts for example, some, every, last_index I could include a find_index_all_non_strict. I'm not sure how many of these are worthwhile and how many are just bloat.
    • one big question I have is how to handle the callback scripts. Is my current method the best way of doing so?

    Extension specific questions:
    • It seems like pure GML extensions should be compatible with all systems by default?
    • What does the Extra Platforms mean in Extensions?


    Here is what I am going to consider the final list of scripts to be included in the extension and asset:

    - array_valid_index
    - array_swap_positions
    - array_copy_shallow
    - array_copy_deep
    - array_add_to_end
    - array_insert
    - array_delete
    - array_splice
    - array_join
    - array_combine_strings
    - array_to_list_shallow
    - array_to_list_deep
    - list_to_array_shallow
    - array_equals_deep
    - array_flatten
    - array_find_index
    - array_find_index_custom
    - array_find_index_last
    - array_find_index_last_custom
    - array_find_index_all
    - array_find_index_all_custom
    - array_every
    - array_some
    - array_filter
    - array_reduce
    - array_sort
    - array_shuffle
    - array_reverse
    - array_for_each_copy
    - array_for_each

    - callback_template
    - sort_template

    - sort_alphabetical_safe
    - sort_ascending
    - sort_ascending

    I will work on the actual asset later today and tomorrow. The market place version will only include the above for simplicity sake, though the github repository will include the testing suit and demo functions and script versions.
    Last edited: Nov 16, 2019 at 6:13 PM
  11. samspade

    samspade Member

    Feb 26, 2017
    It's up on the marketplace now. If you find any bugs let me know on the asset page or on github. You can asset page here.
  12. YellowAfterlife

    YellowAfterlife ᴏɴʟɪɴᴇ ᴍᴜʟᴛɪᴘʟᴀʏᴇʀ Forum Staff Moderator

    Apr 21, 2016
    array_delete can use array_copy to join two halves of new array.
    array_filter can make use of a ds_list to collect elements before allocating an array for them.
    there was something else when I was looking at it

Share This Page

  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice