GML Reusable Data Unit Proposal

[GMWolf]

Something that has always been a bit tricky in Gamemaker is managing data and their types.
This problem is most apparent when attempting to use multiple libraries together.

This is why I suggest we, the community, come up with a standard way of storing data.
A Reusable Data Unit (RDU).

What is the point?
An RDU borrows ideas from Java Beans, and aims to achieve many of the same goals.

It would allow libraries to reflectively inspect the type of data given to it, and access its fields, without it having been designed for that piece of data in particular.
For example, a library could be written to save and load RDU's, and have it work on any kind of data. This would be achievable because even though the library would not know about the type of data before hand, it could find out enough about it at run time.

Another advantage of using such RDU's would be their reusability. An RDU returned from one library could easily be used by another.
For instance, you could imagine having two libraries: one would request data from a web server, whilst the other would display arbitrary data on screen.
Ordinarily, you would have to convert the data returned from the web server into a form the display library can understand.
If both libraries are "RDU compliant", the data could be passed directly from one library to the other.

Finally, Having a standard way of defining interfaces for data means that code easier to read and understand, as libraries can rely on shared preconceived notions.

Naturally, such RDU's make most sense when used with libraries that are designed to work with arbitrary data. That is not to say libraries that work on specific data could not benefit from RDU's either; being able to return RDU's makes them more compatible with other libraries.

Here is my first proposal:

Requirements:

1. An RDU must be able to hold data in fields.
2. An RDU must be able to hold other RDUs.
3. An RDU must expose each field through a standard mechanism.
4. An RDU field must be typed.
5. An RDU must be constructable with no parameters
6. Agnostic to representation of RDU data
7. An RDU must not hold transient data.
8. An RDU must be typed.
9. An RDU's representation must be inherently unique. (instance id, array...)
10. An RDU must be distinguishable from non RDU data.
11. Non RDU data must be distiguishable from RDU data.
12. Virtual Destruction of RDU's through a single function.
13. Avoid or report clashing of RDU types.

What an RDU does not need to be the fastest, or most memory efficient way to store data.


Proposal:

• Interface to an RDU made of constructor, destructor and getter-setter scripts.
  Getter-Setter scripts both get and set data, based on argument count. (reduces number of scripts in resource tree)
• Interface to RDU defined using script prefixed with "rdu_". Script index is used as type index of rdu. (this avoids naming clashes, as GM will complain if scripts clash).
• Constructor must call a register script to register the RDU type with the framework

Proposed implementation:

RDU script

Spoiler

gml_pragma("global", "RDU()");

global._rdu_types = ds_map_create();

global._rdu_constructor_table = ds_map_create();
global._rdu_destructor_table = ds_map_create();
global._rdu_field_table = ds_map_create();

global._rdu_rdus = ds_list_create();

global._rdu_id = -1;

#macro rdu_constructor global._rdu_constructor_table[? global._rdu_id]
#macro rdu_destructor global._rdu_destructor_table[? global._rdu_id]
#macro rdu_fields global._rdu_field_table[? global._rdu_id]

//Register all RDU's

for(var script = 0; script_exists(script); script++) {
    var s_name = script_get_name(script);
 
    if (string_copy(s_name, 1, 4) == "rdu_") {
        var rdu_type = script;
        var rdu_name = string_delete(s_name, 1, 4);
  
        ds_list_add(global._rdu_rdus, rdu_type);
        script_execute(script);
    }
}

r script

Spoiler

gml_pragma("forceinline");
global._rdu_types[? argument0] = argument1;
return argument0;

destroy script

Spoiler

var type = rdu_typeof(argument0);

var destructor = global._rdu_destructor_table[? type];

script_execute(destructor);
ds_map_delete(global._rdu_types, argument0);

rdu_typeof script

Spoiler

return global._rdu_types[? argument0];

Example Usage
rdu_vector script

rdu_constructor = vector;
rdu_destructor = vector_destroy;

rdu_fields = [ ["x", vector_x],
               ["y", vector_y] ];

This script defines the vector RUD type

vector script

var v = //whatever you like. An array would be nice, could be an instance, etc
return r(v, rdu_vector);

The vector constructor script. r() must be called to register the type of the RUD

vector_destroy, vector_x and vector_y can be anything you like, depending on how you chose to define your vector.

Things that still need working out:

• A mechanism to get RDU field names from RDU type
• A mechanism to get RDU field by name.
• A typing mechanism for RDU fields

Thankfully the first two are not super hard to figure out. (I just didn't want to have too much code here before we agree on a format, or even if this is really needed).

So what do you guys think? Is this something GM needs?
Would you change anything to the requirements, proposal or requirements?
Do you have a proposal of your own?
Lets discuss!

excuse any mistakes i may have made, I may well have made this thread a little hastily...

 

[FrostyCat]

I would rather have the type definitions in nested array form:

[
  [type_name, constructor, destructor, serializer, deserializer],
  [field, type, getset],
  [field, type, getset],
  ...
]

That way it can be returned by a utility script that organizes the specifications in a visual manner:

///rdu_v3d();
return RduClass("Vector3d",
  "constructor", v3d_constructor,
  "destructor", v3d_destructor,
  "serializer", v3d_serialize,
  "deserializer", v3d_deserialize,
  "x", "real", v3d_gs_x,
  "y", "real", v3d_gs_y,
  "z", "real", v3d_gs_z
);

Note that I have also added the possibility of a special "serializer" and "deserializer" script to help with cases where the serialized result may need further handling, e.g. transforming script IDs to script names for portability between projects or flattening maps/lists..

 

[GMWolf]

I would rather have the type definitions in nested array form:

[
  [type_name, constructor, destructor, serializer, deserializer],
  [field, type, getset],
  [field, type, getset],
  ...
]

That way it can be returned by a utility script that organizes the specifications in a visual manner:

///rdu_v3d();
return RduClass("Vector3d",
  "constructor", v3d_constructor,
  "destructor", v3d_destructor,
  "serializer", v3d_serialize,
  "deserializer", v3d_deserialize,
  "x", "real", v3d_gs_x,
  "y", "real", v3d_gs_y,
  "z", "real", v3d_gs_z
);

Note that I have also added the possibility of a special "serializer" and "deserializer" script to help with cases where the serialized result may need further handling, e.g. transforming script IDs to script names for portability between projects or flattening maps/lists..

I like that idea. It is neater than using the macros i suppose.

and rdu_v3d can still be called through reflection too.

I do think the serializer scripts should be optional, as basic POD types can easily be automatically serialized.

 

[FrostyCat]

I like that idea. It is neater than using the macros i suppose.

and rdu_v3d can still be called through reflection too.

I do think the serializer scripts should be optional, as basic POD types can easily be automatically serialized.

I think it's optional too, but we should make sure the basic types have enough coverage. Having a constructor just makes the defaults easier to leave out and adds potential for further expansion later (e.g. parents, interfaces, etc).

 

[Nocturne]

Having typed data who's fields can easily be accessed through reflection would make a multitude of libraries far easier to use. For instance conversion of RDU's to and from JSON could be done entirely reflectively (think newtonSoft json), making the use of such files far easier.

I'd love to give an opinion but I didn't understand a single word of that.

Before making suggestions like this, consider that a large portion of the user base have only ever programmed in GML or only have a basic understanding of other languages, and as such this may mean little or nothing to them. Try explaining the terms you use and maybe explain why this is so important or such a good idea, given that most people are making games quite happily without it and may just brush the idea off as not applicable to them. If you want community support, you have to cater to all parts of said community and not only the few that have a wide understanding of computer sciences.

 

[GMWolf]

I'd love to give an opinion but I didn't understand a single word of that.

Before making suggestions like this, consider that a large portion of the user base have only ever programmed in GML or only have a basic understanding of other languages, and as such this may mean little or nothing to them. Try explaining the terms you use and maybe explain why this is so important or such a good idea, given that most people are making games quite happily without it and may just brush the idea off as not applicable to them. If you want community support, you have to cater to all parts of said community and not only the few that have a wide understanding of computer sciences.

eehh... I guess i did use a lot of jargon...
you are right, Ill try to make it clearer later... perhaps give more examples too

 

[Nocturne]

you are right, Ill try to make it clearer later... perhaps give more examples too

Cheers! I'd appreciate that!

 

[Cloaked Games]

Also see if you can find some articles or something for people who may want to research the sort of thing you're talking about. I'd read a few of those.

 

[GMWolf]

Cheers! I'd appreciate that!

I think I managed to clarify a lot of it, and gave more concrete examples. What do you think?

Also see if you can find some articles or something for people who may want to research the sort of thing you're talking about. I'd read a few of those.

I would suggest reading about JavaBeans online. They are the closest thing I know to what I want RDU's to be like.
(For instance, look at their use with hibernate, a java database library).
(on second thought hibernate is not the best place to look because of complexity...)

I would also look at reflection, as this is a common way these sorts of data units are accessed.

 

[Cloaked Games]

Yes, that is a good explanation. My only question is in the case of GM, what types of arbitrary data are you suggesting could be used this way. Are you talking about the difference between strings and reals, or like, instances and data structures, etc?

 

[GMWolf]

Yes, that is a good explanation. My only question is in the case of GM, what types of arbitrary data are you suggesting could be used this way. Are you talking about the difference between strings and reals, or like, instances and data structures, etc?

Im thinking of structs.

Say you are building a high score system: you will need to model entries. Each entry has a score, a player name and a date. You could represent these three pieces of data as a single unit.
Then you could store these units in to a list, to have a list of scores.

Another example would be weapons: Each weapon would have an strength, speed, etc...

 

[Cloaked Games]

Ooh, okay, thank you.

 

[Yal]

"Recursive-able first-class-citizen structs" sounds like a pretty good suggestion in my book~