• Hey! Guest! The 36th GMC Jam will take place between February 27th, 12:00 UTC - March 2nd, 12:00 UTC. Why not join in! Click here to find out more!

Asset - Extension Apollo - use Lua code in GM games! (Win/Mac/Linux)

YellowAfterlife

ᴏɴʟɪɴᴇ ᴍᴜʟᴛɪᴘʟᴀʏᴇʀ
Forum Staff
Moderator
upload_2017-6-9_1-10-40.png
Marketplace link: https://marketplace.yoyogames.com/assets/5192/_
Interactive demo: https://yellowafterlife.itch.io/gamemaker-lua
Documentation: https://yal.cc/r/17/lua/
GML->Lua converter: https://yal.cc/r/17/lua/gen/
Blog post: (coming soon)
Price: USD 14.99
Modules: Windows, Mac, Linux

Apollo is an extension allows to integrate Lua scripting into your GameMaker: Studio 1 / GameMaker Studio 2 games.

The extension allows GML code to call Lua functions and for Lua code to call exposed GML scripts and/or work with exposed GML instances. This means that you can use the extension to implement high-end modding APIs for your game(s).

Other features include support for multi-value returns between GML and Lua, error handling, live reloading (see demo), and support for multi-step execution of functions (coroutines).

Known issues:
  • If you want to allow changing GM instance' variables from Lua code, you'll need to use Early Access release of GMS1 or a regular release of GMS2 (due to absence of variable_instance_ functions in the current stable release of GMS1).
    Unfortunately, there are currently also other nuances with these functions in GMS1, but workarounds exist.
  • Mac support is experimental and requires tinkering (see doc). I'm currently in process of figuring out how to improve the situation.
 
Last edited:

GMWolf

aka fel666
oooh! this looks very very good!
I am very much considering getting this. (but im not all that familiar with LUA).
Definitely bookmarking this for later!
 

lolslayer

Member
Oh man, I want this so much even if I don't know for what yet xD

Sadly enough I never really buy projects of the GM Marketplace so not sure if I'll ever touch this sexy beast D:
 

YellowAfterlife

ᴏɴʟɪɴᴇ ᴍᴜʟᴛɪᴘʟᴀʏᴇʀ
Forum Staff
Moderator
oooh! this looks very very good!
I am very much considering getting this. (but im not all that familiar with LUA).
Definitely bookmarking this for later!
Thanks! I've got a thing just for you then:
http://yal.cc/r/17/lua/gen/
You can paste GML code, hit a button, and it will generate equivalent Lua code for it.
Syntax set is the same as what's used in GMLive.
Generated code is not always 100% correct/optimal (Lua has it's own quirks and strong points), but gives a good idea of how different syntax structures translate, and can be used to largely automate the process of porting code for use on the Lua side.

Oh man, I want this so much even if I don't know for what yet xD

Sadly enough I never really buy projects of the GM Marketplace so not sure if I'll ever touch this sexy beast D:
Thanks! It is also available via itch.io, but it surely is a bit of a a shame that I'm currently not in a position where I could just develop these multi-week projects and release them for free while making a sustainable income.
 

lolslayer

Member
Thanks! It is also available via itch.io, but it surely is a bit of a a shame that I'm currently not in a position where I could just develop these multi-week projects and release them for free while making a sustainable income.
I understand, and it's great that this helps you with being able to have a sustainable income, goodluck with your other pojects! :)
 

YellowAfterlife

ᴏɴʟɪɴᴇ ᴍᴜʟᴛɪᴘʟᴀʏᴇʀ
Forum Staff
Moderator
Now that is very impressive indeed! Powered by haxe?

But I think I would much rather write Lua code over GML anyday!
Indeed! The thing shares some code with GMLive (since it also needs to build and work with abstract syntax tree for GML code), but has a different code generator (since Lua isn't quite like JS).
 

YellowAfterlife

ᴏɴʟɪɴᴇ ᴍᴜʟᴛɪᴘʟᴀʏᴇʀ
Forum Staff
Moderator
Updated - now works on Linux too. Mac... either tomorrow or on the weekend.
 

YellowAfterlife

ᴏɴʟɪɴᴇ ᴍᴜʟᴛɪᴘʟᴀʏᴇʀ
Forum Staff
Moderator
Updated, now works on Mac as well. Also renamed to "Apollo" as that's way easier to memorize.
is anyone actually going to buy the thing, or needing a blog post and/or an explanation comic with this happy cog
upload_2017-6-9_1-24-4.png
 

Samuel Venable

Time Killer
It appears you renamed it. That was a good move, I wasn't really a fan of the previous name. This is an impressive extension. Well done!
 
OMG! I missed the tweet announcing this! I've been waiting for your LUA integration since I saw you were working on it and before I finish this messag... Got it! You have one more buy ^^
 

Rukiri

Member
I love lua, but are we able to create classes and functions? Or is this just calling GML functions in which case.. cool but then it's GML just in well lua syntax.

Or we just bound to GM events like draw, collision, step(begin, end), etc?
Are we able to load other lua scripts within another script? "I'm making an RPG, and this asset will help immensely if I can actually program lua vs... just a lua/gml face"
 

YellowAfterlife

ᴏɴʟɪɴᴇ ᴍᴜʟᴛɪᴘʟᴀʏᴇʀ
Forum Staff
Moderator
I love lua, but are we able to create classes and functions? Or is this just calling GML functions in which case.. cool but then it's GML just in well lua syntax.

Or we just bound to GM events like draw, collision, step(begin, end), etc?
Are we able to load other lua scripts within another script? "I'm making an RPG, and this asset will help immensely if I can actually program lua vs... just a lua/gml face"
Entirety of Lua is supported, including standard libraries (unless you choose to unexpose them); you can call arbitrary functions for loaded Lua state whenever and get/set it's variables (see doc). GML->Lua compiler is just so that you can map existing GML snippets to Lua quicker.

OMG! I missed the tweet announcing this! I've been waiting for your LUA integration since I saw you were working on it and before I finish this messag... Got it! You have one more buy ^^
I haven't announced it on twitter/blog yet, need to finish the blog post but there was a bit too much work last week.
 

trg601

Member
Super cool! I'll buy this sometime soon (I am planning to get it when my project hits a certain stage, but I might buy it earlier because it looks so god damn neat)
It has a lot more features than I imagined, which is great because it means more control.

Speaking of which, I was looking at the documentation and I was wondering if it is possible to whitelist/blacklist certain objects from being accessed from LUA?
I have an idea for a mod api, and I want to know how much control I have over mods (for safety reasons).

Thank you!
 

YellowAfterlife

ᴏɴʟɪɴᴇ ᴍᴜʟᴛɪᴘʟᴀʏᴇʀ
Forum Staff
Moderator
Speaking of which, I was looking at the documentation and I was wondering if it is possible to whitelist/blacklist certain objects from being accessed from LUA?
I have an idea for a mod api, and I want to know how much control I have over mods (for safety reasons).
Access to instance variables is through separate pair of scripts (online documentation spots an example), so you have complete control over what is accessible and what isn't.

Common scheme is to have an array (object index -> boolean) to store "availability" per object type, and separate syntax (e.g. "__" prefix) or a ds_map (name -> boolean) for marking variables that should not be accessible by mods.

On the same notes (from recent experience integrating scripting APIs):
  • If you intend to expose a GML-like instance_create function, would add access checking into it, since spawning controller/internal objects is usually a good way to crash the game.
  • Spawnable objects should have all variables initialized to valid values in create-event.
  • If you expose draw_sprite_ functions, include some error checking in there (sprite_exists), since the GML side can still crash with that nonexistent sprite error.
  • If you expose data structure functions, you can have separate ds_maps (ds index -> boolean) per mod to quickly check if the given data structure belongs to the given mod or not.
  • You can add type checking into variable setter function to prevent from changing your variables to anything inappropriate (causing errors in other logic). Built-in variables are not affected by this, as GM will deny changing their values to mismatching types.
Overall, there's enough functionality to provide "full-scale" scripting support while preventing mods from messing up each other or the core game.
 

klys

Member
wow, this extension look interesting...

I was thinking in make a external GML scripting extension...

With the improvement of a try and catch and some other sorts of object oriented improvements...

Lua is object oriented too? not remember right now... googling...
 

YellowAfterlife

ᴏɴʟɪɴᴇ ᴍᴜʟᴛɪᴘʟᴀʏᴇʀ
Forum Staff
Moderator
Lua is object oriented too? not remember right now... googling...
Lua is sort of object oriented - it has "metatables", which can be used to set default values, set up inheritance, override index access or operators, and so on. Some of the conventional OO features are harder to mimic than the others, but you can find countless libraries/code snippets to aid with streamlining the process.

I was thinking in make a external GML scripting extension...

With the improvement of a try and catch and some other sorts of object oriented improvements...
I'd suggest to be careful with that. Scripting language design is no easy task, and making it be as fast as even non-JIT Lua (or V8, if for JS) is a big technical challenge. I've now seen multiple people make attempts and only end up with something that is too slow to be usable for general tasks.
 
@YellowAfterlife so how hard would it be to make a modding based filesystem with this?

for example how would I have a game directory where I put all my json and lua files and the game loads them automatically and runs them if there are instances?

Code:
/// @description Lua Create
// You can write your code in this editor

state = lua_state_create();
lua_add_function(state, "print", print)
Code:
/// @description Box Create
// You can write your code in this editor

lua_add_file(Lua.state, "Objects\\main.lua");
lua_add_function(Lua.state, "onInit", onInit);
lua_call(Lua.state, "onInit", id);
 

YellowAfterlife

ᴏɴʟɪɴᴇ ᴍᴜʟᴛɪᴘʟᴀʏᴇʀ
Forum Staff
Moderator
@YellowAfterlife so how hard would it be to make a modding based filesystem with this?
With this being intended for use with modding, not very hard.

To look for files in game's directory you can use the standard file_find functions - they work just fine.

Assumption about syntax is largely correct (aside of "Lua.state" in second snippet needing to be just "state") - you create a Lua state via lua_state_create, then you add GML scripts to it that you'd like to be able to call from Lua code via lua_add_function, then you load your Lua code to it via lua_add_file/lua_add_code, and then you can call Lua functions from it via lua_call. Most commonly you would have one Lua state per mod so that they all can have their own global Lua variables.
 
S

SU]Nujuju299

Guest
I'm liking that you have this and all for the possability of mods and all (Thank you so very much!) But I was wondering if there would be a bit of a tutorial series or something to help ease in? The demo is a bit much to take in all at once and I'm sitting here wondering how many of the scripts (from the demo) are actually required? I know a bit of lua, but I'm just brushing up quickly I figure I should also learn the hooks between GML and Apollo, since I figure I can't hook lua to GMS without knowing the... Hook. :p
 

YellowAfterlife

ᴏɴʟɪɴᴇ ᴍᴜʟᴛɪᴘʟᴀʏᴇʀ
Forum Staff
Moderator
I'm liking that you have this and all for the possability of mods and all (Thank you so very much!) But I was wondering if there would be a bit of a tutorial series or something to help ease in? The demo is a bit much to take in all at once and I'm sitting here wondering how many of the scripts (from the demo) are actually required? I know a bit of lua, but I'm just brushing up quickly I figure I should also learn the hooks between GML and Apollo, since I figure I can't hook lua to GMS without knowing the... Hook. :p
A simple example would be just around
Code:
state = lua_state_create();
lua_add_function(state, "some", scr_some); // where scr_some does whatever
lua_add_code(state, "function test() some() end");
lua_call(state, "test"); // ends up calling scr_some
Documentation generally has examples and/or explanations for every function.
 
S

SU]Nujuju299

Guest
Yea it's just hard to piece together all together. I was just looking through everything and I think I have it.... But I'm tired now and would like some sleep. :p Maybe a text document to along side it to help you out? I dug through everything to realize that what your demo was doing was:
-Find and load a lua script
-Declare what GMS can do in *that* lua and what *that* lua can do in GMS
--(Lua requires a reference script for each function it can do)
-run script.
(Scripts are told what and how to do in GMS, so lua is entirely up to you.)

(It took a lot of studying, referencing, examining, and disecting, for me to figure the majority of this out.)
If I comprehend properly, each object or instance is capable of holding multiple lua scripts and can run them as needed? So long as the before handling is taken care of, it seems to me that any game you make in GMS can be made using lua instead, and thus, next to if not 100% moddable.
All the same, thanks for making the lua extension! :) I was getting ready to start working on a script to [sloppily] read and use lua, when I found this. :)
 
M

MarisFrance

Guest
I bought and downloaded Apollo.gmez file from your site. It's not in my library. How can I import it to existing project?

UPD: Got it.
 
Last edited by a moderator:
M

MarisFrance

Guest
Any example of exposing ds_maps to Lua code and use them instead of tables, please?
 

YellowAfterlife

ᴏɴʟɪɴᴇ ᴍᴜʟᴛɪᴘʟᴀʏᴇʀ
Forum Staff
Moderator
I downloaded Apollo.gmez file from your site. It's not in my library. How can I import it to existing project?
Right-click Extensions section, pick "Import", pick the GMEZ

Any example of exposing ds_maps to Lua code and use them instead of tables, please?
Say, have scr_ds_map_create:
Code:
return ds_map_create();
and scr_ds_map_get:
Code:
return argument0[?argument1];
and scr_ds_map_set:
Code:
argument0[?argument1] = argument2;
and expose them:
Code:
lua_add_function(state, "ds_map_create", scr_ds_map_create);
lua_add_function(state, "ds_map_get", scr_ds_map_get);
lua_add_function(state, "ds_map_set", scr_ds_map_set);
and then you could use the three to create maps or work with map indexes passed into Lua code
 
M

MarisFrance

Guest
I'm trying to send large data from Lua to GMS. The only fast method that I found is to send data as a string.
But game crashes without error message.
I suppose that buffer is too small.
Code:
q = lua_state_create();
lua_add_code(q, "a={}; for i=1,800000 do a[i]='1' end; s=table.concat(a)");
s = lua_global_get(q, "s");
 
M

MarisFrance

Guest
Nevermind.
I splitted the string into small parts.
Code:
    local BUFFER_SIZE = 1000;
    local now = 1;
    local i = 1;
    while now < #MAP_STR do
        _G["MAP_STR_" .. i] = MAP_STR:sub(now,now+BUFFER_SIZE-1);
        i = i + 1
        now = now + BUFFER_SIZE
    end
 

YellowAfterlife

ᴏɴʟɪɴᴇ ᴍᴜʟᴛɪᴘʟᴀʏᴇʀ
Forum Staff
Moderator
I'm trying to send large data from Lua to GMS. The only fast method that I found is to send data as a string.
But game crashes without error message.
I suppose that buffer is too small.
Code:
q = lua_state_create();
lua_add_code(q, "a={}; for i=1,800000 do a[i]='1' end; s=table.concat(a)");
s = lua_global_get(q, "s");
Argument buffer is 64KB by default, you can change this at the start of Apollo.gml. Writing values from GML side will auto-expand it as it's a grow-buffer, but not from Lua's side. I'm currently looking into improving a couple of things with this.
 
M

MarisFrance

Guest
Please include Lua CJSON.
I know I can use Lua JSON, but CJSON is mush faster (14 times faster) and supports utf8.
Btw it's the most popular module for Lua.
 
Last edited by a moderator:

YellowAfterlife

ᴏɴʟɪɴᴇ ᴍᴜʟᴛɪᴘʟᴀʏᴇʀ
Forum Staff
Moderator
Please include Lua CJSON.
I know I can use Lua JSON, but CJSON is mush faster (14 times faster) and supports utf8.
Btw it's the most popular module for Lua.
I intend to include the source code with the next release to make it easier to include arbitrary C modules or swap over to LuaJIT if performance is preferable over Lua 5.3 features.
 
M

MarisFrance

Guest
Okay. But I don't know C++.
So it would be great if you add cjson and compile it. :) This is the main module that everbody needs.

P.S. I'm trying to make using Lua as fast as possible. I know that calls Lua->GML and GML->Lua are slow. So I'm trying to cut down on using those calls to reduce call overhead. What else can I do?
My game is coop game (uses network). So I pack tables via json and send them over network as a string. That's why I need fast cjson module.
Btw I love your extension very much. :)
 
Last edited by a moderator:

YellowAfterlife

ᴏɴʟɪɴᴇ ᴍᴜʟᴛɪᴘʟᴀʏᴇʀ
Forum Staff
Moderator
So I'm trying to cut down on using those calls to reduce call overhead. What else can I do?
My game is coop game (uses network). So I pack tables via json and send them over network as a string. That's why I need fast cjson module.
Sending JSON over network is not exactly how to make things faster. You should looking for libraries that serialize tables into binary format (maybe "protocol buffers") or writing out data into stream conventionally.
 

YellowAfterlife

ᴏɴʟɪɴᴇ ᴍᴜʟᴛɪᴘʟᴀʏᴇʀ
Forum Staff
Moderator
@YellowAfterlife I am curious if this will receive mobile platform support.
I could not make any concrete promises on that part - due to the current extension mechanism, you end up with GML code -> C++ code (runner) -> Java/ObjC code (extension wrapper) -> C++ code (your actual extension), which makes for far more headache than one would prefer.

If you want some scripting language but not necessarily Lua, you can take a look at this (pure GML solution with basic GML-like syntax and explanations).
 

Posh Indie

Member
I could not make any concrete promises on that part - due to the current extension mechanism, you end up with GML code -> C++ code (runner) -> Java/ObjC code (extension wrapper) -> C++ code (your actual extension), which makes for far more headache than one would prefer.

If you want some scripting language but not necessarily Lua, you can take a look at this (pure GML solution with basic GML-like syntax and explanations).
This is also interesting. I am curious about the performance of it, though.
 

YellowAfterlife

ᴏɴʟɪɴᴇ ᴍᴜʟᴛɪᴘʟᴀʏᴇʀ
Forum Staff
Moderator
This is also interesting. I am curious about the performance of it, though.
Assuming that you use YYC, performance is decent - of course, it could be much better if you could cut the middleman and just add a bit of C++ code (but that's for later into this year), but the approach still proven sufficient for a similar system in Nuclear Throne Together to allow for huge variety of mods.
 

Posh Indie

Member
Assuming that you use YYC, performance is decent - of course, it could be much better if you could cut the middleman and just add a bit of C++ code (but that's for later into this year), but the approach still proven sufficient for a similar system in Nuclear Throne Together to allow for huge variety of mods.
I just played with it a bit (Very limited, but it was very easy to make custom functions) and it is just awesome. Any chance for documentation, or is it "Read it and figure it out", syntax-wise. I noticed some jump statements and some things like that.

What would the process be like for allowing users to provide external scripts that will be picked up by the game you create? Write something to turn their script file into a code string that you feed into the execute function? Are all instance variables exposed by default within the script or is there a way to "sandbox" it a bit? (It is very possible that I overlooked these things as I only messed with it for a few minutes, haha)

If it compiles for all platforms, throw it up on the marketplace and I will buy it just to show support for the things you do.
 

YellowAfterlife

ᴏɴʟɪɴᴇ ᴍᴜʟᴛɪᴘʟᴀʏᴇʀ
Forum Staff
Moderator
Any chance for documentation, or is it "Read it and figure it out", syntax-wise. I noticed some jump statements and some things like that.
There are two links in repository description which cover most of the supported syntax and it's implementation up to the last commit of part-2 branch.

Then the person that was sponsoring the second part of the tutorial paid me a bit more and I added a few more features that currently reside on master branch - labels, jumps, simple subroutines (just going back to where you jumped in from), state serialization, and `select` structure (pure syntactic sugar - see the demo object).

As it goes, if anyone is willing to give me money, I can write up detailed documentation and/or explanation on implementation for these, or add any additional syntactic features.

What would the process be like for allowing users to provide external scripts that will be picked up by the game you create? Write something to turn their script file into a code string that you feed into the execute function?
As this does not currently spot any syntax for defining multiple independent "scripts" in one input string, you would "compile" one txr_program per user function and resolve calls between them using "default function" (see global.extra_functions in the demo).

Are all instance variables exposed by default within the script or is there a way to "sandbox" it a bit?
You can customize behaviour for ident (get variable) / set_ident (set variable) in txr_thread_resume

If it compiles for all platforms, throw it up on the marketplace and I will buy it just to show support for the things you do.
The intent was to have something free and publicly-available while relying on donations/contributions for further development, so it'd be a free download on marketplace. But yeah, I'll get that and an itch page (for easier donations) up later today
 
Top