Asset - Extension Apollo - use Lua code in GM games! (Win/Mac/Linux)

Discussion in 'Marketplace' started by YellowAfterlife, Jun 4, 2017.

  1. YellowAfterlife

    YellowAfterlife ᴏɴʟɪɴᴇ ᴍᴜʟᴛɪᴘʟᴀʏᴇʀ Forum Staff Moderator

    Joined:
    Apr 21, 2016
    Posts:
    2,383
    upload_2017-6-9_1-10-40.png
    Marketplace link: https://marketplace.yoyogames.com/assets/5192/_
    Interactive demo: https://yellowafterlife.itch.io/gamemaker-lua
    Documentation: https://yal.cc/r/17/lua/
    GML->Lua converter: https://yal.cc/r/17/lua/gen/
    Blog post: (coming soon)
    Price: USD 14.99
    Modules: Windows, Mac, Linux

    Apollo is an extension allows to integrate Lua scripting into your GameMaker: Studio 1 / GameMaker Studio 2 games.

    The extension allows GML code to call Lua functions and for Lua code to call exposed GML scripts and/or work with exposed GML instances. This means that you can use the extension to implement high-end modding APIs for your game(s).

    Other features include support for multi-value returns between GML and Lua, error handling, live reloading (see demo), and support for multi-step execution of functions (coroutines).

    Known issues:
    • If you want to allow changing GM instance' variables from Lua code, you'll need to use Early Access release of GMS1 or a regular release of GMS2 (due to absence of variable_instance_ functions in the current stable release of GMS1).
      Unfortunately, there are currently also other nuances with these functions in GMS1, but workarounds exist.
    • Mac support is experimental and requires tinkering (see doc). I'm currently in process of figuring out how to improve the situation.
     
    Last edited: Jun 25, 2017
    MarisFrance, GMpeaks, Reign and 11 others like this.
  2. GMWolf

    GMWolf aka fel666

    Joined:
    Jun 21, 2016
    Posts:
    3,394
    oooh! this looks very very good!
    I am very much considering getting this. (but im not all that familiar with LUA).
    Definitely bookmarking this for later!
     
  3. lolslayer

    lolslayer Member

    Joined:
    Jun 23, 2016
    Posts:
    688
    Oh man, I want this so much even if I don't know for what yet xD

    Sadly enough I never really buy projects of the GM Marketplace so not sure if I'll ever touch this sexy beast D:
     
  4. YellowAfterlife

    YellowAfterlife ᴏɴʟɪɴᴇ ᴍᴜʟᴛɪᴘʟᴀʏᴇʀ Forum Staff Moderator

    Joined:
    Apr 21, 2016
    Posts:
    2,383
    Thanks! I've got a thing just for you then:
    http://yal.cc/r/17/lua/gen/
    You can paste GML code, hit a button, and it will generate equivalent Lua code for it.
    Syntax set is the same as what's used in GMLive.
    Generated code is not always 100% correct/optimal (Lua has it's own quirks and strong points), but gives a good idea of how different syntax structures translate, and can be used to largely automate the process of porting code for use on the Lua side.

    Thanks! It is also available via itch.io, but it surely is a bit of a a shame that I'm currently not in a position where I could just develop these multi-week projects and release them for free while making a sustainable income.
     
    andev and hippyman like this.
  5. GMWolf

    GMWolf aka fel666

    Joined:
    Jun 21, 2016
    Posts:
    3,394
    Now that is very impressive indeed! Powered by haxe?

    But I think I would much rather write Lua code over GML anyday!
     
  6. lolslayer

    lolslayer Member

    Joined:
    Jun 23, 2016
    Posts:
    688
    I understand, and it's great that this helps you with being able to have a sustainable income, goodluck with your other pojects! :)
     
    YellowAfterlife likes this.
  7. YellowAfterlife

    YellowAfterlife ᴏɴʟɪɴᴇ ᴍᴜʟᴛɪᴘʟᴀʏᴇʀ Forum Staff Moderator

    Joined:
    Apr 21, 2016
    Posts:
    2,383
    Indeed! The thing shares some code with GMLive (since it also needs to build and work with abstract syntax tree for GML code), but has a different code generator (since Lua isn't quite like JS).
     
  8. YellowAfterlife

    YellowAfterlife ᴏɴʟɪɴᴇ ᴍᴜʟᴛɪᴘʟᴀʏᴇʀ Forum Staff Moderator

    Joined:
    Apr 21, 2016
    Posts:
    2,383
    Updated - now works on Linux too. Mac... either tomorrow or on the weekend.
     
  9. YellowAfterlife

    YellowAfterlife ᴏɴʟɪɴᴇ ᴍᴜʟᴛɪᴘʟᴀʏᴇʀ Forum Staff Moderator

    Joined:
    Apr 21, 2016
    Posts:
    2,383
    Updated, now works on Mac as well. Also renamed to "Apollo" as that's way easier to memorize.
    is anyone actually going to buy the thing, or needing a blog post and/or an explanation comic with this happy cog
    upload_2017-6-9_1-24-4.png
     
  10. amusudan

    amusudan Lousiest of Potatoes

    Joined:
    Jun 20, 2016
    Posts:
    183
    I will buy it once I get some money :), the happy cog looks cute, it deserves to be used in a post!
     
    YellowAfterlife likes this.
  11. Samuel Venable

    Samuel Venable Time Killer

    Joined:
    Sep 13, 2016
    Posts:
    1,240
    It appears you renamed it. That was a good move, I wasn't really a fan of the previous name. This is an impressive extension. Well done!
     
  12. Curial Lloses

    Curial Lloses Member

    Joined:
    Jun 20, 2016
    Posts:
    78
    OMG! I missed the tweet announcing this! I've been waiting for your LUA integration since I saw you were working on it and before I finish this messag... Got it! You have one more buy ^^
     
  13. Rukiri

    Rukiri Member

    Joined:
    Sep 25, 2016
    Posts:
    141
    I love lua, but are we able to create classes and functions? Or is this just calling GML functions in which case.. cool but then it's GML just in well lua syntax.

    Or we just bound to GM events like draw, collision, step(begin, end), etc?
    Are we able to load other lua scripts within another script? "I'm making an RPG, and this asset will help immensely if I can actually program lua vs... just a lua/gml face"
     
  14. YellowAfterlife

    YellowAfterlife ᴏɴʟɪɴᴇ ᴍᴜʟᴛɪᴘʟᴀʏᴇʀ Forum Staff Moderator

    Joined:
    Apr 21, 2016
    Posts:
    2,383
    Entirety of Lua is supported, including standard libraries (unless you choose to unexpose them); you can call arbitrary functions for loaded Lua state whenever and get/set it's variables (see doc). GML->Lua compiler is just so that you can map existing GML snippets to Lua quicker.

    I haven't announced it on twitter/blog yet, need to finish the blog post but there was a bit too much work last week.
     
  15. Rukiri

    Rukiri Member

    Joined:
    Sep 25, 2016
    Posts:
    141
    Full access to lua? nice! Welp, found a reason to upgrade my GMS1.4x license.
     
  16. trg601

    trg601 Member

    Joined:
    Jun 21, 2016
    Posts:
    144
    Super cool! I'll buy this sometime soon (I am planning to get it when my project hits a certain stage, but I might buy it earlier because it looks so god damn neat)
    It has a lot more features than I imagined, which is great because it means more control.

    Speaking of which, I was looking at the documentation and I was wondering if it is possible to whitelist/blacklist certain objects from being accessed from LUA?
    I have an idea for a mod api, and I want to know how much control I have over mods (for safety reasons).

    Thank you!
     
  17. YellowAfterlife

    YellowAfterlife ᴏɴʟɪɴᴇ ᴍᴜʟᴛɪᴘʟᴀʏᴇʀ Forum Staff Moderator

    Joined:
    Apr 21, 2016
    Posts:
    2,383
    Access to instance variables is through separate pair of scripts (online documentation spots an example), so you have complete control over what is accessible and what isn't.

    Common scheme is to have an array (object index -> boolean) to store "availability" per object type, and separate syntax (e.g. "__" prefix) or a ds_map (name -> boolean) for marking variables that should not be accessible by mods.

    On the same notes (from recent experience integrating scripting APIs):
    • If you intend to expose a GML-like instance_create function, would add access checking into it, since spawning controller/internal objects is usually a good way to crash the game.
    • Spawnable objects should have all variables initialized to valid values in create-event.
    • If you expose draw_sprite_ functions, include some error checking in there (sprite_exists), since the GML side can still crash with that nonexistent sprite error.
    • If you expose data structure functions, you can have separate ds_maps (ds index -> boolean) per mod to quickly check if the given data structure belongs to the given mod or not.
    • You can add type checking into variable setter function to prevent from changing your variables to anything inappropriate (causing errors in other logic). Built-in variables are not affected by this, as GM will deny changing their values to mismatching types.
    Overall, there's enough functionality to provide "full-scale" scripting support while preventing mods from messing up each other or the core game.
     
    trg601 likes this.
  18. klys

    klys Member

    Joined:
    Jun 21, 2016
    Posts:
    110
    wow, this extension look interesting...

    I was thinking in make a external GML scripting extension...

    With the improvement of a try and catch and some other sorts of object oriented improvements...

    Lua is object oriented too? not remember right now... googling...
     
  19. YellowAfterlife

    YellowAfterlife ᴏɴʟɪɴᴇ ᴍᴜʟᴛɪᴘʟᴀʏᴇʀ Forum Staff Moderator

    Joined:
    Apr 21, 2016
    Posts:
    2,383
    Lua is sort of object oriented - it has "metatables", which can be used to set default values, set up inheritance, override index access or operators, and so on. Some of the conventional OO features are harder to mimic than the others, but you can find countless libraries/code snippets to aid with streamlining the process.

    I'd suggest to be careful with that. Scripting language design is no easy task, and making it be as fast as even non-JIT Lua (or V8, if for JS) is a big technical challenge. I've now seen multiple people make attempts and only end up with something that is too slow to be usable for general tasks.
     
  20. Shadowblitz16

    Shadowblitz16 Member

    Joined:
    Jun 23, 2016
    Posts:
    595
    @YellowAfterlife so how hard would it be to make a modding based filesystem with this?

    for example how would I have a game directory where I put all my json and lua files and the game loads them automatically and runs them if there are instances?

    Code:
    /// @description Lua Create
    // You can write your code in this editor
    
    state = lua_state_create();
    lua_add_function(state, "print", print)
    
    Code:
    /// @description Box Create
    // You can write your code in this editor
    
    lua_add_file(Lua.state, "Objects\\main.lua");
    lua_add_function(Lua.state, "onInit", onInit);
    lua_call(Lua.state, "onInit", id);
    
     
  21. YellowAfterlife

    YellowAfterlife ᴏɴʟɪɴᴇ ᴍᴜʟᴛɪᴘʟᴀʏᴇʀ Forum Staff Moderator

    Joined:
    Apr 21, 2016
    Posts:
    2,383
    With this being intended for use with modding, not very hard.

    To look for files in game's directory you can use the standard file_find functions - they work just fine.

    Assumption about syntax is largely correct (aside of "Lua.state" in second snippet needing to be just "state") - you create a Lua state via lua_state_create, then you add GML scripts to it that you'd like to be able to call from Lua code via lua_add_function, then you load your Lua code to it via lua_add_file/lua_add_code, and then you can call Lua functions from it via lua_call. Most commonly you would have one Lua state per mod so that they all can have their own global Lua variables.
     
  22. Shadowblitz16

    Shadowblitz16 Member

    Joined:
    Jun 23, 2016
    Posts:
    595
    ok I'm going to move this to the programming subforum
     
  23. SU]Nujuju299

    SU]Nujuju299 Member

    Joined:
    Jul 12, 2016
    Posts:
    9
    I'm liking that you have this and all for the possability of mods and all (Thank you so very much!) But I was wondering if there would be a bit of a tutorial series or something to help ease in? The demo is a bit much to take in all at once and I'm sitting here wondering how many of the scripts (from the demo) are actually required? I know a bit of lua, but I'm just brushing up quickly I figure I should also learn the hooks between GML and Apollo, since I figure I can't hook lua to GMS without knowing the... Hook. :p
     
  24. YellowAfterlife

    YellowAfterlife ᴏɴʟɪɴᴇ ᴍᴜʟᴛɪᴘʟᴀʏᴇʀ Forum Staff Moderator

    Joined:
    Apr 21, 2016
    Posts:
    2,383
    A simple example would be just around
    Code:
    state = lua_state_create();
    lua_add_function(state, "some", scr_some); // where scr_some does whatever
    lua_add_code(state, "function test() some() end");
    lua_call(state, "test"); // ends up calling scr_some
    
    Documentation generally has examples and/or explanations for every function.
     
  25. SU]Nujuju299

    SU]Nujuju299 Member

    Joined:
    Jul 12, 2016
    Posts:
    9
    Yea it's just hard to piece together all together. I was just looking through everything and I think I have it.... But I'm tired now and would like some sleep. :p Maybe a text document to along side it to help you out? I dug through everything to realize that what your demo was doing was:
    -Find and load a lua script
    -Declare what GMS can do in *that* lua and what *that* lua can do in GMS
    --(Lua requires a reference script for each function it can do)
    -run script.
    (Scripts are told what and how to do in GMS, so lua is entirely up to you.)

    (It took a lot of studying, referencing, examining, and disecting, for me to figure the majority of this out.)
    If I comprehend properly, each object or instance is capable of holding multiple lua scripts and can run them as needed? So long as the before handling is taken care of, it seems to me that any game you make in GMS can be made using lua instead, and thus, next to if not 100% moddable.
    All the same, thanks for making the lua extension! :) I was getting ready to start working on a script to [sloppily] read and use lua, when I found this. :)
     
  26. MarisFrance

    MarisFrance Member

    Joined:
    Jul 26, 2018
    Posts:
    22
    I bought and downloaded Apollo.gmez file from your site. It's not in my library. How can I import it to existing project?

    UPD: Got it.
     
    Last edited: Jul 27, 2018
  27. MarisFrance

    MarisFrance Member

    Joined:
    Jul 26, 2018
    Posts:
    22
    Any example of exposing ds_maps to Lua code and use them instead of tables, please?
     
  28. YellowAfterlife

    YellowAfterlife ᴏɴʟɪɴᴇ ᴍᴜʟᴛɪᴘʟᴀʏᴇʀ Forum Staff Moderator

    Joined:
    Apr 21, 2016
    Posts:
    2,383
    Right-click Extensions section, pick "Import", pick the GMEZ

    Say, have scr_ds_map_create:
    Code:
    return ds_map_create();
    and scr_ds_map_get:
    Code:
    return argument0[?argument1];
    and scr_ds_map_set:
    Code:
    argument0[?argument1] = argument2;
    and expose them:
    Code:
    lua_add_function(state, "ds_map_create", scr_ds_map_create);
    lua_add_function(state, "ds_map_get", scr_ds_map_get);
    lua_add_function(state, "ds_map_set", scr_ds_map_set);
    and then you could use the three to create maps or work with map indexes passed into Lua code
     
    MarisFrance likes this.
  29. MarisFrance

    MarisFrance Member

    Joined:
    Jul 26, 2018
    Posts:
    22
    I'm trying to send large data from Lua to GMS. The only fast method that I found is to send data as a string.
    But game crashes without error message.
    I suppose that buffer is too small.
    Code:
    q = lua_state_create();
    lua_add_code(q, "a={}; for i=1,800000 do a[i]='1' end; s=table.concat(a)");
    s = lua_global_get(q, "s");
    
     
  30. MarisFrance

    MarisFrance Member

    Joined:
    Jul 26, 2018
    Posts:
    22
    Nevermind.
    I splitted the string into small parts.
    Code:
        local BUFFER_SIZE = 1000;
        local now = 1;
        local i = 1;
        while now < #MAP_STR do
            _G["MAP_STR_" .. i] = MAP_STR:sub(now,now+BUFFER_SIZE-1);
            i = i + 1
            now = now + BUFFER_SIZE
        end
     
  31. YellowAfterlife

    YellowAfterlife ᴏɴʟɪɴᴇ ᴍᴜʟᴛɪᴘʟᴀʏᴇʀ Forum Staff Moderator

    Joined:
    Apr 21, 2016
    Posts:
    2,383
    Argument buffer is 64KB by default, you can change this at the start of Apollo.gml. Writing values from GML side will auto-expand it as it's a grow-buffer, but not from Lua's side. I'm currently looking into improving a couple of things with this.
     
    MarisFrance likes this.
  32. MarisFrance

    MarisFrance Member

    Joined:
    Jul 26, 2018
    Posts:
    22
    Please include Lua CJSON.
    I know I can use Lua JSON, but CJSON is mush faster (14 times faster) and supports utf8.
    Btw it's the most popular module for Lua.
     
    Last edited: Aug 7, 2018
  33. YellowAfterlife

    YellowAfterlife ᴏɴʟɪɴᴇ ᴍᴜʟᴛɪᴘʟᴀʏᴇʀ Forum Staff Moderator

    Joined:
    Apr 21, 2016
    Posts:
    2,383
    I intend to include the source code with the next release to make it easier to include arbitrary C modules or swap over to LuaJIT if performance is preferable over Lua 5.3 features.
     
    MarisFrance likes this.
  34. MarisFrance

    MarisFrance Member

    Joined:
    Jul 26, 2018
    Posts:
    22
    Okay. But I don't know C++.
    So it would be great if you add cjson and compile it. :) This is the main module that everbody needs.

    P.S. I'm trying to make using Lua as fast as possible. I know that calls Lua->GML and GML->Lua are slow. So I'm trying to cut down on using those calls to reduce call overhead. What else can I do?
    My game is coop game (uses network). So I pack tables via json and send them over network as a string. That's why I need fast cjson module.
    Btw I love your extension very much. :)
     
    Last edited: Aug 11, 2018
  35. YellowAfterlife

    YellowAfterlife ᴏɴʟɪɴᴇ ᴍᴜʟᴛɪᴘʟᴀʏᴇʀ Forum Staff Moderator

    Joined:
    Apr 21, 2016
    Posts:
    2,383
    Sending JSON over network is not exactly how to make things faster. You should looking for libraries that serialize tables into binary format (maybe "protocol buffers") or writing out data into stream conventionally.
     
  36. Posh Indie

    Posh Indie Member

    Joined:
    Dec 5, 2016
    Posts:
    200
    @YellowAfterlife I am curious if this will receive mobile platform support.
     
  37. YellowAfterlife

    YellowAfterlife ᴏɴʟɪɴᴇ ᴍᴜʟᴛɪᴘʟᴀʏᴇʀ Forum Staff Moderator

    Joined:
    Apr 21, 2016
    Posts:
    2,383
    I could not make any concrete promises on that part - due to the current extension mechanism, you end up with GML code -> C++ code (runner) -> Java/ObjC code (extension wrapper) -> C++ code (your actual extension), which makes for far more headache than one would prefer.

    If you want some scripting language but not necessarily Lua, you can take a look at this (pure GML solution with basic GML-like syntax and explanations).
     
  38. Posh Indie

    Posh Indie Member

    Joined:
    Dec 5, 2016
    Posts:
    200
    This is also interesting. I am curious about the performance of it, though.
     
  39. YellowAfterlife

    YellowAfterlife ᴏɴʟɪɴᴇ ᴍᴜʟᴛɪᴘʟᴀʏᴇʀ Forum Staff Moderator

    Joined:
    Apr 21, 2016
    Posts:
    2,383
    Assuming that you use YYC, performance is decent - of course, it could be much better if you could cut the middleman and just add a bit of C++ code (but that's for later into this year), but the approach still proven sufficient for a similar system in Nuclear Throne Together to allow for huge variety of mods.
     
  40. Posh Indie

    Posh Indie Member

    Joined:
    Dec 5, 2016
    Posts:
    200
    I just played with it a bit (Very limited, but it was very easy to make custom functions) and it is just awesome. Any chance for documentation, or is it "Read it and figure it out", syntax-wise. I noticed some jump statements and some things like that.

    What would the process be like for allowing users to provide external scripts that will be picked up by the game you create? Write something to turn their script file into a code string that you feed into the execute function? Are all instance variables exposed by default within the script or is there a way to "sandbox" it a bit? (It is very possible that I overlooked these things as I only messed with it for a few minutes, haha)

    If it compiles for all platforms, throw it up on the marketplace and I will buy it just to show support for the things you do.
     
  41. YellowAfterlife

    YellowAfterlife ᴏɴʟɪɴᴇ ᴍᴜʟᴛɪᴘʟᴀʏᴇʀ Forum Staff Moderator

    Joined:
    Apr 21, 2016
    Posts:
    2,383
    There are two links in repository description which cover most of the supported syntax and it's implementation up to the last commit of part-2 branch.

    Then the person that was sponsoring the second part of the tutorial paid me a bit more and I added a few more features that currently reside on master branch - labels, jumps, simple subroutines (just going back to where you jumped in from), state serialization, and `select` structure (pure syntactic sugar - see the demo object).

    As it goes, if anyone is willing to give me money, I can write up detailed documentation and/or explanation on implementation for these, or add any additional syntactic features.

    As this does not currently spot any syntax for defining multiple independent "scripts" in one input string, you would "compile" one txr_program per user function and resolve calls between them using "default function" (see global.extra_functions in the demo).

    You can customize behaviour for ident (get variable) / set_ident (set variable) in txr_thread_resume

    The intent was to have something free and publicly-available while relying on donations/contributions for further development, so it'd be a free download on marketplace. But yeah, I'll get that and an itch page (for easier donations) up later today
     

Share This Page

  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice