Massive Memory Leak with Scripts

RujiK

Member
If you import your GMS2 project into GMS2.3, it converts all of your scripts into functions inside of scripts.

Unfortunately, every time the script is ran, THE SCRIPT RECREATES THE FUNCTION instead of erasing the old function with the same name. This is a huge memory leak. As far as I can tell, the only solution is to enable garbage collection, but garbage collection kills my FPS. Is there anyway to "delete" a function from memory manually?

Here is a simple example:
in create_event: gc_enable(false);

Code:
// THIS IS INSIDE A SCRIPT!
function func_test(argument0,argument1,argument2) {
    var xx = argument0;
    var yy = argument1;
    var zz = argument2;

    return(false);    
}
Then call the script 1000 times a frame and watch as your memory explodes. So my question, can I delete a function from memory or somehow prevent the leak?
 

Evanski

Raccoon Lord
Forum Staff
Moderator
Create the function as a variable foo = function()
then when youre done, free the variable

I dont think this stops it being called if your using the function name but if you only ever call it from the variable it wont be ran
 

Roldy

Member
How are you calling the script? script_execute?

WHY are you calling the script in 2.3?

Just like any other memory allocated in a script it will need to be managed. Best solution; Stop calling scripts and instead call functions. Second Best solution; manage your memory.

Review changes in 2.3 :https://manual.yoyogames.com/#t=Gam...nctions.htm&rhsearch=script execute&ux=search

If you can, provide a simple example that leaks.
 
Last edited:

Evanski

Raccoon Lord
Forum Staff
Moderator
How are you calling the script? script_execute?

WHY are you calling the script in 2.3?

Just like any other memory allocated in a script it will need to be managed. Best solution; Stop calling scripts and instead call functions. Second Best solution; manage your memory.

Review changes in 2.3: http://127.0.0.1:51290/index.htm#t=GameMaker_Language/GML_Reference/Asset_Management/Scripts/script_execute.htm

If you can, provide a simple example that leaks.
 

FrostyCat

Redemption Seeker
It's not the function itself that's calling for the garbage collector, it's the local variables being used inside the function. They don't clean after themselves because you told them not to. On top of that, most people who have been in GC land for too long also forget that strings don't clean after themselves either.
 

Nocturne

Friendly Tyrant
Forum Staff
Admin
Are you calling the SCRIPT or the function inside the script? Like, if you have a script calle "init_functions" and in it you define a function called "my_function", then calling the script 1000 times WILL create 1000 versions of the function "my_function". However calling the script function "my_function" 1000 times shouldn't create a memory leak as the function was only defined once when the game started and GM ran the script automatically.
 
H

Hecate

Guest
It's not the function itself that's calling for the garbage collector, it's the local variables being used inside the function. They don't clean after themselves because you told them not to. On top of that, most people who have been in GC land for too long also forget that strings don't clean after themselves either.
You have to clean them up? How do you do that?
 

FrostyCat

Redemption Seeker
You have to clean them up? How do you do that?
In GML, the garbage collector on native exports cleans after spent local variables, unlinked strings/arrays/structs, etc. in a normal setup. But the original poster shut it off permanently to save some frame rate, and now his "solution" is worse than the problem. He is not getting any of the space back until the garbage collector is back on.
 

Nocturne

Friendly Tyrant
Forum Staff
Admin
Is naming the script and the function the same (compatibility scrunctions) going to cause memory leaks when calling the function (same name as script)? šŸ˜©
Well... my understanding is it shouldn't... but that doesn't mean it doesn't... worth doing some testing and see! And if it does, then probably worth filing a bug report too.
 

Fanatrick

Member
In GML, the garbage collector on native exports cleans after spent local variables, unlinked strings/arrays/structs, etc. in a normal setup. But the original poster shut it off permanently to save some frame rate, and now his "solution" is worse than the problem. He is not getting any of the space back until the garbage collector is back on.
??? He's tackling an entirely different problem than what you're focused on. He has memory leaks from calling scripts, calling a script and GM mistakenly making duplicates should have nothing to do with GC. As for "his solution being worse than the problem" nothing is stopping him from calling gc_collect().

Edit: crappy GMC formatting
 

xenoargh

Member
The GC's a major hit on CPU, though, to be fair, and unlike Java, it doesn't appear to support parameters allowing end-users to tame it. It's also scant on descriptions of its scope.

Having it on also caused my game to crash, presumably because there was something not clean-able. But, given that GMS crashed without any useful error message to point to where things went wrong, other than a generic MALLOC error... finding it will be kind of hard. I found a couple of Surfaces that weren't 100% guaranteed to get deleted and a couple of other minor things and fixed them, but at this point, I don't trust the GC; nothing in GML should crash so opaquely.

That said... I'm going to go look at memory use in Debug and see if there's genuine cause for concern; this is an interesting topic. I certainly haven't seen ballooning RAM over time thus far with GC off, but I've been so busy with new features I haven't done a complete run through my current tech demo in weeks; given the scope of my game (3-5K+ Instances in one Room, being managed) if it'll happen to anybody... it'll happen to me, lol.

Lastly... if gc_enable(false) is called... gc_collect() shouldn't really do much, one would think? So I'm not even sure why it's a thing, honestly, when it appears to be calling practically every frame anyhow.
 

Nocturne

Friendly Tyrant
Forum Staff
Admin
The garbage collector will only clean up methods and structs, and little else currently. So if you are expecting it to clean up surface data you are mistaken. The manual clearly says:

"NOTE: Please note that things like surfaces, data structures, buffers and other dynamic resources are not garbage collected and have their own destroy functions to clean up the memory associated with them. As a rule of thumb, if anything you create at runtime has a destroy function then it won't be garbage collected and you will have to deal with it yourself in code."

Also, if having the GC enabled cause your project to crash, then you should file a bug report and include a link to a YYZ of the project (if you haven't already!)

Lastly... if gc_enable(false) is called... gc_collect() shouldn't really do much, one would think?
This is correct. The GC has to be enabled for it to run and collect things...


Please see this post for information on what the GC collects: https://forum.yoyogames.com/index.p...eference-on-physics-object.79508/#post-472667
 
Last edited:

Evanski

Raccoon Lord
Forum Staff
Moderator
Well... my understanding is it shouldn't... but that doesn't mean it doesn't... worth doing some testing and see! And if it does, then probably worth filing a bug report too.
I've not noticed any memory leaks with this, nor if the function is made in an object and called
 

FrostyCat

Redemption Seeker
??? He's tackling an entirely different problem than what you're focused on. He has memory leaks from calling scripts, calling a script has nothing to do with his issue at hand. As for "his solution being worse than the problem" nothing is stopping him from calling gc_collect().

Edit: crappy GMC formatting
Of course calling the script has something to do with it. Pay attention to the script function he's trying to run:
GML:
function func_test(argument0,argument1,argument2) {
    var xx = argument0;
    var yy = argument1;
    var zz = argument2;

    return(false);   
}
I see 3 extra local variables, 6 if named parameters are also internally implemented as local variables. In a normal setup, the GC would have been responsible for returning them to the memory pool after the call is over, but he shut it off. If this is called 1000 times a frame at 60 FPS, and assuming they are all doubles, then the leak is 1000*6*8*60 = 2880000 bytes every second (2.88MB/s).
Lastly... if gc_enable(false) is called... gc_collect() shouldn't really do much, one would think? So I'm not even sure why it's a thing, honestly, when it appears to be calling practically every frame anyhow.
gc_collect() would do a lot if you've shut off GC explicitly, for reasons I've already mentioned. The garbage would pile up, then gc_collect() returns it to the runner's pool, then there would be that much room to reuse before the runner has to ask the OS heap for more.
The garbage collector will only clean up methods and structs, and little else currently.
Probably strings too, for people who know what it's like working with strings in C.

I'll save the conclusion for actual developers in YoYo's core team, though.
 
Last edited:

Roldy

Member
Of course calling the script has something to do with it. Pay attention to the script function he's trying to run:
GML:
function func_test(argument0,argument1,argument2) {
    var xx = argument0;
    var yy = argument1;
    var zz = argument2;

    return(false);
}
I see 3 extra local variables, 6 if named parameters are also internally implemented as local variables. In a normal setup, the GC would have been responsible for returning them to the memory pool after the call is over, but he shut it off. If this is called 1000 times a frame at 60 FPS, and assuming they are all doubles, then the leak is 1000*6*8*60 = 2880000 bytes every second (2.88MB/s).

gc_collect() would do a lot if you've shut off GC explicitly, for reasons I've already mentioned. The garbage would pile up, then gc_collect() returns it to the runner's pool, then there would be that much room to reuse before the runner has to ask the OS heap for more.

Probably strings too, for people who know what it's like working with strings in C.

I'll save the conclusion for actual developers in YoYo's core team, though.
That doesn't make any sense. It would be interesting if you can replicate that behavior then post the code.

The gc should have nothing to do with locals (or temporaries). Depending on implementation locals would go on the call stack, and be popped when returning from function. They don't get 'allocated.' Strings, arrays etc... yes. But the locals themselves the gc wouldn't be dealing with. I can't replicate any leak from simply calling a function with gc_enabled or disabled. If you can post code to demonstrate a leak from local variables I will happily run it and confirm.
 
Last edited:
H

Hecate

Guest
In GML, the garbage collector on native exports cleans after spent local variables, unlinked strings/arrays/structs, etc. in a normal setup. But the original poster shut it off permanently to save some frame rate, and now his "solution" is worse than the problem. He is not getting any of the space back until the garbage collector is back on.
Ah I see. So I take it that scripts with only arguments wouldn't be much of a problem, then?
 

Fanatrick

Member
GML:
function func_test(argument0,argument1,argument2) {
    var xx = argument0;
    var yy = argument1;
    var zz = argument2;

    return(false);
}
I see 3 extra local variables.
I don't, I see 3 temporary ones never allocated to anything. Makes zero sense for these to be needing any GC whatsoever, it never was the case in GML and isn't the case in any other language I've written before. But alas, YoYo should know more about this.
 
Last edited:

xenoargh

Member
Those three temp variables should be de-allocated at the end of the Function w/out GC needing to intervene, one would think; they're now dangling. If this isn't normal behavior of Functions, something's dreadfully wrong with 2.3's design.

But this did pique my curiosity; perhaps when returning anything declared as a var internally to a Function, we need put the temporary variable outside the scope of the Function now, because the scope isn't automatically Instance-that-called-me? Otherwise, we may get odd behaviors or crashes when referencing deallocated memory?

 

xenoargh

Member
Oh, and!

@RujiK if you repeat the experiment with:

function test_this_1000(){
for(var i = 0; i < 1000; i++){
func_test(1,2,3);
}
}

function func_test(one,two,three) {
return(one);
}

How's that go? That's what I did to all of the Functions the minute I switched over to 2.3; took a while, but it's probably the right thing to do.
 

Roldy

Member
Oh, and!

@RujiK if you repeat the experiment with:

function test_this_1000(){
for(var i = 0; i < 1000; i++){
func_test(1,2,3);
}
}

function func_test(one,two,three) {
return(one);
}

How's that go? That's what I did to all of the Functions the minute I switched over to 2.3; took a while, but it's probably the right thing to do.
That's a good test but like @FrostyCat had mentioned, some of it will depend on what is being sent as arguments and what happens inside the function. If OP did infact disable garbale collection then things like strings or arrays will cause leaks. So OP may be mistaken about the 'functions' leaking memory but he is going to be leaking in all sorts of other ways.

e.g.

GML:
function test_this_1000(){

    for(var i = 0; i < 1000; i++){

    func_test("1","2","3");  // This will potentially leak if GC is off because these strings wont get cleaned up

    }

}



function func_test(one,two,three) {
    return(one);  
}
 

xenoargh

Member
So, wait, what?

If that's remotely so.... how are we supposed to destroy this data? And no wonder the GC's slow, lol.

We created that in the call to the function, in the case of "2" and "3"; we don't even have a pointer, let alone a way to manually manage that. The whole thing's kind of absurd, really. Functions should always, at their base, have their scope limited to the calling Instance, and should be cleared of all vars (or, in this case, var-like data like strings) when they terminate. If they have to return a value (say a Function returns to an Instance's Step()) that var in Step() should be a local copy or the pointer should be retained, imo.

GML:
function test_this_1000(){
    var abc = 0;
    abc[0] = 1;
    abc[1] = 2;
    abc[2] = 3;
    var two = "2";
    
    for(var i = 0; i < 1000; i++){
        func_test(abc,two,three);  // This will potentially leak if GC is off because these strings wont get cleaned up
    }
}

//Why would this leak?  One, Two and Three are pointers to mere data in the first Function's scope!
function func_test(one,two,"3") {
    return(one[1]);
}
What's being described here isn't the expected behavior. But that might explain some stuff.

It seems like it'd be easier/more rational to just put said mere data into a globally-scoped temp variable that gets wiped back to zero when the Function returns to the Instance. Since GMS is single-threaded, it's not like this is going to create a concurrency crisis; just reserve enough variables; 1024 oughtta be enough for anybody's project, and would cost hardly anything while zero'd. Then, if you build a Map in a Function call, it's not a memory leak... it's just a temp variable that gets swept, like pretty much every other language.

In the meantime... "don't perform operations in a Function call"... roger that! <starts refactoring>
 

Nocturne

Friendly Tyrant
Forum Staff
Admin
Okay, let's clarify a few things:

First the GC is currently only valid for functions/methods and structs. Also, the use of strings and local vars and arrays and stuff has no impact at all as they clean up after themselves already (this behaviour is the same now as it was pre-2.3).

Second, if you switch the GC off then it's to be expected that you get memory leaks if you are using functions and structs, BUT you can enable the garbage collector and call gc_collect() then wait a few frames and disable it again to collect those things. This means that you can specifically set up an object to do garbage collection at specific points in the game where it won't impact the FPS.

Finally, this thread has lots of speculation about things that really don't appear to have much to do with the OP, and since the OP hasn't replied to clarify what it is they are doing and whether they are indeed calling a script asset instead of the script function then I'd appreciate it if we stopped with the off topic chit chat about the GC and variables and things until the OP has had time to reply. These things can be discussed in the official 2.3 topic, or you can make your own topic to discuss them, but this topic belongs to @RujiK and as such we should only be discussing their specific issue. ;)


EDIT: Please see this post for information on what the GC collects: https://forum.yoyogames.com/index.p...eference-on-physics-object.79508/#post-472667
 
Last edited:

Roldy

Member
So, wait, what?

If that's remotely so.... how are we supposed to destroy this data? And no wonder the GC's slow, lol.

We created that in the call to the function, in the case of "2" and "3"; we don't even have a pointer, let alone a way to manually manage that. The whole thing's kind of absurd, really. Functions should always, at their base, have their scope limited to the calling Instance, and should be cleared of all vars (or, in this case, var-like data like strings) when they terminate. If they have to return a value (say a Function returns to an Instance's Step()) that var in Step() should be a local copy or the pointer should be retained, imo.

GML:
function test_this_1000(){
    var abc = 0;
    abc[0] = 1;
    abc[1] = 2;
    abc[2] = 3;
    var two = "2";

    for(var i = 0; i < 1000; i++){
        func_test(abc,two,three);  // This will potentially leak if GC is off because these strings wont get cleaned up
    }
}

//Why would this leak?  One, Two and Three are pointers to mere data in the first Function's scope!
function func_test(one,two,"3") {
    return(one[1]);
}
What's being described here isn't the expected behavior. But that might explain some stuff.

It seems like it'd be easier/more rational to just put said mere data into a globally-scoped temp variable that gets wiped back to zero when the Function returns to the Instance. Since GMS is single-threaded, it's not like this is going to create a concurrency crisis; just reserve enough variables; 1024 oughtta be enough for anybody's project, and would cost hardly anything while zero'd. Then, if you build a Map in a Function call, it's not a memory leak... it's just a temp variable that gets swept, like pretty much every other language.

In the meantime... "don't perform operations in a Function call"... roger that! <starts refactoring>
Well to determine how it all works would require some testing, and it would be 'undocumented' and may work differently on different platforms. But like you said for a string the local is just a pointer. So the pointer it self is popped off the stack, but the string it points too has to be GC'd. Think about:

GML:
function () {
var a = "some string";

return a;

}
Where does that 'literal' string exist? I don't believe there is a 'DATA' block in the runner, if there was then "some string" would exist there and would never need to be freed. But I believe, "some string" is actually put in the pool or heap, so it has to be GC'd once all references are gone.

EDIT: I did report a bug. In a new 2.3 project, with a single object in its Step event the following code will leak regardless if GC is enabled or not:

GML:
var _doLeak = true;

function testFunction(_a) {
    var _c = 0 + _a;    // You must reference _a for it to leak
}

for (var _i = 0; _i < 1000; _i++) {
  


    if (_doLeak) {
        testFunction("1");  // This leaks
    } else {
        testFunction(1);    // This does not
   }
}
 
Last edited:

xenoargh

Member
@Nocturne Fair enough. Meanwhile, I'll try enabling the GC in my game's main control Object under certain circumstances and see if there's any way to get reasonable performance and no crashes.
<tests> Nope, still crashy under the circumstances I reported to YYG. Ah well.
 

kraifpatrik

(edited)
GameMaker Dev.
GML:
function () {
var a = "some string";

return a;

}
Where does that 'literal' string exist? I don't believe there is a 'DATA' block in the runner, if there was then "some string" would exist there and would never need to be freed. But I believe, "some string" is actually put in the pool or heap, so it has to be GC'd once all references are gone.
These strings are statically defined in the global scope, similarly to const char* a = "some string";, and they cannot be freed. If you create a string using functions like string_copy, then those are reference counted and freed automatically when they are no longer in use, same as they were prior to GMS2.3, so the new garbage collection has no effect on them.
 

gnysek

Member
Did anyone created same code in 2.3 and 2.2 (or even 1.4) and compared how much memory they use? Cause I still see speculations, but no real project or screens with memory usage (especially from debugger).
 

Fanatrick

Member
If OP did infact disable garbale collection then things like strings or arrays will cause leaks.
Stop with the ignorance, there are at least 4 posts in this thread telling you exactly what GC is supposed to handle. If people would talk less out of their asses we would've figured it out by now :D
 

GMWolf

aka fel666
How are functions GCed?
Isn't a function a static thing?
Or do functions have captures and act like lambda functions in other languages?
 

RujiK

Member
Hey everyone, thanks for all of your comments. After some testing, I've found that GMS 2.3 has quite a few memory leaks besides functions that were affecting my project.

A list of functions with memory leaks:
  • sprite_get_uvs();
  • tileset_get_uvs();
  • matrix_build()
  • matrix_transform_vertex();
  • Running any function from a variable. (Like "var _func = actual_function" and then running "_func");
  • Untested, but probably any built in command that returns an array.
You can see any of these memory leaks yourself by running "repeat(1000) {val = sprite_get_uvs(sprite,0);}" or whatever and watch your memory explode. (gc_enable must be false)

Thanks for all of your comments.

EDIT: I have reported the bugs and yoyo has confirmed the issues.
 
Last edited:

chamaeleon

Member
You can see any of these memory leaks yourself by running "repeat(1000) {val = sprite_get_uvs(sprite,0);}" or whatever and watch your memory explode. (gc_enable must be false)
I wouldn't call it a memory leak if you disable the thing that is supposed to reclaim memory not needed anymore. The reference is not missing (as far as the memory manager is concerned, that is), it is just not being cleaned up at that particular point in time. Just because your code does not have any reference to the allocation does not mean the memory manager does not either.
 

RujiK

Member
@chamaeleon Read the previous post by @Nocturne :
First the GC is currently only valid for functions/methods and structs. Also, the use of strings and local vars and arrays and stuff has no impact at all as they clean up after themselves already (this behaviour is the same now as it was pre-2.3).
It's a bug. But even if what you said was correct, it would be absolutely terrible design to have some memory leaking commands that can't be cleaned manually.

I wouldn't care if the commands leaked if they had a way to clear it like ds_list_destroy or surface_free. (Or just gimme the 2.2 garbage collector back.)
 
Last edited:

chamaeleon

Member
@chamaeleon Read the previous post by @Nocturne :
It's a bug. But even if what you said was correct, it would be absolutely terrible design to have some memory leaking commands that can't be cleaned manually.

I wouldn't care if the commands leaked if they had a way to clear it like ds_list_destroy or surface_free.
If they are not reclaimed when gc is enabled either, it is certainly a memory leak/bug. If they are cleaned up when gc is enabled, I wouldn't consider them leaks. I just went by your comment "gc_enabled must be false", which I took to mean, if gc_enable is true, there is no explosion in memory usage, and the resources are freed.
 
Top