How can I get the full array from an extension asynchronously?

W

Wraithious

Guest
I have a script that gets each pixel's color from a sprite and modifies the color, then populates a single array with the values, this takes an exponentially long time for the script to finish depending on the size of the sprite, I want to make an extension that returns the completed array asynchronesly so the game won't freeze while the script is running, i've looked at some extension tutorials but they are mostly for internet stuff, how can i set up an extension to run the script and send the array back? I tried exporting the script to a gml file and loading it into a created extension, not sure how to set up the async part but without async the extension returns just one value of the array (so the resulting picture is one solid color) and the game still freezes(because it's not currently 'async enabled') please steer me in the right direction and can this actually be done?
 

jo-thijs

Member
What you ask for is possible, but you'll have to make the asynchronous part work in the extension itself, not in GameMaker.
However, the real issue here is that you have some code that runs in exponential time, while it sounds like it should be able to run in linear time.
Can you share the code you're currently using that's freezing the game?
 
W

Wraithious

Guest
yep, this is my script:
EDIT forgot the brackets in colsmake when i posted this
Code:
for(i=0;i<l;i+=1; )
{colspicked=draw_getpixel(m,n);//gets every pixels color
colsr=colspicked & 255;colsg=(colspicked >> 8)& 255;colsb=(colspicked >> 16)& 255;p=colsr+colsg+colsb;//seperates to rgb
if p<=255 cbit=1;if p>255 && p<=510 cbit=2;if p>510 cbit=3;//1=dark,2=medium,3=light
if(global.palrgb=1)//deligates the new colors
{if cbit=1 colsmake[i]=make_colour_rgb(255,colsg,colsb);
if cbit=2 colsmake[i]=make_colour_rgb(colsr,255,colsb);
if cbit=3 colsmake[i]=make_colour_rgb(colsr,colsg,255);}
if(global.palrgb=2)
{if cbit=1 colsmake[i]=make_colour_rgb(255,colsg,colsb);
if cbit=2 colsmake[i]=make_colour_rgb(colsr,colsg,255);
if cbit=3 colsmake[i]=make_colour_rgb(colsr,255,colsb);}
if(global.palrgb=3)
{if cbit=1 colsmake[i]=make_colour_rgb(colsr,255,colsb);
if cbit=2 colsmake[i]=make_colour_rgb(255,colsg,colsb);
if cbit=3 colsmake[i]=make_colour_rgb(colsr,colsg,255);}
if(global.palrgb=4)
{if cbit=1 colsmake[i]=make_colour_rgb(colsr,255,colsb);
if cbit=2 colsmake[i]=make_colour_rgb(colsr,colsg,255);
if cbit=3 colsmake[i]=make_colour_rgb(255,colsg,colsb);}
if(global.palrgb=5)
{if cbit=1 colsmake[i]=make_colour_rgb(colsr,colsg,255);
if cbit=2 colsmake[i]=make_colour_rgb(colsr,255,colsb);
if cbit=3 colsmake[i]=make_colour_rgb(255,colsg,colsb);}
if(global.palrgb=6)
{if cbit=1 colsmake[i]=make_colour_rgb(colsr,colsg,255);
if cbit=2 colsmake[i]=make_colour_rgb(255,colsg,colsb);
if cbit=3 colsmake[i]=make_colour_rgb(colsr,255,colsb);}
if m<drwatx+j && n<=drwaty+k m+=1;if(m=drwatx+j){m=drwatx;if n<drwaty+k n+=1;}
if(i>=l-1){m=drwatx;n=drwaty;colorize=1;}}//activates the draw event when finished
In the for loop, l is the number of pixels in the image,
the create event of the object that calls the script (or extension) sets all that up, this is the create event code:
Code:
if(global.graytocol1=1 && instance_exists(base_picture))//base picture
{j=base_picture.bbox_right-base_picture.bbox_left;//gets width
k=base_picture.bbox_bottom-base_picture.bbox_top;//gets height
drwatx=base_picture.bbox_left//;gets x position on screen
m=base_picture.bbox_left;
drwaty=base_picture.bbox_top;//gets y position on screen
n=base_picture.bbox_top;
l=(j*k);//gets total number of pixels
depth=base_picture.depth-10;
}
for(i=0;i<l;i+=1){colsmake[i]=0;}// sets aray index to hold all pixels
oh, and the arrow keys choose the scheme and sets variable global.palrgb,
there are 6 choices of converting light medium and dark areas of the image
to different colors

then when the script is finished the draw event draws over the sprite and then the picture can be saved
draw event:
Code:
if(colorize=1 && global.effect=8)
{for(o=0;o<l;o+=1;)
{draw_point_colour(m,n,colsmake[o]);
if m<drwatx+j && n<=drwaty+k m+=1;
if(m=drwatx+j){m=drwatx;if n<drwaty+k n+=1;
if n=drwaty+k n=drwaty;}}
}
 
Last edited by a moderator:

jo-thijs

Member
That does not run in exponential time though, it is linear.
I would suggest to let the profiler do its job and determin what functions take up most of the time.
I'm guessing though, it will be draw_point_colour and surface_getpixel.

What's the size of the surfaces you work with?

I can optimize your code a bit, but without fundamental changes to how it works or what it does, my optimizations won't be anything significant.

I remember you were reading everything from PNG files.
It might be better to read data directly from those files.
It'll also be easier to create an extension for it this way.
Such an extension would have 3 functions:
1) A function that starts an asynchronous operation that reads in a PNG file and processes it.
2) A functioon that checks if the previous function has finished yet.
3) A function that returns the result (one by one?) when the first function has finished.
 
R

rui.rosario

Guest
[I'll be honest, I just skimmed the topic]

If you want to send data from GM to your extension with the sprite data, then I suggest:
- Draw the Sprite to a surface
- Transform the Surface into a buffer
- Pass the buffer into the extension
- The extension makes a copy of the buffer and initiates a thread that will process it, return a handle to GM
- Each step event poll the extension with the handle it returned until it returns as finished
- Once finished, call a method from the extension to copy the modified buffer back into GM

EDIT: I say copy the buffer to avoid concurrency issues, of course that if the buffer is very large you'll have to find another way
 
W

Wraithious

Guest
That does not run in exponential time though, it is linear.
I would suggest to let the profiler do its job and determin what functions take up most of the time.
I'm guessing though, it will be draw_point_colour and surface_getpixel.
Its definatly draw_get_pixel that takes the time, a d sorry what i meant by exponential is the time it takes to run the script vs the size of the image.
I'm just using the default application surface where the sprite is being drawn with the draw_get_pixel function, i tried surface_getpixel and it actually took longer, and i also tried deactivating all instances except the script callinginstance but that didn't help either, so i want to just do the scrilt in an extension so i can create a working loading bar to display to the player until te script is done.
The size of the image is what is unknown, the player can load any image they want to, but it will allways be .png and im going to limit the size to either 1024x768 or 640x480.
 
R

rui.rosario

Guest
I'm just using the default application surface where the sprite is being drawn with the draw_get_pixel function, i tried surface_getpixel and it actually took longer, and i also tried deactivating all instances except the script callinginstance but that didn't help either, so i want to just do the scrilt in an extension so i can create a working loading bar to display to the player until te script is done.
Any of the getpixel functions take forever. A more performant version of it is to turn the surface into a buffer and read the pixel data directly from the buffer.

Also, if you just need a loading bar then you don't need to do it asynchronously, you just need to say: "I'll process 2048 (or any other value) pixels each step until I'm out" and then you create the loading bar based on the number of pixels read vs the total number of pixels. It can be done all on the GM side (although asynchronously in an extension would be more performant)
 
W

Wraithious

Guest
Also, if you just need a loading bar then you don't need to do it asynchronously, you just need to say: "I'll process 2048 (or any other value)
I tried making a loading bar but the game freezes while the script runs including the loading bar, thats why i need to use
async, the only thing i need to return to gms is the array after it's built in the script, thats what im askimg tho is if an array with all it's values can be returned to gms or will it only return one value in the array? I know when using a split function you can just say return colsmake without the brackets and read it in a for loop as colsmake[ i ] , would that work in an extension?
 
R

rui.rosario

Guest
In an extension you should use buffer (see buffer_get_address).
 

jo-thijs

Member
I also meant exponential and linear with respect to the image size.
That code performs in linear time.

I'm a bit surprised that surface_getpixel would be this slow.
I'd never have guessed it'd be slower than draw_getpixel.
I also think it is wierd in that case that the manual says draw_getpixel is very slow,
but it says nothing for surface_getpixel.

Anyway, I'll try to create an extension for this as soon as I've got time for it.
I need to know what you want it to do exactly however.
Do you want it to read in filenames of PNG's?
Do you want it to read in buffers? (in that case, the extension would work as rui.rosario described)
Do you want the extension to process a single image at the time or to process multiple images at once?
Do you want extra functionalities besides the ones you already described?
For instance, to have some adaptability/flexibility?

EDIT:
Also, on which targets does it HAVE to work?

EDIT:
Looking at the PNG file format, it might be easier to just write a shader for it.
 
Last edited:
W

Wraithious

Guest
Do you want it to read in filenames of PNG's?
Do you want it to read in buffers? (in that case, the extension would work as rui.rosario described)
Do you want the extension to process a single image at the time or to process multiple images at once?
Do you want extra functionalities besides the ones you already described?
For instance, to have some adaptability/flexibility?
The manual does say that both forms of getpixel are slow. to quote the manual:
Description
This function can be used to get the colour of a specific pixel from a surface, using the local coordinates of the surface where (0,0) is the top left corner. This function should not be used very often as it is extremely slow and may cause a pause in your game.
NOTE: When working with surfaces there is the possibility that they can cease to exist at any time due to them being stored in texture memory. You should ALWAYS check that a surface exists using surface_exists before referencing them directly. For further information see Surfaces.
Example:
col = surface_getpixel(surf, 56, 78 );

If it could be made into a shader that would be the best way,
I can make an extension, all I need to know is how to call the extension to run the script asynchronessly, and how to return the result array from the extension in an async event to gms and what async event to retreive (use social? use save/load?) my script works perfectly fine and does what I want, I just need it to run in the background and not block GMS thats why i wanted to use an async event. I suspect if a shader is capable of what my script does it would be wayyy faster tho but we're talking about coloring a black and white image not the other way around it's totally different.

I need it to work on windows and android (I allready have it working fine on both, but no async)
This is what my script does:
The user loads in a black and white picture
then selects weather the RGB elements to be generated go to light areas, medium areas or dark areas, such as if he picks red for dark, green for medium and blue for light my script takes all the heavilly shaded areas and converts them to red, the medium shaded areas convert to green and the lightly shaded areas convert to blue.

here's some screenshots of the working results:
RGB.png
RGB1.png
GRB.png
GRB1.png


how it works:
FOR EVERY PIXEL:
1. gets pixel color from the picture starting at the left top pixel.
2. records the color of the pixel to a variable
3. splits the result into the rgb values and puts each in a seperate variable
4. adds these 3 variable values together and: if less than= 255 sets a variable for dark areas, if between 255 & 510 sets a variable for medium areas, if > 510 sets a variable for dark areas
5. reads the users choice and sets the array variable at the appropriate index to the output pixel color red, green, or blue with a light, medium, or dark amount of color, the light medium values are implimented by using the original values for those 2 rgb elements. (the rgb element chosen as dark is set to 255)
6. when all pixels are read and adjusted it sets the variable colorize to 1 so the drawing event is activated using the completed array

as you can see I only need to send the script to async so the game doesnt freeze, the script does all the work, and then i only need to retreive the finished array, that's all i need to do and all i need to know is how to return an aray from async
 

Attachments

jo-thijs

Member
Using shaders, I'd suggest you'd do it like this:
Have 6 shaders, 1 for every value of global.palrgb.
In each shader, you'd use the default vertex shader and use something like this fragment shader:
Code:
//
// Simple passthrough fragment shader
//
varying vec2 v_vTexcoord;
varying vec4 v_vColour;

void main()
{
    gl_FragColor = v_vColour * texture2D( gm_BaseTexture, v_vTexcoord );
   
    // global.palrgb == 1
    switch (ceil(gl_FragColor.r + gl_FragColor.g + gl_FragColor.b)) {
    case 0.0:
    case 1.0: gl_FragColor.r = 1.0; break;
    case 2.0: gl_FragColor.g = 1.0; break;
    case 3.0: gl_FragColor.b = 1.0; break;
    }
}
(there is some other variant of this shader I can try if this shader is too slow, but it apparently takes me quite some time to get that one done well, so I'll just post this until it's proven insufficient)

You would then let a switch structure over global.palrgb decide which shader gets executed.

As for the extension, the asynchronous part depends on what language you're using.
As for returning the result in an array, that's impossible for as far as I know,
but you can create functions that behave like typical iterator methods (an init function to calculate the result, a check function to check if it is done (and if so, how large the resulting array is), a next function to get the next result value).
Alternatively, I think you can also write the result to a buffer of which you've passed the pointer.
 
W

Wraithious

Guest
Ok i can give the shader a try, for the extension im just using my exported script so the language is GML which is great because every platform available can use it.
As far as the size of the array that is allways known as soon as the image is loaded because the index is allways the image width x image height.
 
W

Wraithious

Guest
I believe I have a solution to this, since it's impossible to get an array from an async event I'm going to split the picture into chunks and send them to my script one by one, I'm not a mathamagician so this allready has become quite complicated but I've developed a plan to go by, first I need to change my variables m,n,l,drwatx and drwaty to arrays, and add at least 4 more variables- q,r,s,t.the new arrays drwatx[ q ] and drwaty[ r ] are going to have static values relitive to their starting x,y positions. It turns out there are going to be 3 seperate case senerios to take into account, 1- the image is <= 50 x 50 pixels in size. 2- the image is an even multiple of 50 x 50 pixels in size. and 3- the image is an uneven multiple of 50 x 50 pixels in size. the spoiler below outlines what I have to do, but like I said I'm not a mathemagician so this is going to be a little insane to make it work:
evilplan.png
 
W

Wraithious

Guest
unfortunatly the problem is it still has to execute the script to get the pixel values, which takes too long
 
R

rui.rosario

Guest
I believe I have a solution to this, since it's impossible to get an array from an async event I'm going to split the picture into chunks and send them to my script one by one, I'm not a mathamagician so this allready has become quite complicated but I've developed a plan to go by, first I need to change my variables m,n,l,drwatx and drwaty to arrays, and add at least 4 more variables- q,r,s,t.the new arrays drwatx[ q ] and drwaty[ r ] are going to have static values relitive to their starting x,y positions. It turns out there are going to be 3 seperate case senerios to take into account, 1- the image is <= 50 x 50 pixels in size. 2- the image is an even multiple of 50 x 50 pixels in size. and 3- the image is an uneven multiple of 50 x 50 pixels in size. the spoiler below outlines what I have to do, but like I said I'm not a mathemagician so this is going to be a little insane to make it work:
You could do a different approach:

Taking the divide and conquer method, you divide the image into X number of blocks (X being the number of getpixel you want to call in a single step) You then instantiate a specific worker object that has properties for blockXStart, blockYStart, blockWidth and blockHeight. The instance would then process one pixel each step, taking into account the processing window it has (from blockXStart to blockXStart + blockWidth and the same to the Y). You would show a processing bar while any of those instances existed, and once none existed the task would be over (for this the instances needs o destroy itself when its work is finished)


Not sure I made myself clear :p
 
Top