Legacy GM [Unsolved but Useless] Compress sprite to binary alpha values using shader

angelwire

Member
EDIT: After some tests, it turns out that the speed of buffer_get_surface for an 8x8 surface is pretty much the same speed as what it would be for a 256x256 surface. So not only will this not make my game faster, but it would slow it down. Never figured out why the shader didn't work but it's not even necessary any more.(hence the unsolved but useless)

I'm using GMS 1.4.1763

Apologies for the confusing title. Here's a slightly better explanation of what I'm trying to do:

What I need is a shader that will take a sprite and convert the alpha values to either 1(visible) or 0 (invisible).
Then the shader should store the values in the color information that it's drawing (the gl_FragColor's rgba). The color of the each pixel would be based on 32 of the 1 or 0 values. Basically, the first 8 pixels' visibility would be stored as binary information in the Red channel of a color, the second 8 would be stored in the Blue etc..

The idea is to draw a sprite to a surface and use buffer_get_surface, but I only need to know whether or not a pixel is visible or invisible. (I don't need to know the rgb or even how visible it is) So if I can use buffer_get_surface on a surface that's a quarter of the width and height I can save quite a bit of time.

So for example if I drew a 32x32 sprite the shader would compress it down to 1024 bits (whether or not each pixel is visible or invisible) which could fit into an 8x8 texture which would be 1/16th of the number of pixels that would need to be converted with buffer_get_surface.

Hopefully you can understand what I'm trying to do.

Here's what I have for the fragment shader, you may notice that I'm very good with shaders and not even use what I have: (but at least you might be able to understand what I'm trying to do)
Code:
varying vec2 v_vTexcoord;
varying vec4 v_vColour;
uniform float one_pixel; //This is being set to 1/sprite_width
float look_x = 0.0;
float cc[32]; //whether or not a pixel is visible or invisilble
float binary(float n1,float n2, float n3, float n4, float n5, float n6, float n7, float n8); //Returns an 8 bit number from the 8 given numbers
int ii = 0; //this is for the loop

void main()
{
    look_x = (v_vTexcoord.x * 32.0) + v_vTexcoord.y; //This keeps me from counting the same pixel multiple times
    while (ii < 31)
    {
    look_x += one_pixel; //Look for the next pixel over
    cc[ii] = sign(texture2D( gm_BaseTexture, vec2(look_x,(floor(look_x))*one_pixel)).a);
//Find the pixel alpha and move the position down if it has gone too far to the right
//I use sign() to take any value greater than 0 and turn it into a 1, but all the values of 0 stay as 0.
    ii+=1; //increment counter
    }
                             
    gl_FragColor = vec4(binary(cc[0],cc[1],cc[2],cc[3],cc[4],cc[5],cc[6],cc[7]),
binary(cc[8],cc[9],cc[10],cc[11],cc[12],cc[13],cc[14],cc[15]),
binary(cc[16],cc[17],cc[18],cc[19],cc[20],cc[21],cc[22],cc[23]),
binary(cc[24],cc[25],cc[26],cc[27],cc[28],cc[29],cc[30],cc[31]));
//Take all 32 of the values and convert them into 8-bit numbers for each part of the gl_FragColor
}

float binary(float n1,float n2, float n3, float n4, float n5, float n6, float n7, float n8) //This is the binary function
{
return n8 + n7*2.0 + n6*4.0 + n5*8.0 + n4*16.0 + n3*32.0 + n2*64.0 + n1*128.0;
}
When the shader is compiled it throws this error
array reference cannot be used as an l-value; not natively addressable, forcing loop to unroll
I have typed the loop out (by copying and pasting it 32 times) and the error was gone but nothing was drawn.
I've also tried passing in a sampler2D and using it instead of the gm_BaseTexture but that didn't work.

I was really hoping this would be simple but nothing I've tried seems to work. I either get an error thrown at me, nothing is drawn, or a blank white square is drawn.

Hopefully what I'm trying to accomplish is understandable, if not I'd be happy to explain anything that I haven't made clear. I would try to go into more detail but I feel like that could make things more confusing. I've worked out the math several times so I'm pretty confident it's right, but I could show my work if that would help.

Alternatively, if there's any way I can get buffer_get_surface to run faster or just collect the alpha values I would welcome that.

Thanks in advance for your help.

Edit: I updated parts of my code to reflect some changes that are being made
 
Last edited:

Tthecreator

Your Creator!
So if you'd remove the while it does work? I guess so.
if you remove the second cc[ii] line in the loop does it then not show the error?
Is the error a game maker error? In that case can you ignore it or are you forced to abort?

Probably not the most helpful answer, but if you could answer my questions maybe I could get an idea.
 

angelwire

Member
Thanks for the reply
The error is a gamemaker error and I am forced to abort.
If I keep the While and comment out the cc[ii] line in the loop then there is no error and nothing is drawn.
If I remove the while and copy/paste this 32 times then nothing is drawn:
Code:
    look_x += vv * one_pixel; //Look for the next pixel over
    cc[ii] = sign(texture2D( gm_BaseTexture, vec2(look_x,(floor(look_x))*one_pixel)).a);
    ii+=1; //increment counter
    vv+= 1.0; //increment the value that I use to multiply
And some info I forgot to share in the first post is that the "one_pixel" variable is set to 1 / the sprite width (which is a 32x32 sprite)
 

Tthecreator

Your Creator!
mm this is weird.
I do have a hypothesis, though I don't have enough knowledge of gpu architecture or assembly to confirm it.
What I think is happening is that it can't dynamically use a value in an array. I guess it has to preassign a specific memory location or something.
Perhaps the error you get isn't even that problematic, but game maker just forces you to quit anyway. (it just throws that type of error whenever the shader compiler returns something)
The weird thing is that nothing actually draws when you type everything out.
What result do you get when you remove the sign() part, is that something you would expect to get without the sign() part? Perhaps your error is in the 'vec2(look_x,(floor(look_x))*one_pixel)', or any variables adjacent to that.
 

angelwire

Member
Okay so I saw something super dumb in my code: incrementing look_x by vv * one_pixel and then incrementing vv each time was not at all what I should be doing. So I've changed it to look_x += one_pixel; and entirely removed the vv variable.

So now running this 32 times:
Code:
    look_x += one_pixel; //Look for the next pixel over
    cc[ii] = texture2D( gm_BaseTexture, vec2(look_x,(floor(look_x))*one_pixel)).a;
    ii+=1; //increment counter
But even with that fixed it doesnt work.

I've tried removing the sign() but there's still nothing being drawn.
 
H

HammerOn

Guest
The error happens because you are incrementing ii in an unpredictable way (for the compiler).
A for (int ii = 0; ii < 31; ++i) will work better because the compiler can guess the values of ii in the first line of the loop block.

Wouldn't better do it directly without a shader? Draw the sprite in a surface, use buffer_get_surface, and do the algorithm with the buffer functions. The shader will run 1024 for a 32x32 but you only need 32 iterations for the algorithm.
 

angelwire

Member
I've replaced the while loop with this:
Code:
    for (int ii = 0; ii<31; ++ii)
    {
    look_x += one_pixel; //Look for the next pixel over
    cc[ii] = texture2D( gm_BaseTexture, vec2(look_x,(floor(look_x))*one_pixel)).a;
    }
and the same error is popping up.

Wouldn't better do it directly without a shader? Draw the sprite in a surface, use buffer_get_surface, and do the algorithm with the buffer functions. The shader will run 1024 for a 32x32 but you only need 32 iterations for the algorithm.
The reason I need the shader is because I want to use buffer_get_surface on as small of a surface as possible. So I can draw a 32x32 sprite onto an 8x8 surface, and then use buffer_get_surface on the 8x8 surface which should (in theory) be 16x faster than using buffer_get_surface on a 32x32 surface. I guess this also brings up the question of whether or not using this shader would improve the performance that much compared to just using buffer_get_surface on something 4x the size.
 

angelwire

Member
If anyone can help, here's what I'm doing in the object's draw event:
Code:
x = mouse_x+64;
y = mouse_y+64;
shader_set(compression_shader)
shader_set_uniform_f(one_pixel,1/sprite_width);
draw_self();
shader_reset();
And here's the latest from what I have in my fragment shader:
Code:
varying vec2 v_vTexcoord;
varying vec4 v_vColour;
uniform float one_pixel;
float look_x = 0.0;
float cc[32]; //whether or not a pixel is visible or invisilble
float binary(float n1,float n2, float n3, float n4, float n5, float n6, float n7, float n8); //Returns an 8 bit number from the 8 given numbers

void main()
{  
    look_x = (v_vTexcoord.x * 32.0) + v_vTexcoord.y; //This keeps me from counting the same pixel multiple times

    for (int ii = 0; ii<31; ++ii)
    {
    look_x += one_pixel; //Look for the next pixel over
    cc[ii] = texture2D( gm_BaseTexture, vec2(look_x,(floor(look_x))*one_pixel)).a;
    }          

gl_FragColor = vec4(binary(cc[0],cc[1],cc[2],cc[3],cc[4],cc[5],cc[6],cc[7]),
binary(cc[8],cc[9],cc[10],cc[11],cc[12],cc[13],cc[14],cc[15]),
binary(cc[16],cc[17],cc[18],cc[19],cc[20],cc[21],cc[22],cc[23]),
binary(cc[24],cc[25],cc[26],cc[27],cc[28],cc[29],cc[30],cc[31]));
//Take all 32 of the values and convert them into 8-bit numbers for each part of the gl_FragColor
}

float binary(float n1,float n2, float n3, float n4, float n5, float n6, float n7, float n8) //This is the binary function
{
return n8 + n7*2.0 + n6*4.0 + n5*8.0 + n4*16.0 + n3*32.0 + n2*64.0 + n1*128.0;
}
Again, it's throwing me an error because of the loop, but even when I copy and paste it 32 times nothing is drawn. Any thoughts would be appreciated.
 

Bart

WiseBart
Hi,

If I understand correctly, you want that sgn value for each 32 consecutive pixels to be stored in a single pixel's bits.

To get the gain in efficiency you're aiming for (8x8 px texture instead of 32x32), you'll need a way to get only part of the render surface in the buffer. Otherwise you're still copying the full surface's contents to a buffer with buffer_get_surface.
Or perhaps not... You only need to draw to 8x8 pixels in total, yet the texture that goes in is actually larger in size (you're doing a loop on a range of pixels and write them to a single pixel). So in the ideal case, the texture you send in is full size, yet the render target only needs to be 8x8 px.

The error that's shown tells you that you cannot use a reference into an array on the left-hand side of an assigment.
So it's best to get rid of that array assignment and the array as a whole, together with that binary function.
You should be able to rewrite the while/for loop and assign directly to gl_FragColor (a vec4) inside the loop. You can index gl_FragColor as gl_FragColor[0], gl_FragColor[1], gl_FragColor[ii mod ...], ..., and combine this with a bit shift to write the value to the relevant bit. The first 8 values go into gl_FragColor[0], the next 8 go into gl_FragColor[1], ..., each of them with the appropriate bit shift (or multiplication by multiple of 2).
Something like this (I haven't worked out the math to determine the values of i and j):
Code:
for(int ii = 0;ii < 31;ii++) {    // Preferrably use for, since number of iterations is known in advance
    // Determine i and j
    
    // Get value and assign to correct bit in correct component of gl_FragColor
    float sgn = sign(texture2D( gm_BaseTexture, vec2(look_x,(floor(look_x))*one_pixel)).a);
    gl_FragColor[i] = gl_FragColor[i] | (sgn << j);
}
 
H

HammerOn

Guest
The reason I need the shader is because I want to use buffer_get_surface on as small of a surface as possible. So I can draw a 32x32 sprite onto an 8x8 surface, and then use buffer_get_surface on the 8x8 surface which should (in theory) be 16x faster than using buffer_get_surface on a 32x32 surface. I guess this also brings up the question of whether or not using this shader would improve the performance that much compared to just using buffer_get_surface on something 4x the size.
You are overcomplicating a simple operation.
Memory is fetched and cached in chunks. Chances are that the CPU will load a bigger chunk of bytes than what is needed for a 8x8 sprite. Therefore, there is no difference between small amounts like 256 bytes (8x8) or 4096 bytes (32x32). Even if it wasn't the case, a shader is fast but calling it is not. Setting the shader, uniforms, drawing something and resetting the render pipeline will be slower than using only buffers.

Also, why is speed a concern? If you are doing it just with sprites, you can process them a single time in the create event even before the game start for the player. I don't see a need to do it at real-time or being super fast at all.
 

angelwire

Member
@HammerOn you're totally right. I did some tests and running buffer_get_surface is practically the same speed on an 8x8 surface as it is on a 256x256. It never occurred to me to test that before because it seemed pretty straight forward, thank you for pointing that out to me. Although the shader not working is still bugging me, it would end up slowing down the game. So I guess I'll mark this post: [Unsolved but Useless]

Thank you @Tthecreator @Bart and @HammerOn
 
Top