• Hey Guest! Ever feel like entering a Game Jam, but the time limit is always too much pressure? We get it... You lead a hectic life and dedicating 3 whole days to make a game just doesn't work for you! So, why not enter the GMC SLOW JAM? Take your time! Kick back and make your game over 4 months! Interested? Then just click here!

Shaders Shaders and Surfaces. Matching wrong colors?

mattmls00

Member
I am using the below shader to try and put transparency on the shadows in my room. There are two main problems with it that I cannot figure out.

So what I am doing is drawing all instances to one surface, then drawing the light mask on another. Then I draw the light mask on top of the instances layer to hide objects that are not in the light. The shader is meant to decrease the alpha value of any black areas so that the player can still see their surroundings.

The issue is, the shader only matches a black pixel when I check for (rgb = 1, 1, 1) when it should be (rgb = 0, 0, 0) to my understanding. Also, the shader is applying transparency to my objects (which are flat red with black details) which is even more confusing because red shouldnt match all zeros or all ones for color value.

I've been playing around with this for a while now, but I can't figure out what it is doing. Even if I remove the lighting mask and just draw the objects, they still are drawn partially transparent even though they shouldn't match the color specified in my IF statement.

Any ideas what might be going on here?


Code:
varying vec2 v_vTexcoord;
varying vec4 v_vColour;

vec4 fColor;

void main()
{
    
    fColor.r = v_vColour.r;
    fColor.g = v_vColour.g;
    fColor.b = v_vColour.b;
    
    
    if(fColor.r == 1.0 && fColor.g == 1.0 && fColor.b == 1.0){
        fColor.a = 0.2;   
    }
    
    else{
        fColor.a = 1.0;   
    }
    
    
    gl_FragColor = fColor * texture2D( gm_BaseTexture, v_vTexcoord );
}
 
It isn't testing the color of a pixel at all. It is testing v_vColour, which if you are using the default vertex shader code, is the vertex colour. When drawing sprites, this has the same value as image_blend, or the colour argument to a draw_sprite function.
 

mattmls00

Member
It isn't testing the color of a pixel at all. It is testing v_vColour, which if you are using the default vertex shader code, is the vertex colour. When drawing sprites, this has the same value as image_blend, or the colour argument to a draw_sprite function.
Ok great, that explains what is going on. Thank You.

Is there a way to make it operate on the color of the pixels in the sprite?

EDIT: I may have found it. Is this how you would get the sprite colors into a vector?

Code:
vec4 texColor = texture2D(gm_BaseTexture, v_vTexcoord);
 
Yes.

Are you drawing sprites with this shader, or are you drawing a surface? If you are drawing sprites, are you drawing them onto a surface? I'm assuming you are drawing a surface.
 

mattmls00

Member
Yes.

Are you drawing sprites with this shader, or are you drawing a surface? If you are drawing sprites, are you drawing them onto a surface? I'm assuming you are drawing a surface.

I am drawing a surface.

I have the lighting part working now, I just have weird things happening with my sprite colors and alpha.
 

mattmls00

Member
So what is actually drawn onto this surface? Is it just black and white?
The whole process goes like this:

I draw all objects onto an "instances" surface

I then draw the lighting mask onto a 'lighting' surface so when i clear the screen with black it doesn't draw over my objects.

I then draw the lighting mask over top of the instances surface.

then I draw the instances surface to the main application surface. This is where I am using the shader.

Thank you for the help by the way.
 
Do you need to have a seperate surface for your instances? Wouldn't you be able to draw the light surface directly onto the application surface? The reason I ask, is that if you can do it this way, less can go wrong.
 

mattmls00

Member
Do you need to have a seperate surface for your instances? Wouldn't you be able to draw the light surface directly onto the application surface? The reason I ask, is that if you can do it this way, less can go wrong.
The reason I am doing it this way is because I only want the portion of the sprite that the light is touching to be visible. If I just draw them to the application surface, you can see them all the time when there is transparency on the 'darkness'.

This way the darkness in the lighting mask draws over the portions I want hidden. There might be better ways to do it, this is just what I came up with.
 
Okay, before I say anything else I'll just answer your question.

vec4 base_color = texture2D( gm_BaseTexture, v_vTexcoord );

float is_black = float(base_color.rgb == vec3(0.0));
or
float is_black = float(dot(base_color.rgb,base_color.rgb)<0.001);

The second option might be better in cases where there might be a little noise in your lighting system. You can adjust the tollerance. But because you are first drawing onto an instance surface, I feel like a lot can still go wrong.

---------------------------

But let me ask if I'm understanding you correctly. You want instances not in light to be totally invisible, but you want backgrounds to be partially visible even when in shadw?
 

mattmls00

Member
Yes, you have the right idea of What I am trying to do
Okay, before I say anything else I'll just answer your question.

vec4 base_color = texture2D( gm_BaseTexture, v_vTexcoord );

float is_black = float(base_color.rgb == vec3(0.0));
or
float is_black = float(dot(base_color.rgb,base_color.rgb)<0.001);

The second option might be better in cases where there might be a little noise in your lighting system. You can adjust the tollerance. But because you are first drawing onto an instance surface, I feel like a lot can still go wrong.

---------------------------

But let me ask if I'm understanding you correctly. You want instances not in light to be totally invisible, but you want backgrounds to be partially visible even when in shadw?
Yes, you have the right idea of What I am trying to do
 
I think it would be a good idea to draw the light and instance surface together using a shader with multiple texture samplers, instead of first drawing the lighting surface onto the instance surface.

It's one fewer surface draw, and you don't have to worry about the instance colors interfering with the light mask.

Code:
varying vec2 v_vTexcoord;
uniform sampler2D light_tex;
void main() {
    vec4 inst_color = texture2D( gm_BaseTexture, v_vTexcoord );
    vec4 light_color = texture2D( light_tex, v_vTexcoord );
    float light_level = light_color.r; //assuming light level is either 0 or 1.
    gl_FragColor = vec4( mix(light_color.rgb, inst_color.rgb, light_level), (inst_color.a-0.8)*light_level + 0.8);
}
Code:
shader_set(sh_lighting);
texture_set_stage(shader_get_sampler_index(sh_lighting,"light_tex"),surface_get_texture(light_surf));  //can get sampler index in create event (but not surface texture, because the surface texture index can change!)
draw_surface(inst_surf,0,0);
shader_reset();
 

mattmls00

Member
This is interesting. A little over my head as far as shaders go, but I think I see what is going on.

So in your example, I would still draw my instances and lighting mask to their own surface, but the shader would render them both in one draw call right?

My lighting mask is created by clearing the screen to black, and then using a subtract blend mode to reduce the alpha in certain areas. So since my light level is not always 0 or 1, can I replace light_color.r with light_color.a and have similar results?

I am hoping this code you have given solves the issue I am having where the instances are being drawn extremely dark in low light areas.
 
It will produce weird results if the light is ever anything other than 0 or 1. Like when will you ever encounter a situation where you can see the floor partially beneith an opaque object, just because the object is in partial shadow? It would make more sense if the object remained visible, but was just totally black when in shadow.
 
Top