Directional Edge Shader

Luke Peña

Member
I am making a pixel art game and I want to be able to edge-light clouds from any direction. Here is the effect I'm after:

I want to be able to:
  • supply a direction
  • supply a thickness
  • supply a color
And then it would color it like above.

I honestly have NO idea how to work with shaders, so here is what I have so far:
Code:
varying vec2 v_vTexcoord;
varying vec4 v_vColour;
uniform float pixelH;
uniform float pixelW;
uniform float set_r;
uniform float set_g;
uniform float set_b;

void main()
{

    vec2 offsetx;
    offsetx.x = pixelW;
    vec2 offsety;
    offsety.y = pixelH;
    
    float rr = set_r;
    float gg = set_g;
    float bb = set_b;
    
    gl_FragColor = v_vColour * texture2D(gm_BaseTexture, v_vTexcoord );
    gl_FragColor.r *= ceil( texture2D( gm_BaseTexture, v_vTexcoord + offsety).a);
    gl_FragColor.g *= ceil( texture2D( gm_BaseTexture, v_vTexcoord + offsety).a);
    gl_FragColor.b *= ceil( texture2D( gm_BaseTexture, v_vTexcoord + offsety).a);
    
}

Here is how that turns out:


So right now I can only color it from above. And I don't know how to get an exact color, I only know how to flat change it to black. Plus since it is just displacing its alpha to get the color, it can leave odd spots like the "oops" in the image that don't work.

If anybody is awesome with shaders and could help out, I'd really appreciate it!
 

Bingdom

Googledom
What you want to work with is called a normal light shader.

I didn't write this code, I just pinched it somewhere from this very forums. I haven't tested it either, but it looks like it should work. I think this does what you're asking for. I'm guessing that the bump map is the place where you want the light to shine. Probably just make it the colour you want it to be.

Code:
varying vec2 v_vTexcoord;
varying vec4 v_vColour;
uniform sampler2D bump; //bump map
uniform float angle; //Angle in degrees
uniform float texsize; // size of the texture in px
void main()
{
    float A = radians(angle-45.0);//Angle in radians
    vec2 Offset = vec2(1.3/texsize*sin(A),-1.3/texsize*cos(A));
    vec4 Emboss = texture2D( bump, v_vTexcoord - Offset ) - texture2D( bump, v_vTexcoord+Offset ) ;
    gl_FragColor = v_vColour * texture2D( gm_BaseTexture, v_vTexcoord ) + Emboss ;
}
 

Luke Peña

Member
I actually ended up getting mine to work mostly. It has a few imperfections I can ignore. Here is what I cam up with:

Code:
varying vec2 v_vTexcoord;
varying vec4 v_vColour;
uniform float pixelH;
uniform float pixelW;
uniform float set_r;
uniform float set_g;
uniform float set_b;

void main()
{

    vec2 offsetx;
    offsetx.x = pixelW;
    vec2 offsety;
    offsety.y = pixelH;
    
    float alpha = (ceil( texture2D( gm_BaseTexture, v_vTexcoord + offsety + offsetx).a));
    float add = 1.0-(alpha);
    
    float rr = set_r*add;
    float gg = set_g*add;
    float bb = set_b*add;
    
    vec4 col = v_vColour * texture2D(gm_BaseTexture, v_vTexcoord );
    
    gl_FragColor = col;
    gl_FragColor.r = (col.r * alpha) + rr;
    gl_FragColor.g = (col.g * alpha) + gg;
    gl_FragColor.b = (col.b * alpha) + bb;
    
}

And here is the result:


So the issue right now is that I'm passing in three different values to get the color. R,G, and B all have to be separate values. Which it would be much better to pass through one single color and get the same result.

Right now, my guess is that I could pass in a color, bring it in as a vec3, and then get the col.r, col.g, and col.b from that vector. I'm gonna give that a try, but if anybody has a better way, I'll try anything else.
 
One way would be to just use v_vColour.

If you draw with an ext function like draw_sprite_ext the colour you pass in there is v_vColour in the shader.

To access the colours inside the fragment shader you can use v_vColour.rgb (or i.e. v_vColour.r if you just want to get the red)

Then of course you would need to remove v_vCoilour from this line completely though:
Code:
vec4 col = v_vColour * texture2D(gm_BaseTexture, v_vTexcoord );

Another way:
If you want to keep v_vColour for some reason you can pass in all three colours with one line. In your draw event you'd write:
Code:
shader_set_uniform_f(handler, red, green, blue);
and in the fragment shader:
Code:
uniform vec3 light_colour;
(If you want to learn more about this, click the link to the shader tutorial series in my signature)
 
I want to show you another way of writing your shader.
not shown: varying variable declarations, nor multiplication by the v_vColour.
Code:
vec2 offset;
vec3 shade_color;
void main() {
    float mix_amount = ceil( texture2D( gm_BaseTexture, v_vTexcoord + offset ).a );
    vec4 base_color = texture2D( gm_BaseTexture, v_vTexcoord );
    gl_FragColor = vec4( mix( shade_color, base_color.rgb, mix_amount ), base_color.a );
}
to point out first that the operations that you perform on a component wise basis can be combined (in most cases).
and second that there is a built in function called "mix" which you can use to duplicate the process of "a * (1-c) + b * c"
 

Luke Peña

Member
I want to show you another way of writing your shader.
not shown: varying variable declarations, nor multiplication by the v_vColour.
Code:
vec2 offset;
vec3 shade_color;
void main() {
    float mix_amount = ceil( texture2D( gm_BaseTexture, v_vTexcoord + offset ).a );
    vec4 base_color = texture2D( gm_BaseTexture, v_vTexcoord );
    gl_FragColor = vec4( mix( shade_color, base_color.rgb, mix_amount ), base_color.a );
}
to point out first that the operations that you perform on a component wise basis can be combined (in most cases).
and second that there is a built in function called "mix" which you can use to duplicate the process of "a * (1-c) + b * c"
Hey, that worked perfectly! Here is the new shader:
Code:
varying vec2 v_vTexcoord;
varying vec4 v_vColour;
uniform vec3 lightCol;
uniform float pixelH;
uniform float pixelW;

void main()
{
    vec2 offsetx;
    offsetx.x = pixelW;
    vec2 offsety;
    offsety.y = pixelH;
    
    float mix_amount = (ceil( texture2D( gm_BaseTexture, v_vTexcoord + offsety + offsetx).a));
    vec4 col = v_vColour * texture2D(gm_BaseTexture, v_vTexcoord );
    
    gl_FragColor = vec4 ( mix(lightCol,col.rgb,mix_amount), col.a);
}

So "mix" in GLSL is basically like a lerp but for more advanced values like vectors, right? Or just for any two values?
 
mix:
val1 & val2 must be of the same datatype. i.e. float, vec2, vec3, vec4
if the two vals are a vector, they get mixed component-wise so with a vec2 it's like this

mix(vecA, vecB, weight)
is the same as
mix(vecA.x, vecB.x, weight)
mix(vecA.y, vecB.y, weight)
combined

amount is always a float but can be below 0 or above 1

I was curious about this effect and played around a bit. If you got GMS2 I can export and post my version.

Pixel_Cloud.gif

But ultimately in most cases @Bingdom's reply is probably the best or most flexible option. With a normal map you'd be able to not only shade the border. you could also add light to inside the cloud like this:
pixelcloud.png

But I haven't worked with normals yet so I can't explain how to :)
 
Last edited:

Luke Peña

Member
mix:
val1 & val2 must be of the same datatype. i.e. float, vec2, vec3, vec4
if the two vals are a vector, they get mixed component-wise so with a vec2 it's like this

mix(vecA, vecB, weight)
is the same as
mix(vecA.x, vecB.x, weight)
mix(vecA.y, vecB.y, weight)
combined

amount is always a float but can be below 0 or above 1

I was curious about this effect and played around a bit. If you got GMS2 I can export and post my version.

View attachment 13782

But ultimately in most cases @Bingdom's reply is probably the best or most flexible option. With a normal map you'd be able to not only shade the border. you could also add light to inside the cloud like this:
View attachment 13783

But I haven't worked with normals yet so I can't explain how to :)
THAT IS SO MUCH BETTER THAN MINE. Yes, I do have GMS2. If you could send that to me, that'd be amazing!

EDIT: so right now, I have all of the clouds pointing their individual directions at the sun, and the brightness of the cloud's edge is based on their distance to the sun. It looks like in your version of the shader, each pixel has its own direction set on a pixel-by-pixel basis. Could the brightness of the edge ALSO be set on a pixel-by-pixel basis? So edges light themselves based on how far they are from the point-position, not based on how far the cloud's origin is.
 
Last edited:
The shader calculates the distance anyways. So to make the distance brighten the shade all I had to do it adding the inversion. But you'll see that inside the fragment shader. How much brighter and up to what distance can be changed with the macros in the header of the shader.

Here's the project to download:
https://www.dropbox.com/s/j6jkrg054t97lur/Pixel_Cloud_Shading.yyz?dl=0

And here the new preview:


The two clouds are one sprite btw to show it's on a per pixel basis.

But again. All that and more you could do with bingdoms tip and depending on what you want to do, his could be more performant.

Edit: if you cant use GMS2, here's the code:
https://www.dropbox.com/s/zhzysj1i480otee/obj_shade_pixel_cloud - code and sprite.zip?dl=0
 
Last edited:

Luke Peña

Member
The shader calculates the distance anyways. So to make the distance brighten the shade all I had to do it adding the inversion. But you'll see that inside the fragment shader. How much brighter and up to what distance can be changed with the macros in the header of the shader.

Here's the project to download:
https://www.dropbox.com/s/j6jkrg054t97lur/Pixel_Cloud_Shading.yyz?dl=0

And here the new preview:


The two clouds are one sprite btw to show it's on a per pixel basis.

But again. All that and more you could do with bingdoms tip and depending on what you want to do, his could be more performant.
Alright, here is the effect implemented in my game:


I had to tweak a few things a bit because my game is at a high resolution, so the shader had smooth edges on some parts of it. So I found a work around for that. Also, I found that I couldn't change the blend of the sprite being drawn. But since the clouds are a static color, I just fed it a second "cloud_col" and then set the rest of the image to use that.

But I also used your brightness variable to not only increase the brightness of the light, but to fade back into the original cloud color.
 

mako

Member
What you want to work with is called a normal light shader.

I didn't write this code, I just pinched it somewhere from this very forums. I haven't tested it either, but it looks like it should work. I think this does what you're asking for. I'm guessing that the bump map is the place where you want the light to shine. Probably just make it the colour you want it to be.

Code:
varying vec2 v_vTexcoord;
varying vec4 v_vColour;
uniform sampler2D bump; //bump map
uniform float angle; //Angle in degrees
uniform float texsize; // size of the texture in px
void main()
{
    float A = radians(angle-45.0);//Angle in radians
    vec2 Offset = vec2(1.3/texsize*sin(A),-1.3/texsize*cos(A));
    vec4 Emboss = texture2D( bump, v_vTexcoord - Offset ) - texture2D( bump, v_vTexcoord+Offset ) ;
    gl_FragColor = v_vColour * texture2D( gm_BaseTexture, v_vTexcoord ) + Emboss ;
}

I know this is an old thread, but I'm trying to learn to use shaders and this is exactly what I need, but I can't get it to work right. What exactly do I put into my "Create" and "Draw" events to call this shader? I've tried several different things, but when I run the game, I just get the background and none of the objects I need lit.
 

Bingdom

Googledom
I know this is an old thread, but I'm trying to learn to use shaders and this is exactly what I need, but I can't get it to work right. What exactly do I put into my "Create" and "Draw" events to call this shader? I've tried several different things, but when I run the game, I just get the background and none of the objects I need lit.
  • You need to get the handler of the shader uniforms by using shader_get_uniform and shader_get_sampler_index
  • Once you have the handler, you can use that to set the uniforms in the shader by using shader_set_uniform or texture_set_stage
  • To pass a surface/sprite into a sampler, you need to use the respective *_get_texture functions
 

mako

Member
@Bingdom: Awesome! So I was part of the way there. 😅 I only have one question now: what about the uniform float "texsize"? Is that the width of the texture, the height, or both multiplied?
 

Bingdom

Googledom
@Bingdom: Awesome! So I was part of the way there. 😅 I only have one question now: what about the uniform float "texsize"? Is that the width of the texture, the height, or both multiplied?
Ahh, a fatal flaw in their code.
You may want to use a vec2 instead of a float.

With the original code, it's expected you'll be using a square.

I have a little more time now, so here's the shader updated to support a vec2 instead of a float. This is still untested.
GML:
varying vec2 v_vTexcoord;
varying vec4 v_vColour;
uniform sampler2D bump; //bump map
uniform float angle; //Angle in degrees
uniform vec2 texsize; // size of the sprite/surface in px
void main()
{
    float A = radians(angle-45.0);//Angle in radians
    vec2 Offset = vec2(1.3/texsize.x*sin(A),-1.3/texsize.y*cos(A));
    vec4 Emboss = texture2D( bump, v_vTexcoord - Offset ) - texture2D( bump, v_vTexcoord+Offset ) ;
    gl_FragColor = v_vColour * texture2D( gm_BaseTexture, v_vTexcoord ) + Emboss ;
}
From memory, you may want to use sprite_get_width/height, and not the texture size itself.

The idea is that shaders use coordinates from 0-1. 0,0 is the top-left, and 1,1 being the bottom-right.
To make this pixel-perfect, we need to get the size of a pixel on the sprite/surface to use on the shader.
 

superschure

Member
The shader calculates the distance anyways. So to make the distance brighten the shade all I had to do it adding the inversion. But you'll see that inside the fragment shader. How much brighter and up to what distance can be changed with the macros in the header of the shader.

Here's the project to download:
https://www.dropbox.com/s/j6jkrg054t97lur/Pixel_Cloud_Shading.yyz?dl=0

And here the new preview:


The two clouds are one sprite btw to show it's on a per pixel basis.

But again. All that and more you could do with bingdoms tip and depending on what you want to do, his could be more performant.

Edit: if you cant use GMS2, here's the code:
https://www.dropbox.com/s/zhzysj1i480otee/obj_shade_pixel_cloud - code and sprite.zip?dl=0
Firstly, thanks for this. It's a godsend as I've been clacking keys trying to create something similar and have been coming up short. I'd like to ask if you'd know how to add a transparency channel to this shader? I've tried converting the vec3 "col_light" into a vec4 while adding a vec4 v_vColour but nothing came of it. I've also tried messing with the alpha in the draw event but that only affects the sprite and ignores the "illuminated area".

EDIT: I figured it out by watching @The Reverend's Youtube series on shaders, did this to add alpha...

GML:
//in the VERTEX add these lines on top...
attribute vec4 in_Colour;                  
varying vec4 v_vColour;

//and in the void main() area add this line...
v_vColour = in_Colour;

//in the FRAGMENT add this line on top...
varying vec4 v_vColour;

//and in the void main() area change this line
vec4 base_col = texture2D( gm_BaseTexture, v_vTexcoord );

//into this line...
vec4 base_col = v_vColour * texture2D( gm_BaseTexture, v_vTexcoord );

//and at the very end replace this line...
gl_FragColor = base_col;

//with this line..., "0.5" being the alpha...
gl_FragColor = vec4(base_col.rgb, base_col.a * 0.5);
 
Last edited:
Top