SOLVED How to change the colour of an outline shader?

S

shadow7692

Guest
I've followed Shaun Spalding's tutorial at youtube[dot]com/watch?v=zWrpHbc6fmc, which basically sets up a one-pixel outline for sprites using shaders. Currently, the outline is black. How do I change this?

I've tried messing with gl_FragColor but it either messes up the transparency (no longer an outline but a filled in version) or it tints the entire sprite that colour. Below is the code in shOutline.fsh.

Code:
varying vec2 v_vTexcoord;
varying vec4 v_vColour;
uniform float PixelHeight;
uniform float PixelWidth;

void main()
{
    vec2 OffsetX;
    OffsetX.x = PixelWidth;
    vec2 OffsetY;
    OffsetY.y = PixelHeight;
  
    float Alpha = texture2D(gm_BaseTexture, v_vTexcoord).a;
  
    Alpha += ceil(texture2D(gm_BaseTexture, v_vTexcoord + OffsetX).a);
    Alpha += ceil(texture2D(gm_BaseTexture, v_vTexcoord - OffsetX).a);
    Alpha += ceil(texture2D(gm_BaseTexture, v_vTexcoord + OffsetY).a);
    Alpha += ceil(texture2D(gm_BaseTexture, v_vTexcoord - OffsetY).a);
  
    gl_FragColor = v_vColour * texture2D( gm_BaseTexture, v_vTexcoord );
    gl_FragColor.a = Alpha;
}
 
Last edited by a moderator:

gnysek

Member
The response for this question was in second comment of the video you mentioned. As this simply modify alpha transparency from 0 to 1, that must mean, that transparent pixels are by default black and 100% transparent.

So...

To change color, you need to fill "empty" space with the color you need, but set alpha to 0. To make this, select the color you want in sprite editor, and double click on bigger rectangle with that color, there will be "Alpha":

1586527172779.png

Then paint the whole empty space with that color. You won't see a change until game is run.
 
S

shadow7692

Guest
The response for this question was in second comment of the video you mentioned. As this simply modify alpha transparency from 0 to 1, that must mean, that transparent pixels are by default black and 100% transparent.

So...

To change color, you need to fill "empty" space with the color you need, but set alpha to 0. To make this, select the color you want in sprite editor, and double click on bigger rectangle with that color, there will be "Alpha":

View attachment 30024

Then paint the whole empty space with that color. You won't see a change until game is run.
Thank you for the explanation.

Is there a way to do this via code? I want multiple colours and manually doing this for each spritesheet isn't as modular as I hoped for.

EDIT: Just tried exactly the way you said but the outline remains black when Alpha is 0. I tried an Alpha of 1 and the sprite suddenly has a solid white box background. Any reasons why this could be? Also I am using draw_sprite_part if that makes a difference.
 
Last edited by a moderator:
Code:
    float Initial_Alpha = texture2D(gm_BaseTexture, v_vTexcoord).a;
  
    float Alpha = Initial_Alpha;
    Alpha += ceil(texture2D(gm_BaseTexture, v_vTexcoord + OffsetX).a);
    Alpha += ceil(texture2D(gm_BaseTexture, v_vTexcoord - OffsetX).a);
    Alpha += ceil(texture2D(gm_BaseTexture, v_vTexcoord + OffsetY).a);
    Alpha += ceil(texture2D(gm_BaseTexture, v_vTexcoord - OffsetY).a);
  
    gl_FragColor = v_vColour * texture2D( gm_BaseTexture, v_vTexcoord );
    gl_FragColor.a = Alpha;
    gl_FragColor.rgb = mix( gl_FragColor.rgb, outline_Color, float(Initial_Alpha==0.0) );
I haven't tested that, but I believe it will work. Basically, it just replaces the default sprite and blend colour with "outline_Color" (which you define, it must be a vec3), if the initial alpha was 0.
 

NightFrost

Member
I haven't tested that, but I believe it will work.
I was just doing some stuff with shaders so I tested this out and yes, it works. Unless of course the pixels are right against the edges of texture region (or texture page edges) but I faintly recall the tutorial did something to make sure there always was empty pixels at the edges of the space.

One thing that'll probably trip people new to shaders there is the "trick" with mix ratio. The comparison of Initial_Alpha==0.0 returns a boolean truthiness which gets converted to float, so it always is either zero or one. Color mix is thus either fully original color or fully outline color.
 

sp202

Member
There are multiple ways to fix the edges issue. In my opinion, from best to worst:

- Using a vertex shader to increase the available space
- Drawing the sprite onto a slightly larger surface and running that through the shader
- Messing with texture settings to ensure there is sufficient padding
 

NightFrost

Member
Hey everyone, very new to shaders, and I am wondering exactly how to go about doing this bit:
You either declare the vec3 variable in the shader itself, or if you want to send the value from GameMaker, declare it as uniform in the shader and use GML's shader uniform commands to send the value.
 
You either declare the vec3 variable in the shader itself, or if you want to send the value from GameMaker, declare it as uniform in the shader and use GML's shader uniform commands to send the value.
I just wanna boost this response. I'll also say that changing uniform variables and gpu settings will break draw batches. Depending on what the needs are, the color can be set in the vertices themselves, but this is definitely the simplest option.
 
Top