Legacy GM Multiple Render Targets [Shaders]

[Binsk]

Basics:

GM Version: Studio v1.4
Target Platform: Windows, Mac OS X, Linux
Expectations: You should understand the very basics of shaders and surfaces


Summary:

This tutorial covers how to set up Multiple Render Targets (or MRTs, for short). That is to say, how to render to multiple surfaces at once. Both GLSL and HLSL shaders are covered.


Tutorial:

When rendering a sprite or a model sometimes, things can get a little taxing on the GPU. If you need several similar but not exactly the same effects to take place on a render, it would not be unexpected to render it several times; once for each effect. The bummer about doing this is that often times you are repeating a lot of calculations for only a slight difference in the pixel / fragment shader. When you come across this scenario where you are using a lot of the same data for these effects, MRTs can come in really handy. Essentially you just perform a single pass but output results to several areas such as, in our case, surfaces.

In this tutorial we are going to make a very simple render. With one pass we are going to render our image normally to one surface, and a black stencil of the image to a second surface. We will first set up the shader in GLSL, then HLSL, then finally we will cover the GML code to pass our data in correctly. If you do not care about one shader language or the other then you can skip to the one you are interested in.

GLSL:

Let's create a basic pass-through shader. Because we will not be modifying the vertex shader at all, we will just focus on the pixel shader. Generally when said and done, it would look something like this:

// Our texture coordinates:
varying vec2 f2_texcoord;

void main()
{
    // Set the output color:
    gl_FragColor = texture2D( gm_BaseTexture, f2_texcoord );
}

This should be rather familiar. gl_FragColor is the important piece that outputs our final color data. But what if we had two outputs?

    vec4 _f4_outputNormal = texture2D( gm_BaseTexture, f2_texcoord );
    vec4 _f4_outputStencil = vec4(0., 0., 0., _f4_outputNormal.a);

    gl_FragColor = _f4_outputNormal;
    // Now what?

We can set the color to one thing, but how do we do two? In GLSL there is another built-in value that we can use called gl_FragData. You can consider gl_FragData as an array of gl_FragColors in a sense. When you write a GLSL shader you can pick which you wish to use, gl_FragColor or gl_FragData. You should never attempt to use them together in the same shader. So how would we use gl_FragData? It is rather simple. In the end our pixel shader would look like this:

// Our texture coordinates:
varying vec2 f2_texcoord;

void main()
{
    vec4 _f4_outputNormal = texture2D( gm_BaseTexture, f2_texcoord );
    vec4 _f4_outputStencil = vec4(0., 0., 0., _f4_outputNormal.a);

    gl_FragData[0] = _f4_outputNormal; // Output to surface 1
    gl_FragData[1] = _f4_outputStencil;  // Output to surface 2
}

That is really all there is to it. The shader setup is very simple. Let's move on!

HLSL:

Let's create a basic pass-through shader. Because we will not be modifying the vertex shader at all, we will just focus on the pixel shader. Generally when said and done, it would look something like this:

// Our texture coordinates:
struct input_t
{
    // Our texture coordinates:
    float2 f2_texcoord : TEXCOORD0;
}

struct output_t
{
    // Output Result:
    float4 f4_color : COLOR;
}

void main(in input_t input, out output_t output)
{
    // Set the output color:
    output.f4_color = tex2D( gm_BaseTexture, input.f2_texcoord );
}

This should be rather familiar. output.f4_color is the important piece that outputs our final color data. But what if we had two outputs?

    float4 _f4_outputNormal = tex2D( gm_BaseTexture, input.f2_texcoord );
    float4 _f4_outputStencil = float4(0., 0., 0., _f4_outputNormal.a);

    output.f4_color = _f4_outputNormal;
    // Now what?

We can set the color to one thing, but how do we do two? In HLSL setting multiple render targets is a little more intuitive than GLSL. If you want to render a single color, you just specify :COLOR in the output. However, if you want to output multiple colors then you can just attach a number to the end of that. For our example would be using :COLOR0 and :COLOR1. In the end, our pixel shader would look something like this:

// Our texture coordinates:
struct input_t
{
    // Our texture coordinates:
    float2 f2_texcoord : TEXCOORD0;
}

struct output_t
{
    // Output Result:
    float4 f4_color0 : COLOR0;
    float4 f4_color1 : COLOR1;
}

void main(in input_t input, out output_t output)
{
    float4 _f4_outputNormal = tex2D( gm_BaseTexture, input.f2_texcoord );
    float4 _f4_outputStencil = float4(0., 0., 0., _f4_outputNormal.a);

    output.f4_color0 = _f4_outputNormal;  // Output to surface 1
    output.f4_color1 = _f4_outputStencil;   // Output to surface 2
}

That is really all there is to it. The shader setup is very simple. Let's move on!

GML:

So I commented in the shaders that using gl_FragData[#] or output.f4_color# would draw to separate surface. But how do we let the shader know what surfaces to use? Well, much like calling surface_set_target when you want to draw to a surface, and where using gl_FragColor or :COLOR would normally output to, you can specify a surface to a certail "color slot" with surface_set_target_ext.

At this point you may be asking if there is a limit to the number of surfaces you can output to. If you look up surface_set_target_ext in the manual, or even check the color-coding for HLSL you will notice that, yes. You are limited to 4 outputs numbered 0-3. Still, that should generally be more than enough.

Alright, how do we set this up? For consistency, we will be using 2 surfaces that we create but do note you can also use the application_surface if you ever desire. Create an object and, in the create event, create two surfaces to render to.

_surfaceNormal = surface_create(128, 128); // Will contain our normal sprite
_surfaceStencil = surface_create(128, 128);  // Will contain the black stencil of our sprite

In the draw event we will need to set the shader as we normally would, then set each surface as a render target before drawing our sprite. In this example I will just draw a circle, but you can easily draw a sprite if you so wish.

Assuming that we named our shader exampleShd this is what our code would look like:

// Make sure our surfaces haven't been cleared:
if (!surface_exists(_surfaceNormal) ||
    !surface_exists(_surfaceStencil))
    exit;

draw_set_color(c_red); // To color my circle. Not needed for this to work

shader_set(exampleShd); // Specify the shader to use, as normal

// Let's set our first surface to slot 0! This would be gl_FragData[0] or :COLOR0 in our shader:
surface_set_target_ext(0, _surfaceNormal);
// Setting the second surface is just the same. This would be gl_FragData[1] or :COLOR1 in our shader:
surface_set_target_ext(1, _surfaceStencil);

// Now draw our sprite as normal, in this case in the middle of the surface:
draw_circle(64, 64, 48, false);

shader_reset(); // Reset our shader
surface_reset_target(); // Notice we only call it once. You do not need to call it for each surface you render to!

That is all there is to it! If you followed my code exactly, you could check each surface and find that _surfaceNormal would contain a red circle and _surfaceStencil a black circle! We were able to render two different results to two separate surfaces in one pass! How about that?

There are numerous applications for MRTs and they come in especially useful when shaders start getting extremely complex as it can really save a lot of computation time.

I hope you found this tutorial easy to follow! All the best with your projects!

 

[Juju]

shader_set_target_ext(0, _surfaceNormal);

shader_set_target_ext doesn't exist as a function. Is this a script you've made yourself?

Edit: Ah, that's probably a typo for surface_set_target_ext

Edit 2: Works great in HLSL9. GLSL ES fires back an error about gl_FragData[] having an invalid index. Is there some constant I need to set inside the shader? A known working example project would be useful here.

Edit 3: Had a chat to MikeD - GLSL ES does not support MRTs. Straight GLSL does (as does HLSL). I don't have a Mac/Linux platform to test on sadly.

 

[Binsk]

Hello Juju,

Terribly sorry about the late reply, I thought I was watching this topic but I wasn't. I'm glad you managed to figure things out.

I fixed the typo in the tutorial, thanks for letting me know.

In regards to GLSL ES, only the very newest version supports MRTs (3.0, I believe). I could be wrong but I think GameMaker uses 1.X. Either way, yes you can only use this with GLSL or HLSL.

 

[Juju]

1.1, I think? It's a very old version regardless.

 

[Tthecreator]

Wow thanks for the quick explanation, The game maker languages tells you way too little about this.

 

[flyingsaucerinvasion]

@Binsk Hi.

I'm wondering if there is anything I need to know about MRT's in conjunction with the depth buffer. I presume that all render targets use the same depth buffer. But then, what if the render targets are different sizes, what is the size of the depth buffer, or does each render target have a different depth buffer? Will switching the number of render targets being used clear the depth buffer? I really wish we had more direct control over depth buffers.

 

[Binsk]

@flyingsaucerinvasion

The depth buffer is checked between the vertex and fragment shaders. Since the MRTs take place in the fragment shader the depth check will have already been performed. This also implies that all targets share the same depth buffer (and it is not cleared between targets). Remember, MRTs essentially just take place at the very LAST step. This last step being determining the actual color of the pixel. All the other math has already been performed in figuring out 'where' that pixel is, etc. MRTs just allow you to go from "I want the pixel to be blue" to "Wait, I want a blue AND a red pixel for this spot!"

Lucky for us, the coordinate system of the pixels are not literal. They are in the range of [0..1] (or [-1..1] for OpenGL I think, been a while). Point being, all pixel positions are within the same range of coordinates regardless of surface size. So if you have MRTs of different sizes things should render to both fully, you won't have things cut off or anything like that.

 

[flyingsaucerinvasion]

@Binsk Just one thing to clear up here. If I switch to a different MRT configuration, i.e. different number of switch out render targets, it wont clear the depth buffer? If so, how can we force it to clear?

 

[Binsk]

You have a couple options that I can think of off hand:

1) Use d3d_set_hidden to disable depth-testing and draw something transparent at the furthest z-value possible. Not truly clearing it but gets a similar result.

2) Make and use your own depth buffer. Not as efficient but you can at least clear it whenever you like.