Cast shadows
Cast shadows appear on fragments obscured from light by another object. One of the most common ways to build such shadows is Shadow Mapping.
First, draw the room from the position of the light source on a separate surface, storing only the depth information. The resulting texture is called a shadow map.
Next, draw the room for the second time, but now from the camera’s position (as usual). Calculate the light depth again, and if it is greater than the corresponding value from the map, then the light is blocked by something.
Since the shadow map is a surface, its size cannot be infinite. Therefore to draw and use it, it is not enough to define just the light direction. We also need to know:
the size of the map;
the length of the light rays;
the position of the light source;
the up and right vectors.
Let's add corresponding vectors and constants to
ob_Draw3D.
ob_Draw3D - Create
#macro LIGHT_SIZE 1024
#macro LIGHT_LENGHT 640
#macro X 0
#macro Y 1
#macro Z 2
lightForward = array_create(3);
lightRight = array_create(3);
lightUp = array_create(3);
lightPosition = array_create(3);
... ... ...
Up to this point, the light direction has been rigid (x = -0.50, y = 1.00, z = 0.75). From now on, let's set the light using a script. It will take two points and a direction as arguments: from, to and up. If you used to work with 3D in GameMaker before, you already know how they work.
Script - set_light
/// @arg from
/// @arg to
/// @arg up
var from = argument0,
to = argument1,
up = argument2;
with (ob_Draw3D)
{
lightPosition[X] = from[X];
lightPosition[Y] = from[Y];
lightPosition[Z] = from[Z];
lightForward[X] = to[X] - from[X];
lightForward[Y] = to[Y] - from[Y];
lightForward[Z] = to[Z] - from[Z];
lightUp[X] = up[X];
lightUp[Y] = up[Y];
lightUp[Z] = up[Z];
normalize(lightForward);
cross_product(lightForward, lightUp, lightRight);
normalize(lightRight);
cross_product(lightRight, lightForward, lightUp);
}
Also, add the ability to turn the light with the keyboard arrows.
ob_Draw3D - Create
... ... ...
lightPosition = array_create(3);
lightForward[@X] = -0.50;
lightForward[@Y] = 1.00;
lightForward[@Z] = 0.75;
normalize(lightForward);
dirl_LightForward = shader_get_uniform(
sh_directional_lighting, "u_LightForward");
lightHor = 30;
lightVer = -60;
lightDist = LIGHT_LENGHT * 0.5;
ob_Draw3D - Step
lightHor += 2 * (
keyboard_check(vk_right) -
keyboard_check(vk_left));
lightVer -= 2 * (
keyboard_check(vk_up) -
keyboard_check(vk_down));
lightVer = clamp(lightVer, -80, 80);
var dis = lengthdir_x(lightDist, lightVer),
lx = lengthdir_y(dis, lightHor),
ly = lengthdir_y(lightDist, lightVer),
lz = lengthdir_x(dis, lightHor);
set_light(
[-lx, -ly, -lz],
[ 0, 0, 0 ],
[ 0, -1, 0 ]);
Finally, let's improve the GUI layer. Since we now know the position of the light source, then the vector should be drawn from it, and not from the origin.
ob_Draw3D - Draw GUI
var length = 100,
position = world_to_GUI(lightPosition),
forward = world_to_GUI([
lightPosition[X] + lightForward[X] * length,
lightPosition[Y] + lightForward[Y] * length,
lightPosition[Z] + lightForward[Z] * length]);
... ... ...
And let's also show up and right vectors to be sure they are mutually perpendicular and directed in the right direction.
ob_Draw3D - Draw GUI
var length = 50,
position = world_to_GUI(lightPosition),
right = world_to_GUI([
lightPosition[X] + lightRight[X] * length,
lightPosition[Y] + lightRight[Y] * length,
lightPosition[Z] + lightRight[Z] * length]),
up = world_to_GUI([
lightPosition[X] + lightUp[X] * length,
lightPosition[Y] + lightUp[Y] * length,
lightPosition[Z] + lightUp[Z] * length]),
forward = world_to_GUI([
lightPosition[X] + lightForward[X] * length,
lightPosition[Y] + lightForward[Y] * length,
lightPosition[Z] + lightForward[Z] * length]);
draw_set_color(c_red);
draw_arrow(
position[X], position[Y],
right[X], right[Y], 10);
draw_set_color(c_lime);
draw_arrow(
position[X], position[Y],
up[X], up[Y], 10);
draw_set_color(c_blue);
draw_arrow(
position[X], position[Y],
forward[X], forward[Y], 10);
Now we can easily choose the light direction without closing the game!
Let's create a shadow map. Create a new shader and call it
sh_shadow_mapping. In the vertex shader, remove all attributes except
in_Position, and add the uniforms
u_LightForward — for the light direction;
u_LightPosition — for the light position;
u_LightLenght — for the length of the rays.
sh_shadow_mapping.vsh
//
// Simple passthrough vertex shader
//
attribute vec3 in_Position; // (x,y,z)
//attribute vec3 in_Normal; // (x,y,z) unused in this shader.
attribute vec4 in_Colour; // (r,g,b,a)
attribute vec2 in_TextureCoord; // (u,v)
varying vec2 v_vTexcoord;
varying vec4 v_vColour;
uniform vec3 u_LightForward;
uniform vec3 u_LightPosition;
uniform float u_LightLenght;
void main()
{
vec4 object_space_pos = vec4( in_Position.x, in_Position.y, in_Position.z, 1.0);
gl_Position = gm_Matrices[MATRIX_WORLD_VIEW_PROJECTION] * object_space_pos;
v_vColour = in_Colour;
v_vTexcoord = in_TextureCoord;
}
Remove all unnecessary in the fragment-shader too.
sh_shadow_mapping.fsh
//
// Simple passthrough fragment shader
//
varying vec2 v_vTexcoord;
varying vec4 v_vColour;
void main()
{
gl_FragColor = vec4(1.0);
}
Before continuing to work on the shader, it would be nice to see what it's doing. Let's go back to
ob_Draw3D and set it up.
ob_Draw3D - Create
... ... ...
lightPosition = array_create(3);
smap_LightForward = shader_get_uniform(
sh_shadow_mapping, "u_LightForward");
smap_LightPosition = shader_get_uniform(
sh_shadow_mapping, "u_LightPosition");
smap_LightLenght = shader_get_uniform(
sh_shadow_mapping, "u_LightLenght");
dirl_LightForward = shader_get_uniform(
sh_directional_lighting, "u_LightForward");
... ... ...
ob_Draw3D - Draw
gpu_set_zwriteenable(true);
gpu_set_ztestenable(true);
shader_set(sh_shadow_mapping);
shader_set_uniform_f_array(smap_LightForward, lightForward);
shader_set_uniform_f_array(smap_LightPosition, lightPosition);
shader_set_uniform_f( smap_LightLenght, LIGHT_LENGHT);
with (ob_Mesh) event_user(0);
shader_reset();
gpu_set_zwriteenable(false);
gpu_set_ztestenable(false);
Of course, the shader is doing nothing for now, but this is about to change.
In the vertex shader, find the vertices depth. To do it, find a difference between the world-space position of the vertex (
world_space_pos) and the position of the light source (
u_LightPosition). Next, calculate a dot product with the light direction (
u_LightForward). The resulting value will be the distance from the light source to the vertex (
light_depth). Divide it by the length of the rays (
u_LightLenght) to get a value in a range of 0 to 1. Pass it to the fragment shader.
sh_shadow_mapping.vsh
attribute vec3 in_Position;
varying float v_vLightDepth;
uniform vec3 u_LightForward;
uniform vec3 u_LightPosition;
uniform float u_LightLenght;
void main()
{
vec4 object_space_pos = vec4( in_Position.x, in_Position.y, in_Position.z, 1.0);
vec4 world_space_pos = gm_Matrices[MATRIX_WORLD] * object_space_pos;
gl_Position = gm_Matrices[MATRIX_WORLD_VIEW_PROJECTION] * object_space_pos;
vec3 light_to_object = vec3(world_space_pos) - u_LightPosition;
float light_depth = dot(light_to_object, u_LightForward);
float light_depth_01 = clamp(light_depth / u_LightLenght, 0.0, 1.0);
v_vLightDepth = light_depth_01;
}
In the fragment shader, just show the
v_vLightDistance as grayscale for now.
sh_shadow_mapping.fsh
varying float v_vLightDepth;
void main()
{
vec3 packed_depth = vec3(v_vLightDepth);
gl_FragColor = vec4(packed_depth, 1.0);
}
Fragments appear darker near the light and become lighter, the further away they are.
As you remember, I wrote earlier that the shadow map should be drawn from the light source, not from the camera. To achieve this, we need to temporarily replace the view and projection matrices. Also, the target for drawing should be a separate surface, not the game window.
Let's add variables for matrices and surface to the
ob_Draw3D.
ob_Draw3D - Create
... ... ...
lightPosition = array_create(3);
lightViewMat = matrix_build_identity();
lightProjMat = matrix_build_identity();
surfShadowMap = -1;
... ... ...
The
set_light script is a good place to update matrices. For the projection matrix use the orthogonal projection.
Script - set_light
... ... ...
normalize(lightForward);
cross_product(lightForward, lightUp, lightRight);
normalize(lightRight);
cross_product(lightRight, lightForward, lightUp );
lightViewMat = matrix_build_lookat(
from[X], from[Y], from[Z],
to[X], to[Y], to[Z],
up[X], up[Y], up[Z]);
lightProjMat = matrix_build_projection_ortho(
-LIGHT_SIZE, LIGHT_SIZE, 0, LIGHT_LENGHT);
... ... ...
Replace the matrices during shadow map drawing.
ob_Draw3D - Draw
var cameraProj = matrix_get(matrix_projection),
cameraView = matrix_get(matrix_view);
gpu_set_zwriteenable(true);
gpu_set_ztestenable(true);
matrix_set(matrix_projection, lightProjMat);
matrix_set(matrix_view, lightViewMat);
shader_set(sh_shadow_mapping);
... ... ...
shader_reset();
matrix_set(matrix_projection, cameraProj);
matrix_set(matrix_view, cameraView);
gpu_set_zwriteenable(false);
gpu_set_ztestenable(false);
Set the drawing target for the shadow map. Before you draw something on the surface, make sure that it exists and clear it.
ob_Draw3D - Draw
var cameraProj = matrix_get(matrix_projection),
cameraView = matrix_get(matrix_view);
if (!surface_exists(surfShadowMap))
surfShadowMap = surface_create(
LIGHT_SIZE * 2, LIGHT_SIZE * 2);
gpu_set_zwriteenable(true);
gpu_set_ztestenable(true);
surface_set_target(surfShadowMap);
matrix_set(matrix_projection, lightProjMat);
matrix_set(matrix_view, lightViewMat);
shader_set(sh_shadow_mapping);
shader_set_uniform_f_array(lmap_LightForward, lightForward);
shader_set_uniform_f_array(lmap_LightPosition, lightPosition);
shader_set_uniform_f( lmap_LightLenght, LIGHT_LENGHT);
draw_clear(c_white);
with (ob_Mesh) event_user(0);
shader_reset();
surface_reset_target();
matrix_set(matrix_projection, cameraProj);
matrix_set(matrix_view, cameraView);
gpu_set_zwriteenable(false);
gpu_set_ztestenable(false);
Draw the
ob_Mesh for the second time using the
sh_directional_lighting shader.
ob_Draw3D - Draw
... ... ...
surface_reset_target();
matrix_set(matrix_projection, cameraProj);
matrix_set(matrix_view, cameraView);
shader_set(sh_directional_lighting);
shader_set_uniform_f_array(dirl_LightForward, lightForward);
with (ob_Mesh) event_user(0);
shader_reset();
gpu_set_zwriteenable(false);
gpu_set_ztestenable(false);
Draw the surface on the GUI layer to see how it looks like.
ob_Draw3D - Draw GUI
... ... ...
draw_surface_ext(surfShadowMap, 0, 0, 0.25, 0.25, 0, c_white, 1);
Run the game. Mesh is been drawn twice now. Rotate the light source - the image on the surface will change.
Add uniforms to the
sh_directional_lighting vertex shader :
u_LightRight — for the right vector;
u_LightUp — for the up vector;
u_LightPosition — for the light source position;
u_LightSize — for the shadow map size;
u_LightLenght — for the ray length;
to the fragment shader:
u_ShadowMap — 2D sempler for shadow map.
Pass the data to said uniforms.
sh_directional_lighting.vsh
... ... ...
varying vec2 v_vTexcoord;
varying vec4 v_vColour;
varying float v_vIllumination;
uniform vec3 u_LightForward;
uniform vec3 u_LightRight;
uniform vec3 u_LightUp;
uniform vec3 u_LightPosition;
uniform float u_LightSize;
uniform float u_LightLenght;
void main()
... ... ...
sh_directional_lighting.fsh
varying vec2 v_vTexcoord;
varying vec4 v_vColour;
varying float v_vIllumination;
uniform sampler2D u_ShadowMap;
void main()
... ... ...
ob_Draw3D - Create
... ... ...
dirl_LightForward = shader_get_uniform(
sh_directional_lighting, "u_LightForward");
dirl_LightRight = shader_get_uniform(
sh_directional_lighting, "u_LightRight");
dirl_LightUp = shader_get_uniform(
sh_directional_lighting, "u_LightUp");
dirl_LightPosition = shader_get_uniform(
sh_directional_lighting, "u_LightPosition");
dirl_LightSize = shader_get_uniform(
sh_directional_lighting, "u_LightSize");
dirl_LightLenght = shader_get_uniform(
sh_directional_lighting, "u_LightLenght");
dirl_ShadowMap = shader_get_sampler_index(
sh_directional_lighting, "u_ShadowMap");
... ... ...
ob_Draw3D - Draw
... ... ...
shader_set(sh_directional_lighting);
shader_set_uniform_f_array(dirl_LightForward, lightForward);
shader_set_uniform_f_array(dirl_LightRight, lightRight);
shader_set_uniform_f_array(dirl_LightUp, lightUp);
shader_set_uniform_f_array(dirl_LightPosition, lightPosition);
shader_set_uniform_f( dirl_LightSize, LIGHT_SIZE);
shader_set_uniform_f( dirl_LightLenght, LIGHT_LENGHT);
var texShadowMap = surface_get_texture(surfShadowMap);
texture_set_stage( dirl_ShadowMap, texShadowMap);
with (ob_Mesh) event_user(0);
shader_reset();
... ... ...
To build cast shadows, first, calculate the light depth in
sh_directional_lighting, just like we did it in
sh_shadow_mapping.
sh_directional_lighting.vsh
... ... ...
varying vec2 v_vTexcoord;
varying vec4 v_vColour;
varying float v_vIllumination;
varying float v_vLightDepth;
... ... ...
void main()
{
vec4 object_space_pos = vec4( in_Position.x, in_Position.y, in_Position.z, 1.0);
vec4 world_space_pos = gm_Matrices[MATRIX_WORLD] * object_space_pos;
vec3 world_space_norm = normalize(mat3(gm_Matrices[MATRIX_WORLD]) * in_Normal);
gl_Position = gm_Matrices[MATRIX_WORLD_VIEW_PROJECTION] * object_space_pos;
float illumination = -dot(world_space_norm, u_LightForward);
vec3 light_to_object = vec3(world_space_pos) - u_LightPosition;
float light_depth = dot(light_to_object, u_LightForward);
float light_depth_01 = clamp(light_depth / u_LightLenght, 0.0, 1.0);
v_vColour = in_Colour;
v_vTexcoord = in_TextureCoord;
v_vIllumination = illumination;
v_vLightDepth = light_depth_01;
}
Next, get the depth value from the map. To do this, take the world-space vertex (
light_to_object) and find a dot product with right and up vectors (
u_LightRight,
u_LightRight). Divide the resulting 2D coordinates by the map size (
u_LightSize) and add 0.5. Now you got the UV texture coordinates for the map. Pass them to the fragment shader.
sh_directional_lighting.vsh
... ... ...
varying vec2 v_vTexcoord;
varying vec4 v_vColour;
varying float v_vIllumination;
varying float v_vLightDepth;
varying vec2 v_vLightMapPosition;
... ... ...
void main()
{
... ... ...
float light_depth = dot(light_to_object, u_LightForward);
float light_depth_01 = clamp(light_depth / u_LightLenght, 0.0, 1.0);
float light_map_U = 0.5 + dot(light_to_object, u_LightRight) / u_LightSize;
float light_map_V = 0.5 + dot(light_to_object, u_LightUp ) / u_LightSize;
v_vColour = in_Colour;
v_vTexcoord = in_TextureCoord;
v_vIllumination = illumination;
v_vLightDepth = light_depth_01;
v_vLightMapPosition = vec2(light_map_U, light_map_V);
}
In the fragment shader, just show the map projection for now.
sh_directional_lighting.fsh
varying vec2 v_vTexcoord;
varying vec4 v_vColour;
varying float v_vIllumination;
varying vec2 v_vLightMapPosition;
uniform sampler2D u_ShadowMap;
void main()
{
gl_FragColor = texture2D(u_ShadowMap, v_vLightMapPosition);
//gl_FragColor = v_vColour * texture2D(gm_BaseTexture, v_vTexcoord);
//float smooth_illumination = smoothstep(0.0, 0.8, v_vIllumination);
//gl_FragColor.rgb *= mix(0.3, 1.0, smooth_illumination);
}
The projection turned out to be incorrect: the farthest corner of the mesh ended up near the light. The UV coordinates need to be flipped vertically.
sh_directional_lighting.vsh
... ... ...
float light_map_U = 0.5 + dot(light_to_object, u_LightRight) / u_LightSize;
float light_map_V = 0.5 - dot(light_to_object, u_LightUp ) / u_LightSize;
... ... ...
Now we getting somewhere! You can even see the outline of the shadow. The only thing left to do is to compare
v_vLightDistance to the map value. If
v_vLightDistance is greater, then a fragment is in a cast shadow.
sh_directional_lighting.fsh
varying vec2 v_vTexcoord;
varying vec4 v_vColour;
varying float v_vIllumination;
varying float v_vLightDepth;
varying vec2 v_vLightMapPosition;
... ... ...
void main()
{
float depth = texture2D(u_ShadowMap, v_vLightMapPosition).r;
//gl_FragColor = v_vColour * texture2D(gm_BaseTexture, v_vTexcoord);
if (v_vLightDepth > depth)
{
gl_FragColor.rgb *= 0.3;
}
else
{
//float smooth_illumination = smoothstep(0.0, 0.8, v_vIllumination);
//gl_FragColor.rgb *= mix(0.3, 1.0, smooth_illumination);
}
}
Well, we got shadows, but everything is covered with strange patterns. There are a couple of reasons for this.
The first problem is a limited depth resolution. Currently, while creating the shadow map, we write the same value in all three RGB channels. In fact, of the available four bytes per pixel, we use only one. Use the functions below to encode map values in the
sh_shadow_mapping shader and decode them in the
sh_directional_lighting shader.
packFloatInto8BitVec3(val01) — takes a float number in a range of 0 to 1 and returns the corresponding RGB color.
unpack8BitVec3IntoFloat(valRGB) — takes the RGB color and returns the corresponding float number in a range of 0 to 1.
sh_shadow_mapping.fsh
varying float v_vLightDepth;
const float SCALE_FACTOR = 256.0 * 256.0 * 256.0 - 1.0;
vec3 packFloatInto8BitVec3(float val01)
{
float zeroTo24Bit = val01 * SCALE_FACTOR;
return floor(
vec3(
mod(zeroTo24Bit, 256.0),
mod(zeroTo24Bit / 256.0, 256.0),
zeroTo24Bit / 256.0 / 256.0
)
) / 255.0;
}
void main()
{
vec3 packed_depth = packFloatInto8BitVec3(v_vLightDepth);
gl_FragColor = vec4(packed_depth, 1.0);
}
sh_directional_lighting.fsh
... ... ...
const float SCALE_FACTOR = 256.0 * 256.0 * 256.0 - 1.0;
const vec3 SCALE_VECTOR = vec3(1.0, 256.0, 256.0 * 256.0) / SCALE_FACTOR * 255.0;
float unpack8BitVec3IntoFloat(vec3 valRGB)
{
return dot(valRGB, SCALE_VECTOR);
}
void main()
{
vec3 packed_depth = texture2D(u_LightMap, v_vLightMapPosition).rgb;
float depth = unpack8BitVec3IntoFloat(packed_depth);
... ... ...
}
The next problem is a limited shadow map resolution.
A simple way to deal with it is to shift the map value slightly forward. This will get rid of the majority of unwanted shadows.
sh_directional_lighting.fsh
... ... ...
if (v_vLightDepth > depth + 0.005)
... ... ...
Finally, hide remaining edge case artifacts in the form-shadows. To do this, just raise a minimal illumination a bit.
sh_directional_lighting.fsh
... ... ...
else
{
float smooth_illumination = smoothstep(0.1, 0.8, v_vIllumination);
gl_FragColor.rgb *= mix(0.3, 1.0, smooth_illumination);
}
... ... ...
And this is it! The cast-shadows are done!
Conclusion
In this tutorial, we covered one of the simplest and efficient ways to create directional lighting with a single light source in the GameMaker Studio 2. The code can be adapted for GMS 1.4, or become a base for more complex effects. Performance can be improved too. You can use/modify code from this tutorial, as you please.
I wish you the best of luck with your projects!