GameMaker Directional Light and Shadows in 3D

GoK

Member

GM Version: GMS 2.2.5
Target Platform: All
Download:
Links: N/A



Summary

Hi! In this tutorial, we will go through the process of creating directional lighting and cast shadows. Note, it's not intended for the complete beginners. You should have at least some knowledge on how 3D graphics, shaders, and surfers work in GMS 2.
Before we start, I want to apologize for my English. This is not my native language, and I am writing something so long for the first time. If you find grammatic errors, please let me know.



Tutorial

To draw the shadows, we need a camera and a 3D mesh that will cast said shadows. It's already done in the starter project (see a link above).
After downloading it and running a game, you will see something like this:

Controls:
Mouse — orbits the camera around;​
WASD — rotates the model.​

Currently, the standard shader is responsible for drawing a room, and as you can see, it doesn’t bother with lighting at all.
There are two types of shadows: form shadows and cast shadows.




Form shadows

Form shadows appear on polygons that face away from light.
To determine how dark the particular polygon is, we need to project its normal onto the light vector. The result will be greater than zero for the shaded polygons.

The normals are already written in the mesh, but we don’t know anything about the light. Let's define the light vector as an array lightForward[X, Y, Z], assign its initial value and normalize it.

ob_Draw3D - Create
#macro X 0
#macro Y 1
#macro Z 2
lightForward = array_create(3);
lightForward[X] = -0.50;
lightForward[Y] = 1.00;
lightForward[Z] = 0.75;
normalize(lightForward);

Let's also draw this vector on the GUI layer so we can see in which direction the light shines.

ob_Draw3D - Draw GUI
var length = 100,
position = world_to_GUI([0, 0, 0]),
forward = world_to_GUI([
lightForward[X] * length,
lightForward[Y] * length,
lightForward[Z] * length]);

draw_set_color(c_blue);
draw_arrow(

position[X], position[Y],
forward[X], forward[Y], 10);

Create a new shader, call it sh_directional_lighting. In the vertex shader, remove a comment from an in_Normal attribute and add a u_LightForward uniform for the light vector.

sh_directional_lighting.vsh
//
// Simple passthrough vertex shader
//


attribute vec3 in_Position; // (x,y,z)
//attribute vec3 in_Normal; // (x,y,z) unused in this shader.
attribute vec4 in_Colour; // (r,g,b,a)
attribute vec2 in_TextureCoord; // (u,v)

varying vec2 v_vTexcoord;
varying vec4 v_vColour;

uniform vec3 u_LightForward;

void main()
... ... ...

Now go back to the ob_Draw3D. Set up a drawing through the sh_directional_lighting shader and pass the light vector to it.

ob_Draw3D - Create
... ... ...
normalize(lightForward);


dirl_LightForward = shader_get_uniform(
sh_directional_lighting, "u_LightForward");

ob_Draw3D - Draw
gpu_set_zwriteenable(true);
gpu_set_ztestenable(true);


shader_set(sh_directional_lighting);
shader_set_uniform_f_array(dirl_LightForward, lightForward);
with (ob_Mesh) event_user(0);
shader_reset();

gpu_set_zwriteenable(false);
gpu_set_ztestenable(false);

The mesh has been drawn through our new shader. However, if you run the game, you will notice that nothing has changed. This is because although the shader receives the light information, it doesn’t do anything with it yet.
Find the dot product of a vertex normal from the in_Normal attribute and a vector from the u_LightForward uniform. As both are normalized, the result will be a float value between -1 and 1, corresponding to the length of the projection of the normal onto the light vector. For the lit polygons, this value will be less than 0. To fix it, simply negate the result. Pass it to the fragment shader.

sh_directional_lighting.vsh
... ... ...
varying vec2 v_vTexcoord;
varying vec4 v_vColour;

varying float v_vIllumination;

uniform vec3 u_LightForward;

void main()
{

vec4 object_space_pos = vec4(in_Position.x, in_Position.y, in_Position.z, 1.0);
gl_Position = gm_Matrices[MATRIX_WORLD_VIEW_PROJECTION] * object_space_pos;
float illumination = -dot(in_Normal, u_LightForward);
v_vColour = in_Colour;
v_vTexcoord = in_TextureCoord;
v_vIllumination = illumination;
}

In the fragment shader, multiply the color of the target fragment by its illumination.

sh_directional_lighting.fsh
varying vec2 v_vTexcoord;
varying vec4 v_vColour;

varying float v_vIllumination;

void main()
{

gl_FragColor = v_vColour * texture2D(gm_BaseTexture, v_vTexcoord);
gl_FragColor.rgb *= v_vIllumination;
}

It's working! We got the shaded polygons, but there are still some problems.
Try to turn the cube with the WASD buttons and you will see that the shadows seem to “stick” to their faces. This is happening because the in_Normal normals are written in the object-space and they don't change when the mesh gets transformed in the world-space.

We must transform the normals manually. To do this, you need to multiply them with the world matrix (gm_Matrices[MATRIX_WORLD]), but there are some complications: firstly, we must exclude translation from the matrix (since normals are directions, not points); secondly, we must normalize the result.

sh_directional_lighting.vsh
... ... ...
vec4 object_space_pos = vec4(in_Position.x, in_Position.y, in_Position.z, 1.0);
vec3 world_space_norm = normalize(mat3(gm_Matrices[MATRIX_WORLD]) * in_Normal);
gl_Position = gm_Matrices[MATRIX_WORLD_VIEW_PROJECTION] * object_space_pos;
float illumination = -dot(world_space_norm, u_LightForward);
v_vColour = in_Colour;
v_vTexcoord = in_TextureCoord;
v_vIllumination = illumination;
... ... ...

That was a simplified method for transforming normals. It only works correctly if the object is scaled uniformly (with the same scale along all axes). If nonuniform scaling is necessary, then you need to invert and transpose the world matrix first.


Alright, we fix the normals! However, there is still an issue: the picture as a whole becomes darker. This happens because we multiply the color of the fragment by its illumination, and it is less than 1 almost everywhere. We can deal with it using a smoothstep and mix functions.
smoothstep(A, B, x) — performs smooth Hermite interpolation between 0 and 1 when A < x < B.​
mix(x, y, A) — linearly interpolate between x and y.​

sh_directional_lighting.fsh
... ... ...
gl_FragColor = v_vColour * texture2D(gm_BaseTexture, v_vTexcoord);
float smooth_illumination = smoothstep(0.0, 0.8, v_vIllumination);
gl_FragColor.rgb *= mix(0.3, 1.0, smooth_illumination);
... ... ...

Form shadows are done!
 
Last edited:

GoK

Member
Cast shadows

Cast shadows appear on fragments obscured from light by another object. One of the most common ways to build such shadows is Shadow Mapping.
First, draw the room from the position of the light source on a separate surface, storing only the depth information. The resulting texture is called a shadow map.

Next, draw the room for the second time, but now from the camera’s position (as usual). Calculate the light depth again, and if it is greater than the corresponding value from the map, then the light is blocked by something.

Since the shadow map is a surface, its size cannot be infinite. Therefore to draw and use it, it is not enough to define just the light direction. We also need to know:
the size of the map;​
the length of the light rays;​
the position of the light source;​
the up and right vectors.​
Let's add corresponding vectors and constants to ob_Draw3D.

ob_Draw3D - Create
#macro LIGHT_SIZE 1024
#macro LIGHT_LENGHT 640

#macro X 0
#macro Y 1
#macro Z 2

lightForward = array_create(3);

lightRight = array_create(3);
lightUp = array_create(3);
lightPosition = array_create(3);

... ... ...

Up to this point, the light direction has been rigid (x = -0.50, y = 1.00, z = 0.75). From now on, let's set the light using a script. It will take two points and a direction as arguments: from, to and up. If you used to work with 3D in GameMaker before, you already know how they work.

Script - set_light
/// @arg from
/// @arg to
/// @arg up

var from = argument0,

to = argument1,
up = argument2;

with (ob_Draw3D)
{

lightPosition[X] = from[X];
lightPosition[Y] = from[Y];
lightPosition[Z] = from[Z];
lightForward[X] = to[X] - from[X];
lightForward[Y] = to[Y] - from[Y];
lightForward[Z] = to[Z] - from[Z];
lightUp[X] = up[X];
lightUp[Y] = up[Y];
lightUp[Z] = up[Z];
normalize(lightForward);
cross_product(lightForward, lightUp, lightRight);
normalize(lightRight);
cross_product(lightRight, lightForward, lightUp);
}

Also, add the ability to turn the light with the keyboard arrows.

ob_Draw3D - Create
... ... ...
lightPosition = array_create(3);
lightForward[@X] = -0.50;

lightForward[@Y] = 1.00;
lightForward[@Z] = 0.75;

normalize(lightForward);

dirl_LightForward = shader_get_uniform(

sh_directional_lighting, "u_LightForward");

lightHor = 30;
lightVer = -60;
lightDist = LIGHT_LENGHT * 0.5;
ob_Draw3D - Step
lightHor += 2 * (
keyboard_check(vk_right) -
keyboard_check(vk_left));
lightVer -= 2 * (
keyboard_check(vk_up) -
keyboard_check(vk_down));
lightVer = clamp(lightVer, -80, 80);

var dis = lengthdir_x(lightDist, lightVer),

lx = lengthdir_y(dis, lightHor),
ly = lengthdir_y(lightDist, lightVer),
lz = lengthdir_x(dis, lightHor);

set_light(
[-lx, -ly, -lz],
[ 0, 0, 0 ],
[ 0, -1, 0 ]);

Finally, let's improve the GUI layer. Since we now know the position of the light source, then the vector should be drawn from it, and not from the origin.

ob_Draw3D - Draw GUI
var length = 100,
position = world_to_GUI(lightPosition),
forward = world_to_GUI([
lightPosition[X] + lightForward[X] * length,
lightPosition[Y] + lightForward[Y] * length,
lightPosition[Z] + lightForward[Z] * length]);
... ... ...

And let's also show up and right vectors to be sure they are mutually perpendicular and directed in the right direction.

ob_Draw3D - Draw GUI
var length = 50,
position = world_to_GUI(lightPosition),
right = world_to_GUI([
lightPosition[X] + lightRight[X] * length,
lightPosition[Y] + lightRight[Y] * length,
lightPosition[Z] + lightRight[Z] * length]),
up = world_to_GUI([
lightPosition[X] + lightUp[X] * length,
lightPosition[Y] + lightUp[Y] * length,
lightPosition[Z] + lightUp[Z] * length]),
forward = world_to_GUI([
lightPosition[X] + lightForward[X] * length,
lightPosition[Y] + lightForward[Y] * length,
lightPosition[Z] + lightForward[Z] * length]);

draw_set_color(c_red);
draw_arrow(
position[X], position[Y],
right[X], right[Y], 10);

draw_set_color(c_lime);
draw_arrow(
position[X], position[Y],
up[X], up[Y], 10);


draw_set_color(c_blue);
draw_arrow(
position[X], position[Y],
forward[X], forward[Y], 10);

Now we can easily choose the light direction without closing the game!

Let's create a shadow map. Create a new shader and call it sh_shadow_mapping. In the vertex shader, remove all attributes except in_Position, and add the uniforms
u_LightForward — for the light direction;​
u_LightPosition — for the light position;​
u_LightLenght — for the length of the rays.​

sh_shadow_mapping.vsh
//
// Simple passthrough vertex shader

//
attribute vec3 in_Position; // (x,y,z)

//attribute vec3 in_Normal; // (x,y,z) unused in this shader.
attribute vec4 in_Colour; // (r,g,b,a)
attribute vec2 in_TextureCoord; // (u,v)


varying vec2 v_vTexcoord;
varying vec4 v_vColour;


uniform vec3 u_LightForward;
uniform vec3 u_LightPosition;
uniform float u_LightLenght;


void main()
{

vec4 object_space_pos = vec4( in_Position.x, in_Position.y, in_Position.z, 1.0);
gl_Position = gm_Matrices[MATRIX_WORLD_VIEW_PROJECTION] * object_space_pos;
v_vColour = in_Colour;
v_vTexcoord = in_TextureCoord;
}

Remove all unnecessary in the fragment-shader too.

sh_shadow_mapping.fsh
//
// Simple passthrough fragment shader
//
varying vec2 v_vTexcoord;

varying vec4 v_vColour;

void main()
{

gl_FragColor = vec4(1.0);
}

Before continuing to work on the shader, it would be nice to see what it's doing. Let's go back to ob_Draw3D and set it up.

ob_Draw3D - Create
... ... ...
lightPosition = array_create(3);


smap_LightForward = shader_get_uniform(
sh_shadow_mapping, "u_LightForward");
smap_LightPosition = shader_get_uniform(
sh_shadow_mapping, "u_LightPosition");
smap_LightLenght = shader_get_uniform(
sh_shadow_mapping, "u_LightLenght");

dirl_LightForward = shader_get_uniform(
sh_directional_lighting, "u_LightForward");
... ... ...
ob_Draw3D - Draw
gpu_set_zwriteenable(true);
gpu_set_ztestenable(true);


shader_set(sh_shadow_mapping);
shader_set_uniform_f_array(smap_LightForward, lightForward);
shader_set_uniform_f_array(smap_LightPosition, lightPosition);
shader_set_uniform_f( smap_LightLenght, LIGHT_LENGHT);
with (ob_Mesh) event_user(0);
shader_reset();

gpu_set_zwriteenable(false);
gpu_set_ztestenable(false);

Of course, the shader is doing nothing for now, but this is about to change.
In the vertex shader, find the vertices depth. To do it, find a difference between the world-space position of the vertex (world_space_pos) and the position of the light source (u_LightPosition). Next, calculate a dot product with the light direction (u_LightForward). The resulting value will be the distance from the light source to the vertex (light_depth). Divide it by the length of the rays (u_LightLenght) to get a value in a range of 0 to 1. Pass it to the fragment shader.

sh_shadow_mapping.vsh
attribute vec3 in_Position;

varying float v_vLightDepth;

uniform vec3 u_LightForward;
uniform vec3 u_LightPosition;
uniform float u_LightLenght;

void main()
{

vec4 object_space_pos = vec4( in_Position.x, in_Position.y, in_Position.z, 1.0);
vec4 world_space_pos = gm_Matrices[MATRIX_WORLD] * object_space_pos;
gl_Position = gm_Matrices[MATRIX_WORLD_VIEW_PROJECTION] * object_space_pos;
vec3 light_to_object = vec3(world_space_pos) - u_LightPosition;
float light_depth = dot(light_to_object, u_LightForward);
float light_depth_01 = clamp(light_depth / u_LightLenght, 0.0, 1.0);
v_vLightDepth = light_depth_01;
}

In the fragment shader, just show the v_vLightDistance as grayscale for now.

sh_shadow_mapping.fsh
varying float v_vLightDepth;

void main()
{

vec3 packed_depth = vec3(v_vLightDepth);
gl_FragColor = vec4(packed_depth, 1.0);
}

Fragments appear darker near the light and become lighter, the further away they are.

As you remember, I wrote earlier that the shadow map should be drawn from the light source, not from the camera. To achieve this, we need to temporarily replace the view and projection matrices. Also, the target for drawing should be a separate surface, not the game window.
Let's add variables for matrices and surface to the ob_Draw3D.

ob_Draw3D - Create
... ... ...
lightPosition = array_create(3);

lightViewMat = matrix_build_identity();
lightProjMat = matrix_build_identity();

surfShadowMap = -1;

... ... ...

The set_light script is a good place to update matrices. For the projection matrix use the orthogonal projection.

Script - set_light
... ... ...
normalize(lightForward);
cross_product(lightForward, lightUp, lightRight);
normalize(lightRight);
cross_product(lightRight, lightForward, lightUp );
lightViewMat = matrix_build_lookat(
from[X], from[Y], from[Z],
to[X], to[Y], to[Z],
up[X], up[Y], up[Z]);
lightProjMat = matrix_build_projection_ortho(
-LIGHT_SIZE, LIGHT_SIZE, 0, LIGHT_LENGHT);
... ... ...

Replace the matrices during shadow map drawing.

ob_Draw3D - Draw
var cameraProj = matrix_get(matrix_projection),
cameraView = matrix_get(matrix_view);

gpu_set_zwriteenable(true);
gpu_set_ztestenable(true);


matrix_set(matrix_projection, lightProjMat);
matrix_set(matrix_view, lightViewMat);
shader_set(sh_shadow_mapping);
... ... ...
shader_reset();
matrix_set(matrix_projection, cameraProj);
matrix_set(matrix_view, cameraView);

gpu_set_zwriteenable(false);
gpu_set_ztestenable(false);

Set the drawing target for the shadow map. Before you draw something on the surface, make sure that it exists and clear it.

ob_Draw3D - Draw
var cameraProj = matrix_get(matrix_projection),
cameraView = matrix_get(matrix_view);

if (!surface_exists(surfShadowMap))
surfShadowMap = surface_create(
LIGHT_SIZE * 2, LIGHT_SIZE * 2);

gpu_set_zwriteenable(true);
gpu_set_ztestenable(true);

surface_set_target(surfShadowMap);
matrix_set(matrix_projection, lightProjMat);
matrix_set(matrix_view, lightViewMat);
shader_set(sh_shadow_mapping);
shader_set_uniform_f_array(lmap_LightForward, lightForward);
shader_set_uniform_f_array(lmap_LightPosition, lightPosition);
shader_set_uniform_f( lmap_LightLenght, LIGHT_LENGHT);
draw_clear(c_white);
with (ob_Mesh) event_user(0);
shader_reset();
surface_reset_target();
matrix_set(matrix_projection, cameraProj);
matrix_set(matrix_view, cameraView);
gpu_set_zwriteenable(false);
gpu_set_ztestenable(false);

Draw the ob_Mesh for the second time using the sh_directional_lighting shader.

ob_Draw3D - Draw
... ... ...
surface_reset_target();
matrix_set(matrix_projection, cameraProj);
matrix_set(matrix_view, cameraView);
shader_set(sh_directional_lighting);
shader_set_uniform_f_array(dirl_LightForward, lightForward);
with (ob_Mesh) event_user(0);
shader_reset();

gpu_set_zwriteenable(false);
gpu_set_ztestenable(false);
Draw the surface on the GUI layer to see how it looks like.

ob_Draw3D - Draw GUI
... ... ...
draw_surface_ext(surfShadowMap, 0, 0, 0.25, 0.25, 0, c_white, 1);

Run the game. Mesh is been drawn twice now. Rotate the light source - the image on the surface will change.

Add uniforms to the sh_directional_lighting vertex shader :
u_LightRight — for the right vector;​
u_LightUp — for the up vector;​
u_LightPosition — for the light source position;​
u_LightSize — for the shadow map size;​
u_LightLenght — for the ray length;​
to the fragment shader:
u_ShadowMap — 2D sempler for shadow map.​
Pass the data to said uniforms.

sh_directional_lighting.vsh
... ... ...
varying vec2 v_vTexcoord;
varying vec4 v_vColour;
varying float v_vIllumination;

uniform vec3 u_LightForward;

uniform vec3 u_LightRight;
uniform vec3 u_LightUp;
uniform vec3 u_LightPosition;
uniform float u_LightSize;
uniform float u_LightLenght;


void main()
... ... ...
sh_directional_lighting.fsh
varying vec2 v_vTexcoord;
varying vec4 v_vColour;
varying float v_vIllumination;


uniform sampler2D u_ShadowMap;

void main()
... ... ...
ob_Draw3D - Create
... ... ...
dirl_LightForward = shader_get_uniform(

sh_directional_lighting, "u_LightForward");
dirl_LightRight = shader_get_uniform(
sh_directional_lighting, "u_LightRight");
dirl_LightUp = shader_get_uniform(
sh_directional_lighting, "u_LightUp");
dirl_LightPosition = shader_get_uniform(
sh_directional_lighting, "u_LightPosition");
dirl_LightSize = shader_get_uniform(
sh_directional_lighting, "u_LightSize");
dirl_LightLenght = shader_get_uniform(
sh_directional_lighting, "u_LightLenght");
dirl_ShadowMap = shader_get_sampler_index(
sh_directional_lighting, "u_ShadowMap");
... ... ...
ob_Draw3D - Draw
... ... ...
shader_set(sh_directional_lighting);
shader_set_uniform_f_array(dirl_LightForward, lightForward);
shader_set_uniform_f_array(dirl_LightRight, lightRight);
shader_set_uniform_f_array(dirl_LightUp, lightUp);
shader_set_uniform_f_array(dirl_LightPosition, lightPosition);
shader_set_uniform_f( dirl_LightSize, LIGHT_SIZE);
shader_set_uniform_f( dirl_LightLenght, LIGHT_LENGHT);
var texShadowMap = surface_get_texture(surfShadowMap);
texture_set_stage( dirl_ShadowMap, texShadowMap);
with (ob_Mesh) event_user(0);
shader_reset();
... ... ...

To build cast shadows, first, calculate the light depth in sh_directional_lighting, just like we did it in sh_shadow_mapping.

sh_directional_lighting.vsh
... ... ...
varying vec2 v_vTexcoord;
varying vec4 v_vColour;
varying float v_vIllumination;

varying float v_vLightDepth;
... ... ...
void main()
{

vec4 object_space_pos = vec4( in_Position.x, in_Position.y, in_Position.z, 1.0);
vec4 world_space_pos = gm_Matrices[MATRIX_WORLD] * object_space_pos;
vec3 world_space_norm = normalize(mat3(gm_Matrices[MATRIX_WORLD]) * in_Normal);
gl_Position = gm_Matrices[MATRIX_WORLD_VIEW_PROJECTION] * object_space_pos;
float illumination = -dot(world_space_norm, u_LightForward);
vec3 light_to_object = vec3(world_space_pos) - u_LightPosition;
float light_depth = dot(light_to_object, u_LightForward);
float light_depth_01 = clamp(light_depth / u_LightLenght, 0.0, 1.0);
v_vColour = in_Colour;
v_vTexcoord = in_TextureCoord;
v_vIllumination = illumination;
v_vLightDepth = light_depth_01;
}

Next, get the depth value from the map. To do this, take the world-space vertex (light_to_object) and find a dot product with right and up vectors (u_LightRight, u_LightRight). Divide the resulting 2D coordinates by the map size (u_LightSize) and add 0.5. Now you got the UV texture coordinates for the map. Pass them to the fragment shader.

sh_directional_lighting.vsh
... ... ...
varying vec2 v_vTexcoord;
varying vec4 v_vColour;
varying float v_vIllumination;
varying float v_vLightDepth;

varying vec2 v_vLightMapPosition;
... ... ...
void main()
{

... ... ...
float light_depth = dot(light_to_object, u_LightForward);
float light_depth_01 = clamp(light_depth / u_LightLenght, 0.0, 1.0);
float light_map_U = 0.5 + dot(light_to_object, u_LightRight) / u_LightSize;
float light_map_V = 0.5 + dot(light_to_object, u_LightUp ) / u_LightSize;
v_vColour = in_Colour;
v_vTexcoord = in_TextureCoord;
v_vIllumination = illumination;
v_vLightDepth = light_depth_01;
v_vLightMapPosition = vec2(light_map_U, light_map_V);
}

In the fragment shader, just show the map projection for now.

sh_directional_lighting.fsh
varying vec2 v_vTexcoord;
varying vec4 v_vColour;
varying float v_vIllumination;

varying vec2 v_vLightMapPosition;

uniform sampler2D u_ShadowMap;

void main()
{

gl_FragColor = texture2D(u_ShadowMap, v_vLightMapPosition);
//gl_FragColor = v_vColour * texture2D(gm_BaseTexture, v_vTexcoord);
//float smooth_illumination = smoothstep(0.0, 0.8, v_vIllumination);
//gl_FragColor.rgb *= mix(0.3, 1.0, smooth_illumination);
}

The projection turned out to be incorrect: the farthest corner of the mesh ended up near the light. The UV coordinates need to be flipped vertically.

sh_directional_lighting.vsh
... ... ...
float light_map_U = 0.5 + dot(light_to_object, u_LightRight) / u_LightSize;
float light_map_V = 0.5 - dot(light_to_object, u_LightUp ) / u_LightSize;
... ... ...

Now we getting somewhere! You can even see the outline of the shadow. The only thing left to do is to compare v_vLightDistance to the map value. If v_vLightDistance is greater, then a fragment is in a cast shadow.

sh_directional_lighting.fsh
varying vec2 v_vTexcoord;
varying vec4 v_vColour;
varying float v_vIllumination;

varying float v_vLightDepth;
varying vec2 v_vLightMapPosition;
... ... ...
void main()
{

float depth = texture2D(u_ShadowMap, v_vLightMapPosition).r;
//gl_FragColor = v_vColour * texture2D(gm_BaseTexture, v_vTexcoord);
if (v_vLightDepth > depth)
{
gl_FragColor.rgb *= 0.3;
}
else
{
//float smooth_illumination = smoothstep(0.0, 0.8, v_vIllumination);
//gl_FragColor.rgb *= mix(0.3, 1.0, smooth_illumination);
}
}

Well, we got shadows, but everything is covered with strange patterns. There are a couple of reasons for this.
The first problem is a limited depth resolution. Currently, while creating the shadow map, we write the same value in all three RGB channels. In fact, of the available four bytes per pixel, we use only one. Use the functions below to encode map values in the sh_shadow_mapping shader and decode them in the sh_directional_lighting shader.
packFloatInto8BitVec3(val01) — takes a float number in a range of 0 to 1 and returns the corresponding RGB color.​
unpack8BitVec3IntoFloat(valRGB) — takes the RGB color and returns the corresponding float number in a range of 0 to 1.​

sh_shadow_mapping.fsh
varying float v_vLightDepth;

const float SCALE_FACTOR = 256.0 * 256.0 * 256.0 - 1.0;
vec3 packFloatInto8BitVec3(float val01)
{

float zeroTo24Bit = val01 * SCALE_FACTOR;
return floor(
vec3(
mod(zeroTo24Bit, 256.0),
mod(zeroTo24Bit / 256.0, 256.0),
zeroTo24Bit / 256.0 / 256.0
)
) / 255.0;
}

void main()
{

vec3 packed_depth = packFloatInto8BitVec3(v_vLightDepth);
gl_FragColor = vec4(packed_depth, 1.0);
}
sh_directional_lighting.fsh
... ... ...
const float SCALE_FACTOR = 256.0 * 256.0 * 256.0 - 1.0;
const vec3 SCALE_VECTOR = vec3(1.0, 256.0, 256.0 * 256.0) / SCALE_FACTOR * 255.0;
float unpack8BitVec3IntoFloat(vec3 valRGB)
{

return dot(valRGB, SCALE_VECTOR);
}

void main()
{

vec3 packed_depth = texture2D(u_LightMap, v_vLightMapPosition).rgb;
float depth = unpack8BitVec3IntoFloat(packed_depth);
... ... ...
}

The next problem is a limited shadow map resolution.

A simple way to deal with it is to shift the map value slightly forward. This will get rid of the majority of unwanted shadows.

sh_directional_lighting.fsh
... ... ...
if (v_vLightDepth > depth + 0.005)
... ... ...

Finally, hide remaining edge case artifacts in the form-shadows. To do this, just raise a minimal illumination a bit.

sh_directional_lighting.fsh
... ... ...
else
{

float smooth_illumination = smoothstep(0.1, 0.8, v_vIllumination);
gl_FragColor.rgb *= mix(0.3, 1.0, smooth_illumination);
}
... ... ...

And this is it! The cast-shadows are done!


Conclusion

In this tutorial, we covered one of the simplest and efficient ways to create directional lighting with a single light source in the GameMaker Studio 2. The code can be adapted for GMS 1.4, or become a base for more complex effects. Performance can be improved too. You can use/modify code from this tutorial, as you please.

I wish you the best of luck with your projects!
 

Dragonite

Member
I was just thinking that I want to start experimenting with shadows a few days ago. [casually bookmarks this thread for future reference]
 
  • Like
Reactions: GoK

Alvare

Member
Wow this looks great. However it's been a such a long time since I used Gamemaker. Last time before I quit gml, I recall porting that entire game: Iji to Gamemaker Studio by myself.
Unity is my favorite tool now.. But I have great memories of Gamemaker. It has taught me plenty. :)
 

GoK

Member
Wow this looks great. However it's been a such a long time since I used Gamemaker. Last time before I quit gml, I recall porting that entire game: Iji to Gamemaker Studio by myself.
Unity is my favorite tool now.. But I have great memories of Gamemaker. It has taught me plenty. :)
Yes, Unity is great. C# is a pleasure to code on, and the ability to create my own custom editors is insanely useful. But for some reason, I steel prefer GMS. :D
 

BenRK

Member
There are some parts of the tutorial that just do things without explaining what you're doing, BUT this tutorial helped me a ton and it finally clicked for me. It's not nearly as difficult as I thought it was going to be. Not simple, but I just couldn't reason how to do this with just one shader. "No, you gotta use two." D'OH! Of course! Thanks a ton for this! :D
 
  • Like
Reactions: GoK

GoK

Member
There are some parts of the tutorial that just do things without explaining what you're doing, BUT this tutorial helped me a ton and it finally clicked for me. It's not nearly as difficult as I thought it was going to be. Not simple, but I just couldn't reason how to do this with just one shader. "No, you gotta use two." D'OH! Of course! Thanks a ton for this! :D
Hi, BenRK! I'm very glad that my tutorial was helpful. If this is not too much problem, can you tell me which parts been underexplored? I'll try to improve them.
 

BenRK

Member
Hi, BenRK! I'm very glad that my tutorial was helpful. If this is not too much problem, can you tell me which parts been underexplored? I'll try to improve them.
Well, you do a lot of things without explaining what they do. Yeah it's ok to assume someone knows how to define an array, but you never explain what those values are going to be used for, for example. And there's no reason given why you always spell it as LIGHT_LENGHT instead of LIGHT_LENGTH. Also, maybe it's just because I was adapting it to my own project, but copying the code (or typing it out word for word) from the post into GMS2 would just throw a bunch of errors at me. I'm sure I could figure a lot of those out if you explained what was going on. In the end, at some points, I threw out what you wrote and just wrote what I thought was correct.

Another example is I don't understand how you set up the ortho projection. I clearly got it to work more or less as you can see from the screenshot, but I'm having trouble getting it to actually render the scene if it's further away then it currently is. I can figure out positioning and everything, but too far and it just doesn't render.

1586161066109.png

I feel like my implementation is still incomplete, but even so, this tutorial got me on my feet. I still feel like I was just following along in the end and I'll have to adjust things, BUT I am still very happy to have the tutorial!
 
  • Like
Reactions: GoK
A

Aamon

Guest
This is actually extremely good. I'm not very good at the 3D stuff yet -- I'd be willing to give you at least hundred bucks if you can get this to work with colored spotlights and texture-shaped shadows (like a tree sprite texture casting its shape as a shadow, ignoring the zero alpha portions of the texture). But really, this is very good -- great job.
 
  • Like
Reactions: GoK

GoK

Member
This is actually extremely good. I'm not very good at the 3D stuff yet -- I'd be willing to give you at least hundred bucks if you can get this to work with colored spotlights and texture-shaped shadows (like a tree sprite texture casting its shape as a shadow, ignoring the zero alpha portions of the texture). But really, this is very good -- great job.
It seems pretty straightforward. Give me a couple of days, I'm a bit busy right now.

--- Update ---
Hi, writing from the rabbit hole I'm currently in. Turns out, it's not as straightforward as I thought, and pretty much a whole thing needs to be rewritten. Still doable though.
 
Last edited:
A

Aamon

Guest
It seems pretty straightforward. Give me a couple of days, I'm a bit busy right now.

--- Update ---
Hi, writing from the rabbit hole I'm currently in. Turns out, it's not as straightforward as I thought, and pretty much a whole thing needs to be rewritten. Still doable though.
I thought it would be pretty tough -- I'm serious about giving you money if you can pull it off. Good luck with it, it's beyond my skills.
 

GoK

Member
Well, call me Frankenstein, because my monster IS ALIVE too!



What's new in this version:
  • code has been rewritten using 2.3 features;
  • transparent textures (alpha = 0);
  • multiple lights;
  • colored lights with color masks;
  • directional and spotlights at the same time (point lights are not implemented yet, but for now you can fake their effect by placing 6 spotlights of size 1 in a cube formation)
Here is a download link:
3D_Lighting_and_Shadows.yyz (for GMS 2.3).​

Controls:
Mouse — orbits the camera around;​
WASD — rotates the model;​
Space — create a random light source;​
Backspace — delete a random light source.​

Spotlight shadows work pretty much the same way as directional light but use perspective projection instead of orthographic. Transparency is easy too: all you need is to discard fragments with alpha = 0 in both mapping and lighting fragment-shaders. To show multiple lights at the same time you need to draw a scene without lights first, then draw it again for every light with additive blend mode. Finally, color masks allow you to project sprites onto the scene:
To add a new light use these functions:
Create_LightDirectional(_size, _quality, _power, _color, _mask, _from, _to, _up) — for directional light;​
Create_LightSpot(_size, _quality, _power, _color, _mask, _from, _to, _up) — for spotlight.​
Arguments are:
_size — how wide the light is (preferably, use powers of 2 (0.125, 0.25, 0.5, 1, 2, 4, 8), for spotlights — _size = 1 means 90 degrees FOV);​
_quality — controls, how much larger a shadow map than light (preferably, use powers of 2 (1, 2, 4, 8), spotlights usually need higher quality, than directional ones);​
_power — controls, how bright the light is;​
_color, _mask — the color of the light;​
_from, _to, _up — the position and orientation of the light;​
To delete a light use instance_destroy(light_id), and to temporarily "turn off" a light use instance_deactivate_object(light_id) (naturally, for performance sake you want to use as fewer lights as possible at the same time).

What's next:
  • in the future, I want to implement point lights;
  • experiment with the ways to improve performance;
  • rewrite a whole tutorial on directional lighting.
For now, you're welcome to ask any additional questions about how this whole thing works.
 
A

Aamon

Guest
My god, it's beautiful. Excellent work, if you link me up with some way of paying you, I will do it. Seriously impressive stuff.
 

Kentae

Member
Hi, I just followed this tutorial and got the shadows working. They look great but is there a way to just not "process" the areas outside the light camera's boundries?
I'm getting alot of unwanted shadows in these places and it does kind of throw the effect a bit of at times :)
Great tutorial though ^^
 

GoK

Member
Hi, Kentae! If I understand you correctly, you can take a depth value from the lightmap only if coordinates (v_vLightMapPosition) in 0-1 bounds, otherwise just set it to some default value.

sh_directional_lighting.fsh:
... ... ...

void main()
{

float depth = 1.0;
if (
v_vLightMapPosition.x >= 0.0 && v_vLightMapPosition.x <= 1.0 &&
v_vLightMapPosition.y >= 0.0 && v_vLightMapPosition.y <= 1.0)
{
vec3 packed_depth = texture2D(u_LightMap, v_vLightMapPosition).rgb;
float depth = unpack8BitVec3IntoFloat(packed_depth);
}
... ... ...
}
 

Kentae

Member
@GoK Hey! Thanks for the reply ^^
I've actually found another way of doing it. there are some oddities about it though so I might experiment a bit more with it :)
 

RetroBatter

Member
Well, call me Frankenstein, because my monster IS ALIVE too!



What's new in this version:
  • code has been rewritten using 2.3 features;
  • transparent textures (alpha = 0);
  • multiple lights;
  • colored lights with color masks;
  • directional and spotlights at the same time (point lights are not implemented yet, but for now you can fake their effect by placing 6 spotlights of size 1 in a cube formation)
Here is a download link:
3D_Lighting_and_Shadows.yyz (for GMS 2.3).​

Controls:
Mouse — orbits the camera around;​
WASD — rotates the model;​
Space — create a random light source;​
Backspace — delete a random light source.​

Spotlight shadows work pretty much the same way as directional light but use perspective projection instead of orthographic. Transparency is easy too: all you need is to discard fragments with alpha = 0 in both mapping and lighting fragment-shaders. To show multiple lights at the same time you need to draw a scene without lights first, then draw it again for every light with additive blend mode. Finally, color masks allow you to project sprites onto the scene:
To add a new light use these functions:
Create_LightDirectional(_size, _quality, _power, _color, _mask, _from, _to, _up) — for directional light;​
Create_LightSpot(_size, _quality, _power, _color, _mask, _from, _to, _up) — for spotlight.​
Arguments are:
_size — how wide the light is (preferably, use powers of 2 (0.125, 0.25, 0.5, 1, 2, 4, 8), for spotlights — _size = 1 means 90 degrees FOV);​
_quality — controls, how much larger a shadow map than light (preferably, use powers of 2 (1, 2, 4, 8), spotlights usually need higher quality, than directional ones);​
_power — controls, how bright the light is;​
_color, _mask — the color of the light;​
_from, _to, _up — the position and orientation of the light;​
To delete a light use instance_destroy(light_id), and to temporarily "turn off" a light use instance_deactivate_object(light_id) (naturally, for performance sake you want to use as fewer lights as possible at the same time).

What's next:
  • in the future, I want to implement point lights;
  • experiment with the ways to improve performance;
  • rewrite a whole tutorial on directional lighting.
For now, you're welcome to ask any additional questions about how this whole thing works.
Is there going to be a new tutorial for this? I'm not sure how to get this working with the rewritten code. Nevermind, I figured it out lol
 
Last edited:
Well, call me Frankenstein, because my monster IS ALIVE too!



What's new in this version:
  • code has been rewritten using 2.3 features;
  • transparent textures (alpha = 0);
  • multiple lights;
  • colored lights with color masks;
  • directional and spotlights at the same time (point lights are not implemented yet, but for now you can fake their effect by placing 6 spotlights of size 1 in a cube formation)
Here is a download link:
3D_Lighting_and_Shadows.yyz (for GMS 2.3).​

Controls:
Mouse — orbits the camera around;​
WASD — rotates the model;​
Space — create a random light source;​
Backspace — delete a random light source.​

Spotlight shadows work pretty much the same way as directional light but use perspective projection instead of orthographic. Transparency is easy too: all you need is to discard fragments with alpha = 0 in both mapping and lighting fragment-shaders. To show multiple lights at the same time you need to draw a scene without lights first, then draw it again for every light with additive blend mode. Finally, color masks allow you to project sprites onto the scene:
To add a new light use these functions:
Create_LightDirectional(_size, _quality, _power, _color, _mask, _from, _to, _up) — for directional light;​
Create_LightSpot(_size, _quality, _power, _color, _mask, _from, _to, _up) — for spotlight.​
Arguments are:
_size — how wide the light is (preferably, use powers of 2 (0.125, 0.25, 0.5, 1, 2, 4, 8), for spotlights — _size = 1 means 90 degrees FOV);​
_quality — controls, how much larger a shadow map than light (preferably, use powers of 2 (1, 2, 4, 8), spotlights usually need higher quality, than directional ones);​
_power — controls, how bright the light is;​
_color, _mask — the color of the light;​
_from, _to, _up — the position and orientation of the light;​
To delete a light use instance_destroy(light_id), and to temporarily "turn off" a light use instance_deactivate_object(light_id) (naturally, for performance sake you want to use as fewer lights as possible at the same time).

What's next:
  • in the future, I want to implement point lights;
  • experiment with the ways to improve performance;
  • rewrite a whole tutorial on directional lighting.
For now, you're welcome to ask any additional questions about how this whole thing works.
Hello! Have you updated this? How can i improve the old one with that? I can just follow the first tutorial and then the updated one? Thank you!
 
Top