**Projection matrix with oblique near clip plane**

I've been working on a portal system lately, and figured I'd share some of my findings.

When rendering planar world-space reflections, you usually need to create a reflected virtual camera at the other side of the mirror, sea level or portal, and render the world from this point of view to a surface. This surface is then drawn only within the confines of the mirror's surface, resulting in a realistic real-time reflection.

This method is excellent if there is nothing behind the mirror. By enabling counter-clockwise culling, the mirror itself and the wall behind it won't be drawn to the reflection surface, and you're left with a good-looking reflection:

However, what if there was something behind the mirror? Say there's a hallway at the other side of the wall. How would you prevent the walls and contents of the hallway from drawing to the reflection surface?

One way is to use a custom shader for the world in the virtual camera, with the following pseudo-code in the fragment shader:

Code:

`if (pixel is behind mirror){discard;}`

The graphics pipeline automatically transforms vertices to what's called normalized device coordinates, or NDC space. Once in this space, triangles are cut up so that only the part that is inside the NDC space is kept. Everything outside is culled. This is where you get stuff like the near and far planes, as well as the top, bottom, right and left planes. What if we could utilize this automatic process to cull everything behind the mirror?

In a standard projection matrix, the near and far planes are perpendicular to the looking direction, like so:

Now this isn't particularly useful...

However, there is a way to alter the projection matrix so that the clip planes look like this:

Now this is more like it! With this projection matrix, everything behind the mirror is culled, and we don't even need separate shaders for it! This is incredibly efficient, since it uses a culling process that would already happen anyway. The only problem with it is that the depth buffer gets messed up, and so this kind of projection cannot be used in conjuction with a regular projection. However, this is usually not an issue, since reflections and portals usually are rendered to separate surfaces anyway.

The code for this kind of projection is based on the process outlined on this page:

http://www.terathon.com/code/oblique.html

I haven't found it adapted to GML yet, so I did it myself. So here you go, the script for creating a projection matrix with an oblique near plane:

Code:

```
/// @description projection_make_oblique_near_clip_plane(projMat, clipX, clipY, clipZ, clipZ)
/// @param ProjMat
/// @param clipX
/// @param clipY
/// @param clipZ
/// @param clipW
/*
Modifies the given projection matrix so that it has an oblique near clip plane.
Must supply the camera-space 4D vector of the given plane, as well as the projection matrix.
The resulting projection matrix will mess up your depth buffer and any shaders that use it.
This is useful for virtual cameras like used in portals or reflections, to prevent anything behind the portal/mirror
from being rendered.
Source: http://www.terathon.com/code/oblique.html
Adapted to GML by Sindre Hauge Larsen
*/
var P = argument0;
var clipX, clipY, clipZ, clipW;
clipX = argument1;
clipY = argument2;
clipZ = argument3;
clipW = argument4;
//Calculate the clip-space corner point opposite the clipping plane
//as (sgn(clipPlane.x), sgn(clipPlane.y), 1, 1) and
//transform it into camera space by multiplying it
//by the inverse of the projection matrix
var Qx, Qy, Qz, Qw;
Qx = (sign(clipX) + P[8]) / P[0];
Qy = (sign(clipY) + P[9]) / P[5];
Qz = -1;
Qw = (1 + P[10]) / P[14];
//Replace the third row of the projection matrix
var c = 2 / (clipX * Qx + clipY * Qy + clipZ * Qz + clipW * Qw);
P[2] = clipX * c;
P[6] = clipY * c;
P[10] = clipZ * c;
P[14] = clipW * c;
return P;
```

**camera-space**clip plane.

Here's how to convert from world-space to camera-space clip plane:

Code:

```
//Get the current view and projection matrices
var V = matrix_get(matrix_view);
var P = matrix_get(matrix_projection);
//Find a camera-space position on the plane (it does not matter where on the clip plane, as long as it is on it)
//Wx, Wy, Wz is the world-space plane coordinate
var Px, Py, Pz;
Px = V[0] * Wx + V[4] * Wy + V[8] * Wz + V[12];
Py = V[1] * Wx + V[5] * Wy + V[9] * Wz + V[13];
Pz = V[2] * Wx + V[6] * Wy + V[10]* Wz + V[14];
//Find the camera-space 4D reflection plane vector
//Nx, Ny, Nz is the world-space normal of the plane
var Cx, Cy, Cz, Cw;
Cx = V[0] * Nx + V[4] * Ny + V[8] * Nz;
Cy = V[1] * Nx + V[5] * Ny + V[9] * Nz;
Cz = V[2] * Nx + V[6] * Ny + V[10]* Nz;
Cw = - Cx * Px - Cy * Py - Cz * Pz;
//Modify the projection matrix so that it uses an oblique near clip plane
var _P = projection_make_oblique_near_clip_plane(P, Cx, Cy, Cz, Cw);
matrix_set(matrix_projection, _P);
```

Here's a gif from my portal engine btw:

Last edited: