GMS 2 Normalizing view matrices (3d)

Hey guys! I hope this is complex enough for this section.

I am making a 2d top-down game with (important) 3D elements. This means I am using the matrix projections and all that.

My issue is that the lighting system I was using no longer works, as it doesn't match the world anymore.

Everyone told me it was fixable by using "camera_apply" however I used it in every possible place and yield no results, so I had someone help me.

He had "fixed" it (I put it in quotation marks because while it didn't completely match up and told me it was impossible to completely fix unless I rewrote it for 3d, however it was good enough for my purpose). The problem is he pretty much ghosted me and I don't want to bother him anymore.

He had told me that what he did was simply "normalized view matrices". I can't find any simple documentation online on how to do this and I don't understand the math at all. Some help would be greatly appreciated, as the game "engine" is pretty much complete (after months of work) after this is fixed.

I will post my camera code below.

create event: (I know there's alot of legacy GM1 stuff here...)

surface_resize(application_surface, global.resolution_w, global.resolution_h);
cameradifference = 0
//setting stuff
__view_set(e__VW.WView, 0, global.default_camera_w)
__view_set(- e__VW.HView, 0, -global.default_camera_h )
__view_set( e__VW.HBorder, 0, global.default_camera_w )
__view_set( e__VW.VBorder, 0, global.default_camera_h)
__view_set( e__VW.HSpeed, 0, -1 )
__view_set( e__VW.VSpeed, 0, -1 )
__view_set( e__VW.Object, 0, obj_camera )
__view_set( e__VW.Visible, 1, false )
__view_set( e__VW.Visible, 2, false )
view_enabled[1] = false
view_enabled[2] = false
__view_set( e__VW.Visible, 0, true );
view_enabled[0] = true;
// setting the splitscreen stuff
if !instance_exists(obj_slavecamera_p1) and instance_exists(obj_player1) instance_create(x,y,obj_slavecamera_p1)
if !instance_exists(obj_slavecamera_p2) and instance_exists(obj_player2) instance_create(x,y,obj_slavecamera_p2)

The update script:

obj_camera.cameradistance= (-(700+obj_camera.cameradifference + obj_camera.playermultizoom )* (obj_camera.carzoom))
mLookat = matrix_build_lookat(obj_camera.x,obj_camera.y,obj_camera.cameradistance,obj_camera.x,obj_camera.y-7,190,0,0,1)
//mLookat = matrix_build_lookat(obj_camera.x,obj_camera.y,-900,obj_camera.x,obj_camera.y-7,190,0,0,1)
//Assign the matrix to the camera. This updates were the camera is looking from, without having to unnecessarily update the projection.
camera_set_view_mat(view_camera[0], mLookat);
projMat = matrix_build_projection_perspective_fov(60, (global.default_camera_w)/global.default_camera_h, 32, 32000 );
camera_set_proj_mat(view_camera[0], projMat );
camera_apply( camera_get_active());

 

Without more information on how the lighting system works, my reply will carry limited value, but approaching this from a mathematical/logical standpoint, I want to clear a few things up:

• Transformation matrices used in a 3D projection are used to transform a coordinate from one coordinate space to another.
• A transformation from one coordinate space to another is not guaranteed to preserve properties such as relative distance. E.g. view space and world space retain relative scaling and distance, however coordinates in Clip-space/screen-space (i.e. after you have applied a perspective projection matrix) will not retain these properties.
• For these reasons, ANY coordinates that are being used in a calculation MUST be in the same coordinate space. i.e. if you have a shader which is performing distance calculations to calculate lighting, then the passed in light positions or whatever you may be using also need to be in the same space as the objects in the scene.
  --> A standard deferred lighting approach would work well in this situation, because the actual process works the same as normal 2D lighting wherein you render lights on top of the scene afterwards. The difference being however, is that you would need a conversion process to calculate relative coordinates to points on the scene from where the light is being drawn in screen-space.
  This remapping is normally done by encoding depth information into a depth process, and using the reverse of the previous matrix transformation to get back into a coordinate space that is consistent.
  

^ Basically master and understand this diagram, and 3D/shader programming becomes a whole lot easier.

I've personally never heard of "normalized view matrices", doesn't really make sense, as a standard view matrix will give you a normalized result, wherein the matrix transformation does not scale your values. The view matrix generated by the lookat function is certainly fine.

So, the list of things you can try to fix your lighting: (This won't be perfect, because there is more to it than that, but will atleast get your lights to line-up with the correct position in space)

1) In your light objects, specify their position as a position in 3D world space. If your level editor is top-down, then X and Y will already be correct, you just need to add in a z coordinate which specifies how high up or down it is. (I imagine you should already have this with other objects in your scene).

2) Transform that 3D position by using the two generated matrices: mLookat and projMat. The idea here is to replicate the process that the 3D objects are going through and "project" this 3D coordinate into a 2D coordinate on-screen. For this, USE THE DIAGRAM. This tells you EXACTLY what transformation process coordinates need to go through. We want to get from WORLD SPACE to VIEWPORT space. The steps to do so are as follows:

original_light_3d_position (in world sapce)
- multiply by view matrix. (GM has matrix multiply functions that you can use if you put your values in vector form as an array). the W-component (4th component in the vector) needs to be 1.
- multiply by the projection matrix
- Perform perspective divide -- by taking the current xyz coordinates of the light and dividing through the new w value. This is a normalization step, and removes any unwated scaling introduced during the projection stage.
- Now you should have a coordinate with xy values between -1 and 1. You simply need to map these into screen space to get your xy 2D position for your light.
- This can be done by just adding 1, dividing by 2 to get into a range of 0 to 1, and then multiplying by the screen width/height

3) Perform your lighting as you were before, but drawing the light at this new projected 2D screenspace coordinate.

This is the best answer I can give without seeing screenshots, or understanding what the intended behaviour of your lighting system is. Hope it helps!

 

Holy hell that's alot of information! i'll try something soon and let you know how it does