3D matrix_transform_vertex, matrix_multiply and system handedness

Bart

WiseBart
Hi all.

I'm trying to use matrix_transform_vertex to apply the model-view-projection matrix to a point, similar to what happens in the vertex shader.
For some reason the only point that I can get right is (0, 0, 0). And I currently don't see what is wrong.

Here is my current code:
Code:
/// Create event
cam = view_camera[0];

///*

//mat_proj = matrix_build_projection_perspective_fov(45, 16/9, .1, 32000);
mat_proj = matrix_build_projection_perspective(1280, 720, 200, 32000);
mat_view = matrix_build_lookat(x, y, 0, 0, 0, 0, 0, 0, -1);

camera_set_proj_mat(cam, mat_proj);
camera_set_view_mat(cam, mat_view);

mat_world = matrix_get(matrix_world);                    // identity matrix by default

mat_transform = matrix_multiply(mat_view, mat_proj);    // mat_transform = mat_proj * mat_view;
mat_transform = matrix_multiply(mat_world, mat_transform);

co = [0, 0, 0];

//*/
Code:
/// @description Draw GUI - draw indicator sprite
vert = matrix_transform_vertex(mat_transform, co[0], co[1], co[2]);

draw_sprite(spr_indicator, 0, view_wport[0]/2 - vert[0], view_hport[0]/2 - vert[1]);  // lookat point is at center of screen

draw_text(5, 5, "In: " + string(co));
draw_text(5, 25, "Out: " + string(vert));
The camera is located at (352, 0, 0) and looking towards (0, 0, 0).
A cube of size 100 is drawn at the origin with its center point there.

Using the above code, I'd expect one of the corner points, e.g. (50, 50, -50), to be drawn at the exact location on screen in the Draw GUI event. But it's not:
screen_of_issue.png
Moving the point forward and backward along +x should result in a slight change along the horizontal axis because of the perspective projection but nothing seems to change.

The GUI and the viewport are the same size: 1280 x 720.

Anyone has some insight on what is going wrong here?
It seems like I'm missing something.
 
According to the docs,matrix_transform_vertex() does not appear designed to carry out the operation that you require. It should have a w component for both the input vector and output vector. But it doesn't.

So, you can make your own script to do the matrix-vector multiplication:
(m = matrix, (x,y,z,w) = input vector. 'w' should be equal to 1)

x1 = x*m[0] + y*m[4] + z*m[8] + w*m[12]
y1 = x*m[1] + y*m[5] + z*m[9] + w*m[13]
z1 = x*m[2] + y*m[6] + z*m[10] + w*m[14]
w1 = x*m[3] + y*m[7] + z*m[11] + w*m[15]

And if you are projecting a point using a perspective projection, you can find the coordinates within the view port like this:
In GMS2, you might need to invert the negative sign in front of the 0.5, in case the point is flipped vertically across the center of the screen.
port_w and port_h are the size of the render target that you are drawing onto.

port_x = (x1/w1*0.5 + 0.5)*port_w
port_y = (y1/w1*-0.5 + 0.5)*port_h

Note that z1 is not used, so you can leave it out of your matrix-vector multiplication. Also, since w will equal 1, you don't need to multiply m[12] through m[15] by w.

Also, if w1 is < 0, then the point is behind the camera. You might want to add an if statement to catch that condition, otherwise you'll get points that shouldn't be visible.
 

Bart

WiseBart
Ah, of course. You're right. That w value is required. 1 for a position, 0 for a direction.

And yet, matrix_transform_vertex may already be using that component in the background, setting it to 1 implicitly. Then I guess it should work?...

Thanks for the explanation and the formula, but your answer has got me thinking. As an alternative, shouldn't it be possible to use matrix_multiply instead?
If you consider both input and output vector as 4x4 matrices and use the first column to store the vector, including the value of w.

I've been looking for a solution that uses matrix_multiply but no success so far.
First things first:
Code:
// Apparently this:
mat_transform = matrix_multiply(mat_view, mat_proj);    // mat_transform = mat_proj * mat_view;
// Should really be this:
mat_transform = matrix_multiply(mat_proj, mat_view);    // mat_transform = mat_proj * mat_view;
Then I tried to get the vertex in a matrix representation (using the first column) and multiply the matrices:
Code:
co_h = [
    co[0], 0, 0, 0,
    co[1], 0, 0, 0,
    co[2], 0, 0, 0,
        1, 0, 0, 0,
];
vert_h = matrix_multiply(mat_transform, co_h);  // Should be equivalent to this (?): gl_Position = gm_Matrices[MATRIX_WORLD_VIEW_PROJECTION] * object_space_pos;
vert_h = [vert_h[0], vert_h[4], vert_h[8], vert_h[12]];  // Make it an actual "vec4"

var x1 = vert_h[0], y1 = vert_h[1], z1 = vert_h[2], w1 = vert_h[3];
port_x = (x1/w1*0.5 + 0.5)*1280;
port_y = (y1/w1*-0.5 + 0.5)*720;

//draw_sprite(spr_indicator, 0, view_wport[0]/2 - vert_h[0], view_hport[0]/2 - vert_h[1]);
draw_sprite(spr_indicator, 0, port_x, port_y);
But a quick visual check clearly shows that it's still not working as it should yet.

There must be a flaw in my reasoning or in the code but I don't see where.

EDIT:
Meanwhile, I've been testing with matrix_transform_vertex and it appears that GM does vertex * matrix instead of matrix * vertex? If that's the case, it does seem like the w component is set to 1 in the background. Or did I define the matrix the wrong way?
Code:
mat_test_w = [
    1, 0, 0, 0,
    0, 1, 0, 0,
    0, 0, 1, 0,
    1, 1, 1, 1,
];

/*
mat_test_w = [
    1, 0, 0, 1,
    0, 1, 0, 1,
    0, 0, 1, 1,
    0, 0, 0, 1,
];
*/

vtx_test_w = [16, 0, 8, 1];

vtx_result = matrix_transform_vertex(mat_test_w, vtx_test_w[0], vtx_test_w[1], vtx_test_w[2]);

show_debug_message(vtx_result);
// { { 17,1,9 },  }
 
Last edited:
If you want to try the matrix multiply thing, you have to remember a couple of things.

Code:
co_h = [
   co[0], 0, 0, 0,
   co[1], 0, 0, 0,
   co[2], 0, 0, 0,
       1, 0, 0, 0,
];
That is actually encoding co[0], co[1], co[2], 1, into the first row, not the first column. To do the first column, it should look like this:

Code:
co_h = [
   co[0], co[1], co[2], 1,
   0, 0, 0, 0,
   0, 0, 0, 0,
   0, 0, 0, 0,
];
Then matrix multiple should (I believe) have the arguments go in this order:

matrix_multiply(co_h, mat_transform);

The arguments to matrix_multiply go in backwards order from how you'd write it by hand. Maybe this has to do with function arguments being evaluated from right to left, I'm not sure.

So, I believe the following statement is not correct:
Code:
// Apparently this:
mat_transform = matrix_multiply(mat_view, mat_proj);    // mat_transform = mat_proj * mat_view;
// Should really be this:
mat_transform = matrix_multiply(mat_proj, mat_view);    // mat_transform = mat_proj * mat_view;
There is no matrix_transform_vertex in GMS1, which is what I'm using. So I can't tell you very much more about it. Unfortunately, the documentation is not sufficient to explain unambiguously what the function is supposed to do. It obviously cannot be a generic 4x4 matrix and row or column vector multiplication. And if it was designed to be used with perspective projections, it must return a w component, which it doesn't.
 
Last edited:

Bart

WiseBart
If you want to try the matrix multiply thing, you have to remember a couple of things.

Code:
co_h = [
co[0], 0, 0, 0,
co[1], 0, 0, 0,
co[2], 0, 0, 0,
1, 0, 0, 0,
];
That is actually encoding co[0], co[1], co[2], 1, into the first row, not the first column. To do the first column, it should look like this:

Code:
co_h = [
co[0], co[1], co[2], 1,
0, 0, 0, 0,
0, 0, 0, 0,
0, 0, 0, 0,
];
Hm... That doesn't correspond with what I tested. I'm using GMS2, I'm not sure if there's a difference with 1.4.
But the manual on matrix_build states that:
The matrix itself is created as 16 value 1D array where the first 4 elements are row 1, the second 4 elements are row 2 etc... of a 4x4 matrix.

I did some more testing.
As an example, take the point (10, 8, 0) and a simple rotation of 90° about the z axis as the transformation.
Normally, you'd do matrix * vertex:
Code:
_x = 10;
_y = 8;
_z = 0;

mat_rotz = matrix_build(0, 0, 0, 0, 0, 90, 1, 1, 1);
/*
[
     0, -1,  0,  0,
     1,  0,  0,  0,
     0,  0,  1,  0,
     0,  0,  0,  1,
]
*/

mat_vertex = [
    _x, 0, 0, 0,
    _y, 0, 0, 0,
    _z, 0, 0, 0,
     1, 0, 0, 0,
];

mat_result = matrix_multiply(mat_rotz, mat_vertex);
/*
[
    -8, 0, 0, 0,
    10, 0, 0, 0,
     0, 0, 0, 0,
     1, 0, 0, 0,
]
*/
The result is consistent with how you'd calculate this on paper (using a right-handed axes system).

matrix_transform_vertex returns something different with the same input values, as if the rotation goes the other way:
Code:
mat_result2 = matrix_transform_vertex(mat_rotz, _x, _y, _z);
/*
[
     8.00,
    -10,
     0
]
*/
Which seems to be consistent with how direction and image_angle are handled in a topdown 2d projection (+x right, +y down, +z into screen): adding a positive angle rotates counterclockwise.

So it seems like matrix_multiply and matrix_transform_vertex do the same thing (at least without the w component), but matrix_transform_vertex seems to invert rotations.

I guess 'll need to use matrix_multiply.
 
How are you verifying the configuration of the matrix?

The matrix itself is created as 16 value 1D array where the first 4 elements are row 1, the second 4 elements are row 2 etc... of a 4x4 matrix.
I am almost 100% certain this is wrong. The first 4 elements are actually column 1.

Code:
_x = 10;
_y = 8;
_z = 0;

mat_rotz = matrix_build(0, 0, 0, 0, 0, 90, 1, 1, 1);
/*
[
    0, -1,  0,  0,
    1,  0,  0,  0,
    0,  0,  1,  0,
    0,  0,  0,  1,
]
*/

mat_vertex = [
   _x, 0, 0, 0,
   _y, 0, 0, 0,
   _z, 0, 0, 0,
    1, 0, 0, 0,
];

mat_result = matrix_multiply(mat_rotz, mat_vertex);
/*
[
   -8, 0, 0, 0,
   10, 0, 0, 0,
    0, 0, 0, 0,
    1, 0, 0, 0,
]
*/
What's going on there is consistent with the following:
Code:
x y z 1   0  1 0 0       -y x z 1
0 0 0 0   -1 0 0 0   =   0  0 0 0
0 0 0 0   0  0 1 0       0  0 0 0
0 0 0 0   0  0 0 1       0  0 0 0
Your vector as the first row on the left side matrix (you've actually swapped columns for rows). And your rotation matrix on the middle, results in the matrix on the right.

The rotation matrix shown (when multiplied with a column vector) will produce anticlockwise rotations with positive angles of rotation when viewed from the negative z direction, in a right handed coordinate system.

You've multiplied it with (what amounts to) a row vector, which (when you extract the vector from the first row of the resultant matrix), has produced a clockwise rotation.

Your experiment with matrix_transform_vertex is producing correct results if you presume it is a matrix multiplied with a column vector.
 

Bart

WiseBart
How are you verifying the configuration of the matrix?

I am almost 100% certain this is wrong. The first 4 elements are actually column 1.
If you have a look at the axes in the room editor, you can derive that GM uses a right-handed coordinate system; with +x pointing to the right and +y downward, +z points into the screen (depth).
The formula for a rotation about z in a right-handed coordinate system is this one.
So at row 0, column 1, there's -sin(angle), which becomes -1 for an angle = 90.
matrix_build returns the following as output (it's output from the debugger, but I added some formatting for clarity):
Code:
mat_rotz = matrix_build(0, 0, 0, 0, 0, 90, 1, 1, 1);
/*
[
     0, -1,  0,  0,
     1,  0,  0,  0,
     0,  0,  1,  0,
     0,  0,  0,  1,
]
*/
Seeing that the value at array index 1 is equal to -1 must mean that the quote from the manual is true:
The matrix itself is created as 16 value 1D array where the first 4 elements are row 1, the second 4 elements are row 2 etc... of a 4x4 matrix.
I'd think that's how it should be interpreted, since everything does end up being consistent.

What's going on there is consistent with the following:
I think we're both doing the exact same thing, with the only real difference being a transpose.
 
The coordinate sysem is right-handed.

But given the matrices used in the built-in functions, a positive angle of rotation will result in an anti-clockwise rotation, when a matrix is multiplied with a column vector. If you apply a +90 degree rotation to what is currently being drawn, it will rotate 90 degrees anti-clockwise.

Because it is presumed in the built-in functions that matrices are column major ordered. This is why the matrix_transform_vertex function returns rotations in the opposite direciton than what you are expecting.

You can see that if you confuse column and row ordering, and use matrix_multiply(B,A) with the reversed arguments: matrix_multiply(A,B), it will produce identical results:

Code:
x y z 1   0  1 0 0      -y x z 1
0 0 0 0   -1 0 0 0      0  0 0 0
0 0 0 0   0  0 1 0      0  0 0 0
0 0 0 0   0  0 0 1      0  0 0 0

0 -1 0 0   x 0 0  0   -y 0 0 0
1  0 0 0   y 0 0  0    x 0 0 0
0  0 1 0   z 0 0  0    z 0 0 0
0  0 0 1   1 0 0  0    1 0 0 0
The same results, IF you confuse columns for rows and reverse order of multiplication.

I suppose, it is up to you if you want to do it that way. But, it will not be consistent with how matrices are used in other parts of gamemaker. You will get clockwise rotations with positive angles, whereas everywhere else in gamemaker (except box2d), positive angles result in an anti-clockwise rotation.

Final edit:

You've really got me wondering if I could have got something wrong somewhere. But, assuming we're doing multiplications of matrices with column vectors, then the matrices in gamemaker must be column major ordered.
 
Last edited:

Bart

WiseBart
Thanks again for your insights!

All this has made me pretty confused, to say the least :)

I can indeed get rid of the confusion by assuming that the matrices are column-major ordered.
(That would indeed make things consistent with how GM works, yet not with Box2D, which properly uses a right-handed coordinate system)
But they're not:
Code:
// Formula for a rotation about z-axis over +90° in a right-handed coordinate system
// https://en.wikipedia.org/wiki/Rotation_matrix#Basic_rotations
rotz90 = [
    cos(90), -sin(90), 0, 0,
    sin(90),  cos(90), 0, 0,
          0,        0, 1, 0,
          0,        0, 0, 1,
];

// Formatted output of matrix_build(0, 0, 0, 0, 0, 90, 1, 1, 1)
mat_w = [
    0, -1,  0,  0,
    1,  0,  0,  0,
    0,  0,  1,  0,
    0,  0,  0,  1,
];

// The fact that the above two matrices are the same is the reason for my assumption that the manual is right on the row-major order

// To get correct results with the above transformation matrices, you do need to multiply with a column vector and in this order: matrix * column_vector
I find it confusing that the manual states that matrices are ordered row-major and when comparing the contents of the underlying array with the formula for a positive rotation about z in a right-handed coordinate system, determine that the actual visual rotation, as seen from -z (looking into the screen), turns out to be counter-clockwise.
As if the y axis is treated as pointing up, while it is really pointing down (obviously, this is how GM works when using e.g. direction, point_direction, lengthdir_x/_y, ...).

Here's the most basic code to check all this:
Code:
/// Create Event
mat_w = matrix_build(0, 0, 0, 0, 0, 90, 1, 1, 1);

/// Draw Event
matrix_set(matrix_world, mat_w);    // rotates counter-clockwise visually, while you'd expect clockwise

    draw_self();

matrix_set(matrix_world, matrix_build_identity());
If you use [1, 0, 0] (as a column vector) with the above transformation matrix, the result is [0, 1, 0] and yet, visually, the sprite ends up at [0, -1, 0].
But where does that inversion take place in the above code? It's not encoded anywhere in the transformation matrix. It seems to happen somewhere on the GM side.

This post from quite a while ago seems to explain that, as it suggests that GameMaker uses an inverted right-handed coordinate system.

This discussion may be going a bit too much in-depth but I'm really looking for clarity on how all the coordinate systems and functions are related before I start writing scripts to work with all this.

EDIT: Hmm. I may start to understand what you're getting at. I'm going to have a further look at this.

EDIT2: All right. So I couldn't let go of this thing and after a bit of searching, I think I know what is going on here.
As you state, matrices are in fact treated by all functions as column-major ordered, but in memory (i.e. in the underlying array) they're stored in row-major order. I think that is why the items in the array (the low-level memory) are ordered the way they are.
It's based on a couple of references that I found:
https://stackoverflow.com/questions/37264594/webgl-memory-notation-for-matrices/37281018#37281018
https://stackoverflow.com/questions...athematical-matrix-notation/48367941#48367941

Especially this one:
https://www.khronos.org/opengl/wiki/Interface_Block_(GLSL)#Matrix_storage_order

And this one to explain why in GM it would be in row-major order:

I'd say that's the only plausible explanation to explain both your arguments and mine.
 
Last edited:
OK, I don't like that interpretation, because it would mean that you need to reverse the order of all multiplications, in order to get the same results.

example:
Code:
A C   e g    A*e+C*f A*g+C*h
B D   f h    B*e+B*f B*g+D*h

A B   e f    A*e+B*g A*f+B*h
C D   g h    C*e+D*g C*f+D*h

e f  A B     e*A+f*C e*B+f*D
g h  C D     g*A+h*C g*B+h*D
EDIT: And that will only work with square matrices.

Unless you just presume that the right way to multiply row-major matrices is to reverse the order of multiplication. But if that is the case, then there's really no difference between whether you presume the matrices in gamemaker are row or column major ordered.

I'm just going to stick with thinking matrices are layed out like:
Code:
0, 4, 8,12
1, 5, 9,13
2, 6,10,14
3, 7,11,15
because it is most consistent with everything else I know about how to use matrices.
 
Last edited:
Top