• Hey Guest! Ever feel like entering a Game Jam, but the time limit is always too much pressure? We get it... You lead a hectic life and dedicating 3 whole days to make a game just doesn't work for you! So, why not enter the GMC SLOW JAM? Take your time! Kick back and make your game over 4 months! Interested? Then just click here!

3D Get mouse X and Y in 3D space?

MilesThatch

Member
Ok so one thing to note is that in this particular instance, I do not need the Z coordinates of the mouse. I know that there is a marketplace asset out there that can do mouse collisions with a 3D object but in this case it's not what I am after.

Let's assume that the mouse Z is on a permanent plane, like z = 0. With Z in mind (haha), I need to project a line from x,y,z camera position, that moves through the mouse x and y coordinate in the window view and return the X and Y coordinate when that projection hits the z=0. That will effectively tell me what point in 3D space, the mouse is pointing at. That's all I need.

 
W

Wraithious

Guest
Hmm, I'm thinking you could use a kind of reflection setup for that, such as the x,y,z of the camera to the bottom middle of the arrow sprite and then draw a colision line from there pointing through the tip of the arrow and use the angle difference to determine where the arrow is pointing relitive to the camera xyz, and also use that to distort the arrow sprite's angle, height and width, but one question, is the arrow sprite 3d? If so instead of distorting the arrow you'd only need to change it's xyz rotation.
 

wadaltmon

Member
Alright... long post time. I know we talked about this on your stream, and I will admit that I am VERY rusty when it comes to all the math involved with this (I made these scripts a couple years ago now), but I believe I may have a set of scripts that can help you (from GMS 1.99 Beta, mind you, so you may have to modify them to accommodate GMS2). You'll also have to do a little bit of math on your part for your specific usage.

So from what you talked about on your stream, you want to have several tables on which supplies are located, and then you can click on the supplies on the table. If you know the height that the tables are at on the z-axis (and therefore, know the infinite plane on which they are located, and can simply check x/y coordinates to determine what table or item is being selected), these scripts should work with a little modification. They should also work with what you've mentioned in this thread of just trying to collide with the z=0 plane.

There are several scripts I have here that you can use, and you'd have to call them all in the proper order. I did this all in my draw event of my camera object, although you could do it in any of the step events too I expect. The first that you'd use is mouse_vector, which obtains the vector of the mouse leaving the screen in 3D space. This is defined similarly to your d3d_get_projection in GMS1; frankly, I am not sure what a 1-to-1 equivalent for that is in GMS2 (perhaps someone here on the forums can clue me in? I do not use GMS2), but here is the function call and arguments:

Code:
mouse_vector(xfrom, yfrom, zfrom, xto, yto, zto, xup, yup, zup, fov, aspect, display width, display_height, mouse x, mouse y, display origin vec2);
Meaning of arguments:
xfrom: the x-coordinate you are looking from.
yfrom: the y-coordinate you are looking from.
zfrom: the z-coordinate you are looking from.
xto: the x-coordinate you are looking to (usually the x-coordinate of the player)
yto: the y-coordinate you are looking to (usually the y-coordinate of the player)
zto: the z-coordinate you are looking to (usually the z-coordinate of the player)
fov: the FOV angle.
aspect: the aspect ratio of your window (window_get_width()/window_get_width())
display width: the width of your window (window_get_width())
display_height: the height of your window (window_get_height()
mouse x: In GMS1.99, I used mouse_x.
mouse y: In GMS1.99, I used mouse_y
display origin vec2: a 2-vector consisting of the origin coordinates of the area you're checking for mouse collisions. If it's the whole screen, use vect2(0,0)

Then, you would use plane_equation to find store the equation of the plane with which you are trying to make a collision. For your purposes, this would be the z=h plane, where h is the height of the surface of the table. You need a couple of 3-vectors as the arguments for this script; one is the 3-vector representing the normal vector of the plane (for you, a flat x-y plane would be 0,0,1 I believe), and then another 3-vector representing the xyz coordinates of a point you know to be in that plane (try not to use (0,0,0); if you have a z=0 plane like you mentioned above, then (1,1,0) should be fine).
Code:
plane = plane_equation(vec3_normals, vec3_point)
Meaning of arguments:
vec3_normals: a 3-vector representing the normal vector of the plane with which you are interacting.
vec3_point: a 3-vector representing the xyz coordinates of a point you know to be in that plane

Next, you must create the equation of the infinite line that is created by your mouse vector; this will be in a 6-vector that stores the x-slope, y-slope, and z-slope as well as the individual x-intercept, y-intercept, and z-intercept. Luckily, the input variables here are already known to you. Just use the [0], [1], and [2] entries of your mouse vector and then use your xfrom, yfrom, and zfrom from earlier. So, if my xfrom, yfrom, and zfrom are stored as those variable names, and the output of my mouse_vector function was stored to the variable mCoords, then the call to store the line equation would be:

Code:
line = vec6(mCoords[0], mCoords[1], mCoords[2], xfrom,yfrom,zfrom);
Now, we can finally find the actual point at which the mouse is intersects our plane in 3D space, by finding where the line created by the mouse intersects that 3D plane. The function is line_plane_intersect_at, and it will give you a 3-vector representing the xyz coordinates of the point at which that line intersects that plane. So, given that my plane is stored as "plane" above and my line stored as "line", I would do the following:
Code:
mouse_point = line_plane_intersect_at(line, plane);
Then you can access the x-coordinate, y-coordinate, and z-coordinate of that point individually as follows:
Code:
pointx = click_point[0];
pointy = click_point[1];
pointz = click_point[2];
I used this particular method (combined with p3dc.dll) to create the beginning of an isometric game, where you could click anywhere on an imported level model, it would determine (via p3dc, again) if you could move to that location, and move you there. You obviously wouldn't need p3dc.dll if you're only doing this with the z=0 plane, but here is a video of it working regardless. This is in, again, GMS1 and I used p3dc for the height checking of the little cone dude. I don't use this code anymore, or p3dc, mind you; I use that extension you looked at on your stream instead :p

And, finally... the scripts themselves. You'll need all the scripts I mentioned, plus a few data storage scripts. Plus I'll give you the code that I used as an example (excluding the p3dc parts).

mouse_vector()
Code:
///mouse_vector(xfrom, yfrom, zfrom, xto, yto, zto, xup, yup, zup, fov, aspect, display width, display_height, mouse x, mouse y, display origin vec2);
xfrom = argument0;
yfrom = argument1;
zfrom = argument2;
xto = argument3;
yto = argument4;
zto = argument5;
xup = argument6;
yup = argument7;
zup = argument8;
angle = argument9;
aspect = argument10;
wport = argument11;
hport = argument12;
pointerx = argument13;
pointery = argument14;
vsport = argument15;
xport = vsport[0];
yport = vsport[1];


dx = xto - xfrom;
dy = yto - yfrom;
dz = zto - zfrom;

mm = sqrt( (dx*dx) + (dy*dy) + (dz*dz) );

dx /= mm;
dy /= mm;
dz /= mm;

mm = xup * dx + yup * dy + zup * dz;

xup -= mm * dx;
yup -= mm * dy;
zup -= mm * dz;

mm = sqrt( (xup*xup) + (yup*yup) + (zup*zup) );

xup /= mm;
yup /= mm;
zup /= mm;

vx = yup * dz - dy * zup;
vy = zup * dx - dz * xup;
vz = xup * dy - dx * yup;

tfov = dtan(angle/2);
xup *= tfov;
yup *= tfov;
zup *= tfov;

vx *= tfov * aspect;
vy *= tfov * aspect;
vz *= tfov * aspect;

pointerx -= xport;
pointery -= yport;

mx = dx + xup * (1 - 2 * pointery / hport) + vx * (2 * pointerx / wport - 1);
my = dy + yup * (1 - 2 * pointery / hport) + vy * (2 * pointerx / wport - 1);
mz = dz + zup * (1 - 2 * pointery / hport) + vz * (2 * pointerx / wport - 1);

oVec = vec3(mx, my, mz);

return oVec;

plane_equation()
Code:
///plane_equation(vec3_normals, vec3_point)
//returns the coefficient of x, y, and z for a plane and then returns the intercepts as well.
n_x = argument0[0];
p_x = argument1[0];
n_y = argument0[1];
p_y = argument1[1];
n_z = argument0[2];
p_z = argument1[2];

planeConstant = n_x * p_x + n_y * p_y + n_z * p_z;

return vect4(n_x, n_y, n_z, planeConstant);

line_plane_intersect_at()
Code:
//set variables
lx  = argument0[0]; //x slope
ly  = argument0[1]; //y slope
lz  = argument0[2]; //z slope
lxi = argument0[3]; //x intercept (x-coordinate of any point that lies on the line)
lyi = argument0[4]; //y intercept (y-coordinate of any point that lies on the line)
lzi = argument0[5]; //z intercept (z-coordinate of any point that lies on the line)

px  = argument1[0]; //x slope
py  = argument1[1]; //y slope
pz  = argument1[2]; //z slope
planeConstant = argument1[3];

//put x, y, and z terms into plane equation such that:
//      xt + xi + yt + yi + zt + zi = planeConstant
xt = lx*px;
xi = lxi*px;
yt = ly*py;
yi = lyi*py;
zt = lz*pz;
zi = lzi*pz;

//consolidate all t terms and all constants
at = xt + yt + zt;
bi = xi + yi + zi;

//subtract constant value from planeConstant
planeConstant -= bi;

//divide by consolidated t values to find the t-value
t = planeConstant / at;

//plug new t-value into line equation
oX = lx*t + lxi;
oY = ly*t + lyi;
oZ = lz*t + lzi;

//put into vector and return
return vec3(oX, oY, oZ);

Data storage scripts:
vect2()
Code:
var arr;
arr = array_create(2);
arr[0] = argument0;
arr[1] = argument1;
return arr;

vec3()
Code:
///vec3(dx, dy, dz)
var arr = array_create(3);
arr[0] = argument0;
arr[1] = argument1;
arr[2] = argument2;
return arr;

vect4()
Code:
///vect4(dx, dy, dz, const)
var arr = array_create(4);
arr[0] = argument0;
arr[1] = argument1;
arr[2] = argument2;
arr[3] = argument3;
return arr;

vec6()
Code:
///vec6(dx, dy, dz, x_int, y_int, z_int)
var arr = array_create(6);
arr[0] = argument0;
arr[1] = argument1;
arr[2] = argument2;
arr[3] = argument3;
arr[4] = argument4;
arr[5] = argument5;
return arr;

Example code:
Code:
mCoords = mouse_vector(x-xf,y-yf,z-zf,x,y,z,0,0,1,fov,room_width/room_height,ww, wh, mouse_x, mouse_y,vect2(0, 0));
plane = plane_equation(vec3(nx, ny, nz), vec3(px, py, pz)); //nx, ny, nz are the components of the normal vector that i got from p3dc. px, py, pz are coordinates of a point also gotten from p3dc
line = vec6(mCoords[0], mCoords[1], mCoords[2], x-xf,y-yf,z-zf);
click_point = line_plane_intersect_at(line, plane);
pointx = click_point[0]; //x-coordinate of mouse point in world on the plane
pointy = click_point[1]; //y-coordinate of mouse point in world on the plane
pointz = click_point[2]; //z-coordinate of mouse point in world on the plane

This post is T H I C C. Again, you'll have to figure out the equivalents of the arguments for the GMS1 d3d_set_projection arguments in GMS2, but other than that, everything should transfer over fine. Then, it's up to you to just check the x and y coordinates and compare that to the coordinates of whatever you're trying to interact with whenever you're going to interact with it. ALSO: There is a chance it will overflow if you're pointing exactly parallel to the interactive plane... in which case, you'll have to make a wrapper that considers the mouse coordinates themselves and the angle of the camera to counteract that.

EDIT: For anyone exploring this in the future, the GMS2 equivalents I was talking about before for the mouse_vector script would be the ones of the same respective names that you would put into the matrix_build_lookat() function. If you're doing this in GMS1, then the variables are the same as the same-named ones from the d3d_set_projection_ext function.
 
Last edited:

MilesThatch

Member
This post is T H I C C. Again, you'll have to figure out the equivalents of the arguments for the GMS1 d3d_set_projection arguments in GMS2, but other than that, everything should transfer over fine. Then, it's up to you to just check the x and y coordinates and compare that to the coordinates of whatever you're trying to interact with whenever you're going to interact with it. ALSO: There is a chance it will overflow if you're pointing exactly parallel to the interactive plane... in which case, you'll have to make a wrapper that considers the mouse coordinates themselves and the angle of the camera to counteract that.
Thanks for finding this out. Now... how much would it cost me to ask you to drive this script home and finally make it compatible with GMS2?
 

wadaltmon

Member
As much as I appreciate the offer, I don't own a license to GMS2, nor have I even used it before. Although I will direct you to @orange451 who helped me out in my original thread regarding this topic. Really all that needs changing for GMS2 is that you will need to find out what the arguments are for that first script.
 

MilesThatch

Member
As much as I appreciate the offer, I don't own a license to GMS2, nor have I even used it before. Although I will direct you to @orange451 who helped me out in my original thread regarding this topic. Really all that needs changing for GMS2 is that you will need to find out what the arguments are for that first script.
Projection matrices are way outside of my area of expertise. That's some math I want to go anywhere near. So I guess the offer then extends to @orange451 with this. It does seem kinda weird that the script is not taking Camera x y z angle into account because this camera can swivel around in all directions
 

wadaltmon

Member
Projection matrices are way outside of my area of expertise. That's some math I want to go anywhere near. So I guess the offer then extends to @orange451 with this. It does seem kinda weird that the script is not taking Camera x y z angle into account because this camera can swivel around in all directions
In GMS 1, the camera angle is determined by the xfrom/yfrom/zfrom in relation to xto/yto/zto in the d3d_set_projection. So potentially you could figure out your xfrom/yfrom/zfrom by using the camera rotation vector combined with your current distance from your xto/yto/zto (which is almost always your player object's coordinates).
 

MilesThatch

Member
In GMS 1, the camera angle is determined by the xfrom/yfrom/zfrom in relation to xto/yto/zto in the d3d_set_projection. So potentially you could figure out your xfrom/yfrom/zfrom by using the camera rotation vector combined with your current distance from your xto/yto/zto (which is almost always your player object's coordinates).
I'm not really seeing which part in the scripts I need to substitute with the GMS2 alternative here.

There's a projection function GMS2:
matrix_build_lookat(xfrom, yfrom, zfrom, xto, yto, zto, xup, yup, zup);

You usually have it in the camera update script
 

wadaltmon

Member
Oh. Well there you go then. Again, I have no clue how the projection is built in GMS2. I guess just use those arguments in the mouse_vector script. I didn't know that function existed. Just put whatever you would normally put in there into the mouse_vector script (along with the rest of your regular camera update code) and you should be okay.

Though I'd recommend having a little extra in there to test. Maybe in your draw event you have it draw a distinctive model at the found x/y coordinates to test that it's working.

Also @MishMash is another member of MantaGames who is quite knowledgable on 3D subjects
 
Last edited:
As long as you have a view matrix, you might as well use it to avoid having to do redundant math.

Code:
var _x = aspect_of_port * dtan( fov /2 ) * (2 * mouse_x_position_within_port / width_of_port - 1)
var _y =                  dtan( fov /2 ) * (1 - 2 * mouse_y_position_within_port / height_of_port)
mouse_vector_x = _x * view_matrix[0] + _y * view_matrix[1] + view_matrix[2]
mouse_vector_y = _x * view_matrix[4] + _y * view_matrix[5] + view_matrix[6]
mouse_vector_z = _x * view_matrix[8] + _y * view_matrix[9] + view_matrix[10]

if mouse_vector_z != 0) {  //mouse vector isn't parallel to floor
   var _t = -camera_from_z / mouse_vector_z
   if (_t >= 0) {  //mouse vector is not pointing away from floor
       mouse_floor_intersection_x = cam_from_x + mouse_vector_x * _t
       mouse_floor_intersection_y = cam_from_y + mouse_vector_y * _t
   }
}
I think that is pretty self explanetory. And it should work as is for a floor at z=0. Now, the one thing I'm worried about is that GMS2 does some kind of weird thing with flipping the up vector, in which case something might need to be inverted. Like the other dude, I've only got GMS1.4. mouse_vector is not normalized here, but doesn't need to be for this case.
 
Last edited:

MilesThatch

Member
Hey @flyingsaucerinvasion Thanks for the alternative suggestion.

What value goes into aspect_of_port? Aspect ratio?
and view_matrix[0,1,2 4,5,6 and 8,9,10]? These don't seem to be build in variables and GM complains that this it's looking for an array...
 
Hey @flyingsaucerinvasion Thanks for the alternative suggestion.

What value goes into aspect_of_port? Aspect ratio?
and view_matrix[0,1,2 4,5,6 and 8,9,10]? These don't seem to be build in variables and GM complains that this it's looking for an array...
aspect_of_port is the width of the port divided by the height of the port. And by port, I really just mean whatever (surface) you are currently drawing onto.

view_matrix would be a variable containing your view matrix (layed out as a 1d array). The indexes are entries in the matrix. If you are using GMS2, I presume you are using matrix_build_lookat, which returns a view matrix, and you can just use that to avoid having to replicate the math that went into building it in the first place.
 

MilesThatch

Member
aspect_of_port is the width of the port divided by the height of the port. And by port, I really just mean whatever (surface) you are currently drawing onto.

view_matrix would be a variable containing your view matrix (layed out as a 1d array). The indexes are entries in the matrix. If you are using GMS2, I presume you are using matrix_build_lookat, which returns a view matrix, and you can just use that to avoid having to replicate the math that went into building it in the first place.
Is if the the view_matrix is essentially an array containing all the information I've put in using matrix_build_lookat then what are we trying to access via slots 9 and 10? the function matrix_build_lookat(xfrom, yfrom, zfrom, xto, yto, zto, xup, yup, zup); only has 9 pieces of data. So that's array slots 0 to 8
 
Last edited:

FrostyCat

Redemption Seeker
Is if the the view_matrix is essentially an array-like variable which contains all the information I've put in using matrix_build_lookat then what are we trying to access via slots 9 and 10? the function matrix_build_lookat(xfrom, yfrom, zfrom, xto, yto, zto, xup, yup, zup); only has 9 pieces of data. So that's array slots 0 to 8
That's because the result of matrix_build_lookat() is a 4x4 matrix characterized as a 16-entry array.

projectionmatrix.png

There's a reason why I discourage all 3D work for anyone who has no prior background in linear algebra. You've seen for yourself what the price of not speaking the language of linear algebra is, nobody with the skill to help you with 3D can communicate properly with you. This is a bullet that you have to bite. Hell, most of it is basic arithmetic in batches, maybe a little trigonometry for rotations. It's easier to just learn the basics than to remain ignorant.
 

MilesThatch

Member
The thing is that the error that pops up when trying to compile this is :

Code:
___________________________________________
############################################################################################
FATAL ERROR in
action number 1
of  Step Event0
for object obj_player:

trying to index a variable which is not an array
 at gml_Object_obj_player_Step_0 (line 73) - mouse_vector_x = _x * global.mLookat[0] + _y * global.mLookat[1] + global.mLookat[2];
############################################################################################
--------------------------------------------------------------------------------------------
stack frame is
gml_Object_obj_player_Step_0 (line 73)
Which seems to have nothing to do with the type of math we're working with but the fact that the array variable is not perceived as an array variable.

The Camera update script:

Code:
//Build a matrix that looks from the camera location above, to the room center. The up vector points to +z
global.mLookat = matrix_build_lookat(global.cam_xx,global.cam_yy,global.cam_zz, obj_player.x,obj_player.y,0, 0,0,1);

//Assign the matrix to the camera. This updates were the camera is looking from, without having to unnecessarily update the projection.
camera_set_view_mat(view_camera[0], global.mLookat);
The reason behind it, I do not know. Maybe because it's a global variable that is set up in the camera update script, maybe something else.
 

Kyon

Member
It says "trying to index a variable which is not an array"
And when I look closely you are using global.mLookat[0]/[1]/[2]. And in your other code you're just using global.mLookat. Without an array?
 

MilesThatch

Member
It says "trying to index a variable which is not an array"
And when I look closely you are using global.mLookat[0]/[1]/[2]. And in your other code you're just using global.mLookat. Without an array?
That's what I was asking about.

view_matrix would be a variable containing your view matrix (layed out as a 1d array). The indexes are entries in the matrix. If you are using GMS2, I presume you are using matrix_build_lookat, which returns a view matrix, and you can just use that to avoid having to replicate the math that went into building it in the first place.
@flyingsaucerinvasion said that the matrix_build_lookat will return a view_matrix and that's what I was supposed to insert. Just copy-paste the needed variables. I'm just following the code given by him. It's trying to access the variables of the view_matrix by using the array accessors.

Unless that's not what I was supposed to... which is why I'm asking
 
Hi. I don't have GMS2, so I might be making a bad assumption. I presume matrix_build_lookat returns an array index. Is it possible you are trying to access the variable before it has been set? For example because you are trying to read the array in the step event, but aren't setting it until the draw event?

By the way, if GMS2 builds a left-handed view matrix just the same way that GMS1 does, then it should be computed and layed out like this:
Code:
zaxis = normalize(cam_to - cam_from)
xaxis = normalize(cross(cam_up, zaxis))
yaxis = cross(zaxis, xaxis)
┌                                            ┐
│ xaxis_x  xaxis_y  xaxis_z  -dot(xaxis, eye)│
│ yaxis_x  yaxis_y  yaxis_z  -dot(yaxis, eye)│
│ zaxis_x  zaxis_y  zaxis_z  -dot(zaxis, eye)│
│ 0        0        0         1              │
└                                            ┘

array indexes mapped to matrix like this:
┌             ┐
│ 0  4  8  12 │
│ 1  5  9  13 │
│ 2  6  10 14 │
│ 3  7  11 15 │
└             ┘
 
Last edited:

FrostyCat

Redemption Seeker
The thing is that the error that pops up when trying to compile this is :

Code:
___________________________________________
############################################################################################
FATAL ERROR in
action number 1
of  Step Event0
for object obj_player:

trying to index a variable which is not an array
 at gml_Object_obj_player_Step_0 (line 73) - mouse_vector_x = _x * global.mLookat[0] + _y * global.mLookat[1] + global.mLookat[2];
############################################################################################
--------------------------------------------------------------------------------------------
stack frame is
gml_Object_obj_player_Step_0 (line 73)
Which seems to have nothing to do with the type of math we're working with but the fact that the array variable is not perceived as an array variable.

The Camera update script:

Code:
//Build a matrix that looks from the camera location above, to the room center. The up vector points to +z
global.mLookat = matrix_build_lookat(global.cam_xx,global.cam_yy,global.cam_zz, obj_player.x,obj_player.y,0, 0,0,1);

//Assign the matrix to the camera. This updates were the camera is looking from, without having to unnecessarily update the projection.
camera_set_view_mat(view_camera[0], global.mLookat);
The reason behind it, I do not know. Maybe because it's a global variable that is set up in the camera update script, maybe something else.
Try replacing that line with the equivalent using matrix_transform_vertex().
Code:
var mouse_vector = matrix_transform_vertex(matrix, _x, _y, 1);
mouse_vector_x = mouse_vector[0];
mouse_vector_y = mouse_vector[1];
mouse_vector_z = mouse_vector[2];
 

MilesThatch

Member
Hi. I don't have GMS2, so I might be making a bad assumption. I presume matrix_build_lookat returns an array index. Is it possible you are trying to access the variable before it has been set? For example because you are trying to read the array in the step event, but aren't setting it until the draw event?

By the way, if GMS2 builds a left-handed view matrix just the same way that GMS1 does, then it should be layed out like this:
I've tried to set up a 60 step delay before accessing the variable to test it. Same error occurs.
But it IS created in the camera update script set up by camera_set_update_script() so maybe that could have something to do with it...
 
Last edited:
@MilesThatch Hi, we're starting to go off topic, but I'm curious to know what's going on anyway.

Are you able to do the following without errors?

blah = matrix_build_lookat(0,0,0,100,0,0,0,0,1);
show_message(blah[5]);
 

MilesThatch

Member
Using that in the camera update script gives this:



Just for a test, I've also tried it in a draw even of my master object. Same result.
 

MilesThatch

Member
So no errors. It looks like your variable somehow just isn't set before you are trying to access it. Or maybe you are overwriting it with something else?
This IS a matrix variable so overwriting it would probably yield visible issues?

Also here's the result of every single Search of the my used matrix:
Code:
scr_cam_update at line 65:     global.mLookat = matrix_build_lookat(global.cam_xx,global.cam_yy,global.cam_zz, obj_player.x,obj_player.y,0, 0,0,1);
scr_cam_update at line 68:     camera_set_view_mat(view_camera[0], global.mLookat);
obj_player-Step at line 79:     mouse_vector_x = _x * global.mLookat[0] + _y * global.mLookat[1] + global.mLookat[2];
obj_player-Step at line 79:     mouse_vector_x = _x * global.mLookat[0] + _y * global.mLookat[1] + global.mLookat[2];
obj_player-Step at line 79:     mouse_vector_x = _x * global.mLookat[0] + _y * global.mLookat[1] + global.mLookat[2];
obj_player-Step at line 80:     mouse_vector_y = _x * global.mLookat[4] + _y * global.mLookat[5] + global.mLookat[6];
obj_player-Step at line 80:     mouse_vector_y = _x * global.mLookat[4] + _y * global.mLookat[5] + global.mLookat[6];
obj_player-Step at line 80:     mouse_vector_y = _x * global.mLookat[4] + _y * global.mLookat[5] + global.mLookat[6];
obj_player-Step at line 81:     mouse_vector_z = _x * global.mLookat[8] + _y * global.mLookat[9] + global.mLookat[10];
obj_player-Step at line 81:     mouse_vector_z = _x * global.mLookat[8] + _y * global.mLookat[9] + global.mLookat[10];
obj_player-Step at line 81:     mouse_vector_z = _x * global.mLookat[8] + _y * global.mLookat[9] + global.mLookat[10];
Search Complete (11 results)
 
I don't see anywhere where it is being overwritten. So it must not being set before being used I guess. There's no reason that the view matrix can't be computed in the step event, just before using it for example in the script that handles the mouse vector. If you put it there, you can be sure it will contain the current view matrix when you try to access it. \

If all else fails you can go back to computing the x, y and z axes for your camera like so:

zaxis = normalize(cam_to - cam_from)
xaxis = normalize(cross(cam_up, zaxis))
yaxis = cross(zaxis, xaxis)
 

MilesThatch

Member
Oh wait a minute... moved the camera update script (still the old one) to be executed within the step event of my master object and BAM... x axys are now working... but y is not. so I outta figure that out now.

 
Last edited:
Huh, that's kind of strange, isn't it?

Let's see exactly the code that you wrote. It looks like for some reason you've got a zero value for the _y component of the mouse vector in view space.
 

MilesThatch

Member
Actually I forgot to enter the FOV for the
var _y = dtan( FOV/2 ) * (1 - 2 * display_mouse_get_y() / window_get_height());

line so now it works BUT it's in reverse.
 
It'd be a lot easier to see what was happening if the cursor were visible at the same time as the crosshair.

I think what is happening is caused by the discrepancy between measuring the mouse position relative to the display, but dividing by the size of the window. You should measure the mouse position relative to the window, and not the display.
 
O

orange451

Guest
Since I was @'d...
You can't really get the 3d position of the mouse from a 2d position on the screen, because no one 3d position exists.
What you can easily get, though, is a ray pointing from your camera to the direction your mouse could be.

Even though you already have your answer, I thought I'd try to walk you through the math involved to get it generically through matrices.

If you've ever written any 3d shaders, you'll know about the MVP matrix i.e:
Code:
position = projection * view * model * in_vertex
You're essentially doing the inverse operation. In the code above, that gives you a projected vertex on your screen in 2d. So to undo it, you would do:
Code:
in_vertex = position * iProjection * iView
  • iProjection is the inversed projection matrix
  • iView is the inversed view matrix
  • You don't need iModel matrix because you can assume it's an identity matrix, and therefore redundant.
What's left is a 3d vector position. If you subtract your cameras position then normalize, you then will have a vector that represents a DIRECTION pointing from your camera to the input position.

Some more info:
  • The hardest thing about this will be to write a script that takes a vertex and multiplied it by a matrix.
  • You must use NDC coordinates for all inputs. [-1,1]
  • This is considered the slower approach. It's not optimized like the answer you were given above. However it's much easier to debug, and verify everything is working.
  • You can take the resultant direction vector, and do some ray-casting with it to find where it will collide with your scene.
Here's some code I wrote in java to multiply a vertex by a matrix:
Code:
public Vector3f mulProject(Matrix4f mat, Vector3f dest) {
    float invW = 1.0f / (mat.m03 * x + mat.m13 * y + mat.m23 * z + mat.m33);
    dest.set((mat.m00 * x + mat.m10 * y + mat.m20 * z + mat.m30) * invW,
             (mat.m01 * x + mat.m11 * y + mat.m21 * z + mat.m31) * invW,
             (mat.m02 * x + mat.m12 * y + mat.m22 * z + mat.m32) * invW);
    return dest;
}

Here's some psuedo code for the whole thing:
Code:
projMat = matrix_get_projection();
viewMat = matrix_get_view();
iProjMat = // inverse the projection matrix
iViewMat = // inverse the view matrix

// Define input in NDC space [-1,1]
mCoords = Vector3( xInput, yInput, 1.0f );

// Put Mouse-coords from NDC space into view space
mCoords = mulProject(iProjMat, mCoords);

// Put Mouse-coords from view space into world space
mCoords = mulProject(iViewMat, mCoords);

// Subtract cameras position ( World-space into Object space )
camPos = camera_get().position;
finalCoords = Vector3( mCoords.x - camPos.x, mCoords.y - camPos.y, mCoords.z - camPos.z );

// Normalize
finalCoords = normalize(finalCoords);
 
Last edited:
Top