Hello! Background: I am creating a 3D, isometric RPG. In order to make collisions more expedient, I am using the P3DC extension. As in most isometric RPGs, the mouse is used to control where the player character will move and what the player will interact with. As such, I implemented a ray casting system that will move an object based on the movement of the mouse; the mouse will move in some direction, the deviation from the center of the screen is found, and the object is moved proportionally to this deviation, and the mouse returns to the center of the screen. The ray cast will then point to the location of this object (referred to henceforth as the player object). This is so I can have traditional RPG movement and interaction (you click on a location to move, or click on an element in the world to interact with it). Take a look at this video to see the kind of movement I mean (a good example is at 2:58): Spoiler I went this route because my attempts to simply transform the ray that was being cast failed. I decided that I would create a new object to act as the mouse cursor in the game (since the mouse returns to the center of the screen), so that it would appear directly over where the player object was shown on screen. However, I'm unable to find a way to make it so the mouse movement directly controls how far the object is moved on-screen. What I mean by this is that the farther away the player object is from the camera view, the farther it would have to move (in terms of x-y coordinates) to correspond to the cursor on screen, whereas the closer it is to the camera view, the less it would have to move. However, I'm unable to find a suitable mathematical formula to do this. As such, when the player object is farther away from the camera view, it seems like the mouse is less sensitive almost. Take a look at this video to see the current system (the green triangle is the triangle with which the raycast collides): Spoiler Now for the code I've used. There are 2 objects: one for the controller (player object) itself, and one that simply controls the position of the camera view. Controller (player object) create event (unrelated code removed): Spoiler Code: z=50; x=30; y=30; sw=display_get_width()/2; sh=display_get_height()/2; direction = 0; xf=210.25; yf=-164.26; zf=-422.86; dtc = 500; Controller (player object) step event: Spoiler Code: mult = 2; //MOUSELOOK x-=lengthdir_x( (mult/2 * (2*sh)/dtc) * ((sh-display_mouse_get_y())/2),145); y-=lengthdir_y( (mult/2 * (2*sh)/dtc) * ((sh-display_mouse_get_y())/2),145); x-=lengthdir_x( (mult/2 * (2*sh)/dtc) * ((sw-display_mouse_get_x())/2),235); y-=lengthdir_y( (mult/2 * (2*sh)/dtc) * ((sw-display_mouse_get_x())/2),235); display_mouse_set(sw,sh); Controller (player object) draw event (unrelated code removed): Spoiler Code: d3d_set_projection_ext(oCam.x-210.25,oCam.y-164.26,z+422.86, oCam.x,oCam.y,z,0,0,1,45,view_wview[0]/view_hview[0],1,262144); xd=xf/dtc; yd=-yf/dtc; zd=zf/dtc; dis=p3dc_ray_still(level_colid,x,y,z+5,xd,yd,zd); Camera (oCam) object create event: Spoiler Code: dtc = 500; sw=display_get_width()/2; sh=display_get_height()/2; Camera (oCam) object step event: Spoiler Code: if (keyboard_check(ord('W'))) { x-=lengthdir_x(2,145); y-=lengthdir_y(2,145); } if (keyboard_check(ord('S'))) { x-=lengthdir_x(2,145+180); y-=lengthdir_y(2,145+180); } if (keyboard_check(ord('A'))) { x-=lengthdir_x(2,145+90); y-=lengthdir_y(2,145+90); } if (keyboard_check(ord('D'))) { x-=lengthdir_x(2,145-90); y-=lengthdir_y(2,145-90); } I feel I should also explain a few things about these. 1. z never changes, it is always a constant 50. 2. dtc will eventually be able to change so the player can zoom in, but as of right now, it doesn't change. 3. The numbers in the projection as well as the numbers 145 and such just come from me testing the view angle I wanted to keep constant, and what directions on the x-y plane would correspond to "up" on the player's display. 4. with dtc = 500, the distance from the top to the bottom of the screen in real x-y distance is 565.85 as I have measured. I tried using the "mult" variable to change the sensitivity of movement based on the distance away from the camera object, to no avail (it simply made the mouse really sensitive when it went above the middle of the screen). Here is the code I tried for that: Spoiler Code: if ( (oCam.x >= x) && (oCam.y >= y)) { xback = oCam.x-lengthdir_x(565.85/2, 325); yback = oCam.y-lengthdir_y(565.85/2, 325); disback = abs(point_distance(xback, yback, x, y)); mult = 2 * (disback / 565.85); } else { mult = 4; } However, this code did not yield anything but the cursor being way too sensitive while it was far away from the camera, and at a very low sensitivity when it was close to the camera. I'm sure there is a way to create a raycast such that the place on the level model that the ray connects with will be directly under the mouse cursor; or, there may be a way to scale the movement of my existing object movement system to allow the object to move farther as it gets farther away (and to allow for the mouse cursor to be a separate object that moves in conjunction with this). However, the way to do this... I'm just not seeing. I'd like to see if anyone that has experience in 3D or is seeing something I'm not can help me out with this. Input is greatly appreciated. Thank you in advance. wadaltmon
I found this resource here that somewhat explains the process: http://antongerdelan.net/opengl/raycasting.html So essentially the process is: Real mouse x = ((2 * mouse x coordinate) / screen width) -1; Real mouse y = ((2 * mouse y coordinate) / screen height) -1; Real mouse z = 1; Create new 3D vector with these x,y,z coordinates Create new 4D vector (quaternion?) with the above x and y coordinates, but with -1.0 for z and 1.0 for w. Take the inverse of the current projection matrix and multiply it by the matrix created from the quaternion and store as a 4D vector. Create new 4D vector with the x and y coordinates of the above 4D vector and have -1.0 as the z value and 0.0 as the w value. Take the inverse of the current view matrix and multiply times the matrix created from the above quaternion, and store the x,y,z values in a new 3D vector. Normalize that vector. Use that vector's x,y,z values as the values in the raycast alongside the corresponding "ray points to..." values. I've never used quaternions before, nor have I used matrices in GML (I think that you can only really do a 4x4 matrix in order to use the internal operations of GML?). I know how to do matrix multiplication and inversion. Is this even the right method to do what I'm looking to do? Is anyone able to help me to figure out how to make this into GML code? How do I construct a matrix to represent the current view matrix or current projection matrix? Thank you in advance. wadaltmon
It's the perspective shown in the video above. It's not birds eye view completely, but it's at an angle using the perspective. Orthographic projection is only used to put text and maybe sprites on screen, not for the in-game world. If I get what you're asking. Here's the link to the angle the game is at:
It's quite simple You know your projection matrix, your view matrix, and a 2d point to reverse project to. To get from a 3d vertex position to a 2d coordinate on the screen, internally what is being done is: (projectionMatrix * viewMatrix * vertexPosition) So to reverse this, you take your 2d coordinate, put it into NDC space (-1 to 1), multiply by the inverse projection, then multiply by the inverse view, and you're left with a 3d vector that represents the correct direction that you're looking to raycast against with p3dc.
orange451 - Thank you so much for responding! I got the view matrix and projection matrix by using the matrix_get functions. Is this the correct way to do it? I get the feeling I know what the view and projection matrices are already, but don't know them by that name... Either way, it seems that the view and projection matrices don't change as time goes on. So, in the create event of my controller, I have: Code: ndmx = 0; ndmy = 0; ndmz = -1; pmtx = matrix_get(matrix_projection); vmtx = matrix_get(matrix_view); ivmtx = matrix_inverse_4(vmtx); ipmtx = matrix_inverse_4(pmtx); mmtx = matrix_multiply(ipmtx, ivmtx); (matrix inverse 4 is a script I created to get the inverse of a 4x4 matrix) And in the step event, I have: Code: ndmx = (display_mouse_get_x() - sw) / sw; ndmy = (display_mouse_get_y() - sh) / sh; But as I understand it, the inverse of the view matrix multiplied by the inverse of the projection matrix (both obtained with the matrix_get functions) becomes: 0 0 0 0 0 0 0 0 0 0 0 0 960 520.5 -16000 1 I'm not sure how I would multiply the matrix by the NDC coordinates of the mouse. Any thoughts on this? Did I go about the procedure correctly? Also, the p3dc command requires 2 points to project a ray from and to. However, I'm sure that if I were to convert the mouse position to world coordinates, I'd be able to simply enter that as the "from" point.
Okay, so here is the code that I tried: Code: pmtx = matrix_get(matrix_projection); vmtx = matrix_get(matrix_view); ivmtx = matrix_inverse_4(vmtx); ipmtx = matrix_inverse_4(pmtx); ndmx = (display_mouse_get_x() - sw) / sw; ndmy = (display_mouse_get_y() - sh) / sh; ipmtx[12] *= ndmx; ipmtx[13] *= ndmy; mmtx = matrix_multiply(ipmtx, ivmtx); This is in the step event. In the draw, I have: Code: d3d_set_projection_ext(oCam.x-210.25,oCam.y-164.26,z+422.86, oCam.x,oCam.y,z,0,0,1,45,view_wview[0]/view_hview[0],1,262144); xd=xf/dtc; yd=-yf/dtc; zd=zf/dtc; dis=p3dc_ray_still(level_colid,oCam.x + mmtx[12],oCam.y + mmtx[13],mmtx[14],xd,yd,zd); However, this doesn't produce the desired result. The highlighted triangle is always at the maximum x,y on the level model. What did I do wrong here?
It should be more like: Code: // Get mouse coordinate in screen space [0.0 - 1.0] s_mx = display_mouse_get_x() / sw; s_my = display_mouse_get_y() / sh; // Convert into NDC space [-1.0 - +1.0] ndc_mx = s_mx * 2.0 - 1.0; ndc_my = s_my * 2.0 - 1.0; // Convert from NDC space into View Space mCoords = vector_3( ndc_mx, ndc_my, 1.0 ); // Create an array or something to store this data (x,y,z). mCoords = mul( mCoords, iProjectionMatrix ); // Multiply the coordinates with the inverse projection matrix // Convert from View Space into World Space. mCoords = mul( mCoords, iViewMatrix ); // Calculate final vector finalDirection = vector_3( mCoords[0] - camerax, mCoords[1] - cameray, mCoords[2] - cameraz ); finalDirection = normalize( finalDirection ); // Normalize the result for proper ray casting. The multiplication method I use in my java project looks like this: Code: public static Vector3f mulProject(Vector3f left, Matrix4f right, Vector3f dest) { float invW = 1.0f / (right.m03 * left.x + right.m13 * left.y + right.m23 * left.z + right.m33); dest.set((right.m00 * left.x + right.m10 * left.y + right.m20 * left.z + right.m30) * invW, (right.m01 * left.x + right.m11 * left.y + right.m21 * left.z + right.m31) * invW, (right.m02 * left.x + right.m12 * left.y + right.m22 * left.z + right.m32) * invW); return dest; } Not exactly copy-and-paste, but it should get you in the right direction once you write all the proper background code!
Are you just trying to get the 3d mouse vector? This is the script I use and I know that it works because I just tested it. Code: ///scr_mouse_vector(cam from, cam to, cam up, fov, aspect, surface_width, surface_height, surface_mouse_x, surface_mouse_y ); //------------------------------------------------------------------------- var _l = scr_normalize_vec3(scr_sub_vec3(argument1, argument0)); var _m = scr_dot_vec3(argument2, _l); var _tfov = dtan(argument3 / 2); var _u = scr_mult_vec3_real(scr_normalize_vec3(scr_sub_vec3(argument2, scr_mult_vec3_real(_l, _m))), _tfov); var _v = scr_mult_vec3_real(scr_cross_vec3(_u, _l), argument4); var _screen_x = scr_mult_vec3_real(_v, 2 * argument7 / argument5 - 1); var _screen_y = scr_mult_vec3_real(_u, 1 - 2 * argument8 / argument6); mouse_vector = scr_normalize_vec3(scr_add_3_vec3(_l, _screen_x, _screen_y)); //------------------------------------------------------------------------- //user-defined functions in above script: //vec3 scr_sub_vec3( vec3 minuend, vec3 subtrahend ); //vec3 scr_mult_vec3_real( vec3, real ); //vec3 scr_add_vec3( vec3, vec3 ); //vec3 scr_add_3_vec3( vec3, vec3, vec3 ); //vec3 scr_cross_vec3( vec3, vec3 ); //vec3 scr_normalize_vec3( vec3 ); //real scr_dot_vec3( vec3, vec3); The function names should explain what they do.
Here's the code that I implemented: Code: pmtx = matrix_get(matrix_projection); vmtx = matrix_get(matrix_view); ivmtx = matrix_inverse_4(vmtx); ipmtx = matrix_inverse_4(pmtx); s_mx = display_mouse_get_x() / (sw*2); s_my = display_mouse_get_y() / (sh*2); ndmx = s_mx * 2.0 - 1.0; ndmy = s_my * 2.0 - 1.0; mCoords = vec3(ndmx, ndmy, 1.0); //notice this line. mCoords = scMulMat(mCoords, ipmtx); mCoords = scMulMat(mCoords, ivmtx); finalDirection = vec3(mCoords[0] + oCam.x, mCoords[1] + oCam.y, mCoords[2] + 5); finalDirection = vec3_normalize(finalDirection); As for the scripts used in here, we'll assume that the inverse of the 4x4 matrix script is correct (I've checked it thoroughly, and overall, it's just very long and I wouldn't want to pollute the post with it). However, the other scripts I used are: vec3: Spoiler Code: ///vec3(dx, dy, dz) var arr = array_create(3); arr[0] = argument0; arr[1] = argument1; arr[2] = argument2; return arr; scMulMat: Spoiler Code: ///scMulMat(vec3, matrix_4x4) var arrd = array_create(3); left = argument0; right = argument1; invW = 1 / (right[3] * left[0] + right[7] * left[1] + right[11] * left[2] + right[15]); arrd[0] = ((right[0] * left[0] + right[4] * left[1] + right[8] * left[2] + right[12]) * invW); arrd[1] = ((right[1] * left[0] + right[5] * left[1] + right[9] * left[2] + right[13]) * invW); arrd[2] = ((right[2] * left[0] + right[6] * left[1] + right[10] * left[2] + right[14]) * invW); return arrd; vec3_normalize: Spoiler Code: ///vec3_normalize(vec3) var arrn = array_create(3); ax = argument0[0]; ay = argument0[1]; az = argument0[2]; len = sqrt((ax*ax) + (ay*ay) + (az*az)); ax /= len; ay /= len; az /= len; arrn[0] = ax; arrn[1] = ay; arrn[2] = az; return arrn; However, in the top piece of code, the line that states "notice this line" is the last line of code that actually functions and updates every step. Otherwise, when I multiply the first 2 matrices, the resultant coordinates don't change at all based solely on the mouse position. Even the resultant vector stays the exact same every step, it always stays the same until I move the camera object itself. Even the world coordinates of the mouse cursor don't seem to change. Is there something I'm overlooking here?
There's a few things you can do to verify your matrix functions, if you're worried. For example, to test if your inverse function works, you can use the fact that: Matrix A * Matrix A-1 should result in an Identity Matrix. IE: 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 I am confused, why are you multiplying your screen width and height by 2? Is "sw" and "sh" half of the screen size? Your matrix * vertex script looks fine to me...
Yeah sw and sh are half the screen height and screen width, respectively. I was originally using those variables for mouselook. I found a few mistakes in my 4x4 matrix inverse script, and fixed and tested it. It works as intended now, if I multiply a matrix by its inverse I'll get the identity matrix. My assumption is that the current vector that is being calculated is correct (it matches up with the direction of the raycast of the camera itself when the mouse pointer is at the middle of the screen). The final step is to create the raycast requires an origin point (x, y, z) and a direction (xdirection, ydirection, zdirection). Originally, my vector had an origin point that was the same as the camera's (xfrom, yfrom, zfrom) point, and then I had a custom vector that showed the direction that the camera is constantly pointing. This would make it so that the ray is always being cast from the center of the screen. After reading more of how the P3DC scripts work and understanding more about the vectors used within the existing code, a new problem arises. With the code currently in the project, the point where the ray cast hits on the world model doesn't fall directly under the mouse cursor, which is what I'm trying to accomplish. I think this is because it's essentially projecting a ray in a way similar to if I turned the camera and kept the raycast in the center of the screen (i.e. as I move the mouse, the ray will rotate around the origin point rather than keep the same direction and simply translate the origin point. However, it appears this may already be done in the code both of you (@orange451 and @flyingsaucerinvasion) provided. Orange451, is the mCoords vec3 the mouse coordinates in the world? And flyingsaucerinvasion, the _screen_x and _screen_y variables seem to be 3D vectors in and of themselves; do they contain the mouse's position in the 3D world in some way? EDIT: Orange451 - after taking a look at the code you provided again, I can't seem to get it to work at all once again. I thought I had it working before (aside from the caveat that instigated this post), but perhaps it was a simple coincidence. The vector seems to always have a z-component of 1.00, whereas the other two components do not go above 0.06. Flyingsaucerinvasion's code seems to produce a vector that matches up with the original vector I had, for when the ray is being cast from the center of the screen only. I thought the mistake might have been with the conversion of the mouse y-coordinate to NDC space (instead of mouse_y * 2 / height -1, wouldn't it be -1 * mouse_y * 2 / height + 1 ?), but even so, it still doesn't produce the correct vector. If the mCoords are meant to be the world space of the mouse, then I fear that these would be incorrect as well. At a glance, do all the steps in the following code seem correct? I believe I've followed the process correctly, but perhaps I'm wrong here. Spoiler Code: pmtx = matrix_get(matrix_projection); vmtx = matrix_get(matrix_view); ivmtx = matrix_inverse(vmtx); ipmtx = matrix_inverse(pmtx); s_mx = display_mouse_get_x() / (sw*2); s_my = display_mouse_get_y() / (sh*2); ndmx = s_mx * 2.0 - 1.0; ndmy = -1 * s_my * 2.0 + 1.0; mCoords = vec3(ndmx, ndmy, 1.0); mCoords = matrix_multiply_vec3(ipmtx, mCoords); mCoords = matrix_multiply_vec3(ivmtx, mCoords); finalDirection = vec3(mCoords[0] - oCam.x, mCoords[1] - oCam.y, mCoords[2] - z); finalDirection = vec3_normalize(mCoords);
My script doesn't tell you where the mouse is, it tells you what direction it is pointing. mouse_vector is the direction the mouse is pointing.
Try just printing out the mcoords values without subtracting the camera position and normalizing. Also, I am not sure if the multiplication method I gave you will work in Game Maker straight away. Try transposing the matrix before using it to multiply with the vector, just in case mine was intended for a different matrix layout. Afterall, I gave you Java code, which is primarily OpenGL, whereas Game Maker uses DirectX. My code does give you a 3d point in world space. Which is why the camera position was subtracted afterwards, to get a directional vector. But it's always safe to print out stuff as you go along! I also still do not know why you are multiplying mouse_x/y by 2 twice. Print out your NDC coordinates to verify that they are actually in NDC space (-1 to 1). The conversion I gave you works like this: mousex / maximumx --> This will give you a value between 0 and 1 lastvalue * 2.0 --> This puts it in range of 0 to 2 lastvalue - 1.0 --> This puts it in range of -1 to 1 If I still had the matrix-based fake 3d engine I made on the old GMC forums, I'd give you all the matrix functions you need. But I can't seem to find it
It took a while to get the code fully working, but @orange451, you were correct about the matrix layout. I managed to get your code showing the same vector as that of @flyingsaucerinvasion , and I also verified it with other code. The reason neither was working was my own fault. I misunderstood the way that the P3DC command worked; it does not, in fact, demand 2 points to form the ray, but rather a single point and a direction vector. So, once I got the vector, I simply got the data (3 points) of the last point that ray intersected, and used those 3 points to construct a plane equation for that triangle. Then, I used the vector and the camera location and direction to create a line equation, and found out where the line intersected the plane. I also used the normal of the plane, and used the cross product with a vector along the ground to find the angle between those vectors, so I would know if the location I was clicking on was too steep for the player to move, and disallowed the player to click there. The only problem with the current system is that it does not work in fullscreen (apparently due to aspect ratio issues compared to room size), but I think I can fix this with a bit more work. Here's a video of it working: Spoiler I thank you both IMMENSELY for your help; this is a fundamental system within the game, and it could never have even come close to fruition without your help.