3D Conversion from 3D to 2D and vice-versa slightly wrong.

Anixias

Member
Hello. I found online that in order to convert a 3D point to a 2D vector representing a point on your screen, you must:
  1. Transform the vec3 (3d point) by the camera's View Matrix
  2. Transform the vec3 produced by step 1 by the camera's Projection Matrix
  3. Divide the vec3's x and y values by the z value.
  4. Convert the x and y value from a range of (-1,1) to a range of (0,screen width/height)
I did this using this script I wrote:
Code:
/// @param x
/// @param y
/// @param z
/// @param viewmat
/// @param projmat
/// @param screenwidth
/// @param screenheight
//Returns a vec2 containing screen coordinates
var vec3;
var xx = argument[0], yy = argument[1], zz = argument[2];
var viewmat = argument[3], projmat = argument[4];
var w = argument[5], h = argument[6];

vec3 = matrix_transform_vertex(viewmat,xx,yy,zz);
vec3 = matrix_transform_vertex(projmat,vec3[0],vec3[1],vec3[2]);

vec3[0] /= vec3[2];
vec3[1] /= vec3[2];

var a;

a[0] = w*(vec3[0]+1)/2;
a[1] = h*(vec3[1]+1)/2;

return a;
It works perfectly if the 3D point is the lookat point of the View Matrix, but if it isn't, it will be slightly off, and the error is amplified the more zoomed in the camera is to the 3D point.


Now, I decided to temporarily ignore this issue and try 2D screen coordinate to 3D vector conversion. The result is just... odd? Here is that script:
Code:
/// @param x
/// @param y
/// @param viewmat
/// @param projmat
/// @param screenwidth
/// @param screenheight
//Returns a vec3 containing world coordinates
var vec3;
var xx = argument[0], yy = argument[1];
var viewmat = argument[2], projmat = argument[3];
var w = argument[4], h = argument[5];

var a;
a[0] = 2*(xx/w)-1;
a[1] = 2*(yy/h)-1;
vec3 = matrix_transform_vertex(projmat,a[0],a[1],0);
vec3 = matrix_transform_vertex(viewmat,vec3[0],vec3[1],vec3[2]);
vec3[0] *= -1;
return vec3;
The X-Coordinate was negative even when pointing my mouse towards the lookat-vector (which is the reason for the next-to-last line). The vector itself, in relation to the camera, is facing the opposite way. I simply subtract the vector returned by this script by the camera position vector, normalize it, and then multiply it by 10 so that it is visible. The Y-Coordinate, in relation to the camera, is flipped, so that moving my mouse down on the screen actually moves the 3D point up, rather than down.


What's wrong with my scripts? What are working alternatives, if you have any? Note: I'm using perspective projections, and +Z is up.
 
Last edited:
M

Misu

Guest
Why are you using perspective projections? Anyway I too am in the investigation of an alternative remedy for 3D to 2D conversion (only Im looking for one for shader).

I remember rumaging the old GMC forum and found an old thread by Phantom who used Yourself's 3D conversion technique. It goes something like this:

Code:
///Convert_3d(targetx,targety,targetz,xfrom,from,zfrom,view)

var pX, pY, pZ, mm;
pX = argument0 - argument3;
pY = argument1 - argument4;
pZ = argument2 - argument5;
mm = pX*dX + pY*dY + pZ*dZ;
     
if mm > 0
begin
    pX /= mm;
    pY /= mm;
    pZ /= mm;
end;
else 
begin
    x_2d = 0;
    y_2d = -100;
    return 0;
end;

mm = (pX*vX + pY*vY + pZ*vZ) / sqr((view_wview[argument6]/view_hview[argument6])*tan(45*pi/360));
x_2d = (mm+1)/2*view_wview[argument6];
mm = (pX*uX + pY*uY + pZ*uZ) / sqr(tan(45*pi/360));
y_2d = (1-mm)/2*view_hview[argument6];
return 1;
And he had also the 2D to 3D conversion as well...
Code:
// Convert_2d(x,y,xfrom,yfrom,zfrom,view)
// The script returns the 3d x and y coordinates at z = 0

screenx = 2*argument0/view_wview[argument5]-1;
screeny = 1-2*argument1/view_hview[argument5];
mX = dX + uX*screeny + vX*screenx;
mY = dY + uY*screeny + vY*screenx;
mZ = dZ + uZ*screeny + vZ*screenx;

if mZ != 0
begin
    x_3d = argument2 - argument4 * mX / mZ;
    y_3d = argument3 - argument4 * mY / mZ;
end;
else
begin
    x_3d = argument2 - argument4 * mX;
    y_3d = argument3 - argument4 * mY;
end;
Note that this is from GM8. Youll have to find a way to change the view variables for the camera functions. Even though, this is just an alternative way of doing it, I cant garantee it would be helpful. If anyone has any better solutions with GMS's much advance functionalities, Id be delighted with the info. Good luck.
 

Anixias

Member
Thank you for the reply! I’m using perspective projections because that’s how most 3D games are projected.

However, those scripts don’t explain what u and v variables mean in that context, so they aren’t usable straight out, and I’d also like to figure out what’s wrong with my scripts as well.
 
3d to 2d point within view port if you already have a matrix which is projection * view:
Code:
    var _w =  _x0 * _pv[3] + _y0 * _pv[7] + _z0 * _pv[11] + _pv[15];
    if (_w > 0) {
        var _x = (_x0 * _pv[0] + _y0 * _pv[4] + _z0 * _pv[8]  + _pv[12]) / _w;
        var _y = (_x0 * _pv[1] + _y0 * _pv[5] + _z0 * _pv[9]  + _pv[13]) / _w;
        _x = _half_port_width + _x * _half_port_width;
        _y = _half_port_height - _y * _half_port_height;

        //drawing a sprite at the 2d point (must have changed to orthographic projection):
        draw_sprite(spr_cross,0,_x,_y);
    }
_x0,_y0,_z0 is your 3d point.
_pv is projection * view

if you need to convert a point in object space to 2d, then use projection*view*world instead.

if w is greater than zero, then the point is front of the camera.

But it is not necessarily within the view frustum. If you need to know that, then you also need to compute z/w:

var _z = (_x0 * _pv[2] + _y0 * _pv[6] + _z0 * _pv[10] + _pv[14]) / _w;

If z/w is less than zero or greater than 1, then the point is beyond the near or far clip planes.

Also before this line "_x = _half_port_width + _x * _half_port_width;", if abs(x) or abs(y) > 1, then the point is outside of the left,top,right, or bottom parts of the view frustrum. Same is true after that line if _x or _y is less than zero, or x is greater than port_width or y is greater than port_height.

to go from 2d to 3d, you need a z component to your vector, otherwise how far away is it from the camera?

2d-"ish" to 3d:

Code:
    var _t = dtan( cam_fov / 2 );
    _x0 = cam_aspect * _t * ( 2 * _x0 / port_width - 1 ) * _z0;
    _y0 =              _t * ( 1 - 2 * _y0 / port_height ) * _z0;
    var _x = cam_x + _x0 * v[0] + _y0 * v[1] + _z0 * v[2];
    var _y = cam_y + _x0 * v[4] + _y0 * v[5] + _z0 * v[6];
    var _z = cam_z + _x0 * v[8] + _y0 * v[9] + _z0 * v[10];

    //drawing a block at the 3d point:
    d3d_draw_block(_x-10, _y-10, _z-10, _x+10, _y+10, _z+10, -1, 1, 1)
_x0,_y0 initial value is a 2d point on the view port
_z0 is the depth of that point
_v is the view matrix.
cam_x,cam_y,cam_z is the position of the camera
output is the 3d point _x,_y,_z
 
Last edited:

Anixias

Member
Thanks for the reply, but the first script is actually way off. Here is the script I'm using (slightly modified form of yours):
Code:
/// @param x
/// @param y
/// @param z
/// @param viewmat
/// @param projmat
/// @param screenwidth
/// @param screenheight
//Returns a vec2 containing screen coordinates
var _x0 = argument[0], _y0 = argument[1], _z0 = argument[2];
var viewmat = argument[3], projmat = argument[4];
var w = argument[5], h = argument[6];
var _pv = matrix_multiply(viewmat,projmat);
var _half_port_width = w/2;
var _half_port_height = h/2;
var _w =  _x0 * _pv[3] + _y0 * _pv[7] + _z0 * _pv[11] + _pv[15];
if (_w > 0) {
    var _x = (_x0 * _pv[0] + _y0 * _pv[4] + _z0 * _pv[8]  + _pv[12]) / _w;
    var _y = (_x0 * _pv[1] + _y0 * _pv[5] + _z0 * _pv[9]  + _pv[13]) / _w;
    _x = _half_port_width + _x * _half_port_width;
    _y = _half_port_height - _y * _half_port_height;
    var a;
    a[0] = _x;
    a[1] = _y;
    return a;
}
var a;
a[0] = 0;
a[1] = 0;
return a;
It actually doesn't seem to return the correct value unless I'm looking at a very odd angle.

EDIT:
Reversing the order in which the projection and view matrices are multiplied causes _w to always be greater than 0, causing the script to always return (0,0).
 
ah. I'm checking to see if I'm making any bad assumptions about my code. I don't see anything except I'm presuming everything works the same way in GMS2.

Well, first of all, how are you checking the results? If you draw something at that position, you have to first switch to an orthographic projection at zero, and the size of your port. In gms1.4 that would be d3d_set_projection_ortho(0,0,port_w,port_h,0);

the only other thing I can think of is maybe they switched the multiplication order of matrix_multiply. Becuase in GMS1.4 matrix_multiply(a,b), does b * a, which I always found confusing. Mayby they reversed it?? Long shot, but worth a try. The more likely thing in my mind is that you are trying to use the resulting 2d coordinate in a way which is wrong somehow. You also have to make sure you are stargint with the correct 3d coordinates, of course.

There is another possibility, that the coordinate system is mirrored, and that just might be throwing things off. I am using left-handed coordinate system in GMS1.4. DO you happen to know which one you are using? I suspect GMS2 has not changed this, and that you are still using a left-handed system.
 

Anixias

Member
I use the Draw GUI event to draw the result as a colored circle on the screen to test it, do I still need to switch to an orthographic projection? And yeah, doing matrix_multiply(viewmat,projmat) shows incorrect results, and the opposite shows no results. The 3D point is at the player spawn point, so if I look straight down when the game starts, it should be in the middle of the screen.

EDIT:
I believe I am using a left-hand system.
+X is right, +Y is down (as in 2D), and +Z is up (in 3D), vertically.
 
Last edited:
is the point too close to the camera? within the near clipping plane?

drawing in the gui event should work as long as the ratio between the gui and your 3d port is 1:1.

do you think it would help if i set up a simple project in gms1.4 for you to examine?

edit: your description of your coordinate system is ambiguous to me. Imagine you are looking down positive x in 3d. what direction do positive y and z point? Pretty sure it will still be left-handed in GMS2, though you can change handedness by modifying the view or projection matrices.
 

Anixias

Member
When I walk away from the point, the result gets stranger, and it is not too close. And yes, the ratio is 1:1.

EDIT:
When looking in the direction of +X, +Y is to your right, and +Z is up.
 
Last edited:
yeah that's left-handed.

I just double checked in a fresh project, and what I posted is giving correct results for me. You sure you are providing the right arguments to the script? Not accidentally using port width instead of height, for example?

Let me ask you this, does the 2d point appear maybe mirrored across the center of the screen?
 

Anixias

Member
Here is a video of what is happening. The red circle should draw where the player spawns.
EDIT:
At first, it looks like it's just the Z that is wrong, but at the end, it just looks totally wrong.
 
Based on a quick test I just did, assuming I am not mistaken, the flipped vertical axis can be explained as a change form left-handed (what I'm using in gms1.4) to a right-handed coordinates system. Not 100% sure that is what is happening though.

If you make a line primitive model with a different color to point in each positive axis direction, it is the surest way to visually see what coordinate system you are in.
 
Last edited:

Anixias

Member
Alright. And, by the way, the 2D to 3D script works out of the box. Thank you so much for your help!
 
P

Pinqu

Guest
3d to 2d point within view port if you already have a matrix which is projection * view:
to go from 2d to 3d, you need a z component to your vector, otherwise how far away is it from the camera?

2d-"ish" to 3d:

Code:
    var _t = dtan( cam_fov / 2 );
    _x0 = cam_aspect * _t * ( 2 * _x0 / port_width - 1 ) * _z0;
    _y0 =              _t * ( 1 - 2 * _y0 / port_height ) * _z0;
    var _x = cam_x + _x0 * v[0] + _y0 * v[1] + _z0 * v[2];
    var _y = cam_y + _x0 * v[4] + _y0 * v[5] + _z0 * v[6];
    var _z = cam_z + _x0 * v[8] + _y0 * v[9] + _z0 * v[10];

    //drawing a block at the 3d point:
    d3d_draw_block(_x-10, _y-10, _z-10, _x+10, _y+10, _z+10, -1, 1, 1)
_x0,_y0 initial value is a 2d point on the view port
_z0 is the depth of that point
_v is the view matrix.
cam_x,cam_y,cam_z is the position of the camera
output is the 3d point _x,_y,_z
Hi, I tried using this code, but unfortunately it didn't seem to work for me..

I set it up like this
Code:
var height = display_get_height();
var width = floor(height*1.777777777777778);
var cam_aspect = width/height;
var cam_fov = 60;
var cam_x = x;
var cam_y = y+100;
var cam_z = -200;
var v = camera_get_view_mat(view_camera[0]);
 
_x0 = window_mouse_get_x();
_y0 = window_mouse_get_y();
_z0 = ?

var _t = dtan( cam_fov / 2 );
_x0 = cam_aspect * _t * ( 2 * _x0 / view_wport[0] - 1 ) * _z0;
_y0 =              _t * ( 1 - 2 * _y0 / view_hport[0] ) * _z0;
_x = cam_x + _x0 * v[0] + _y0 * v[1] + _z0 * v[2];
_y = cam_y + _x0 * v[4] + _y0 * v[5] + _z0 * v[6];
var _z = cam_z + _x0 * v[8] + _y0 * v[9] + _z0 * v[10];

draw_sprite(spr_square, 0, _x, _y);
The code is inside the player instance, that is being followed by the camera.
As you might have already noticed I want the _x/_y to follow the mouse.
Now I am not sure what to do with _z0. If I leave it at 0, the sprite is just stays at player.x and y+100.
So do I have to somehow calculate this or is this a fixed value?
If I set it to 200 for example (camera height) is moves the _x, _y, but the _x goes faster then the mouse and the _y seems inverted.. -> This is also funny enought the same behaviour that I get when I use the code that was posted by Misu (2nd post).

Any thoughts on how I might get this to work properly?
 
Hi, I tried using this code, but unfortunately it didn't seem to work for me..
So, you've got the 2d to 3d script. This is actually a bit of a misleading description. Because it is actually just takeing the initial 3d vector _x0, _y0, _z0, and rotating it from camera space into global space.

The output _x, _y, _z, is the 3d vector _x0, _y0, _z0 rotated from camera space into global space. If you try to draw something at _x, _y, you will basically get meaningless results.

What I suspect you want to do is to find the location at which that vector intersects with something else. Maybe you are trying to see where it intersects the ground? Is your ground completely flat? Is the ground located at z = 0? If so, you can see where the vector (_x,_y_z) intersects the ground like this:

Code:
    if (_z != 0) {
        var _t = -cam_z / _z;
        if (_t >= 0) {  //if _t is less than zero, then mouse vector points away from ground
            intersection_x = cam_x + _x * _t;
            intersection_y = cam_y + _y * _t;
        }
    }
if y still seems inverted, you could try to adjust the _y0 computation on these lines to this:

_x0 = cam_aspect * _t * ( 2 * _x0 / view_wport[0] - 1 ) * _z0;
_y0 = _t * ( 2 * _y0 / view_hport[0] - 1 ) * _z0;

The initial value of _z0 doesn't really matter, unless you want a vector of unit length, in which case you can do this:
Code:
        _x0 = cam_aspect * _t * ( 2 * _x0 / view_wport[0] - 1 ) * _z0;
        _y0 =              _t * ( 1 - 2 * _y0 / view_hport[0] ) * _z0;
        var _z0 = 1 / sqrt( _x0 * _x0 + _y0 * _y0 + 1 );
        _x0 *= _z;
        _y0 *= _z;
But it occurs to me, that the floating point errors in the subsquent calculations might make the length of that vector stray farther from 1 than you'd really like. In which case, it might be better to normalize _x,_y,_z, at the end of the whole thing. In which case, we're back to the initial value of _z0 not mattering.
 
P

Pinqu

Guest
Hey, thanks for taking the time to reply!

Code:
_x0 = cam_aspect * _t * ( 2 * _x0 / view_wport[0] - 1 ) * _z0;
_y0 = _t * ( 2 * _y0 / view_hport[0] - 1 ) * _z0;
This fixed the inverted problem :)

Code:
if (_z != 0) {
       var _t = -cam_z / _z;
       if (_t >= 0) {  //if _t is less than zero, then mouse vector points away from ground
           intersection_x = cam_x + _x * _t;
           intersection_y = cam_y + _y * _t;
       }
   }
Not sure how this works.. I got
_z0 = 0; //intitial
cam_z = -200; //initial

var _z = cam_z + _x0 * v[8] + _y0 * v[9] + _z0 * v[10];
show_debug_message("_z"+string(_z)); >>>> outcome -200

var _t = -cam_z / _z;
show_debug_message("_t"+string(_t)); >>>> outcome -1
so (_t >= 0) = false

I just want to be able to use the mouse to click and select an enemy, so the player targets them and if they are in range starts shooting.
In 2d you would just use mouse_x and mouse_y, but that obviously does not work in 3d..
Perhaps it would be easier if I would just post a project file for you to examine. This way it is also easier for you to see what the game looks like.
https://www.dropbox.com/s/szb5wis4cgq2qdq/prototype.yyz?dl=0
(I know I should not do these calculations in the draw event but that's just for prototyping, later on it will be moved to the step event).
Hope this can be done with this code.
 
In your player Step Event, line 6, you have this:

Code:
var target_id = collision_point(mouse_x, mouse_y, obj_enemy1, false, true);
And, I see you have already converted the 3d world position in the Player Draw Event:
Code:
_x = cam_x + _x0 * v[0] + _y0 * v[1] + _z0 * v[2];
_y = cam_y + _x0 * v[4] + _y0 * v[5] + _z0 * v[6];
Because collision_point() is a 2D function, I just replaced mouse_x, mouse_y with _x and _y like this:
Code:
var target_id = collision_point(_x, _y, obj_enemy1, false, true);
And when I test it, I can click on the enemy and the player will shoot them.
 
P

Pinqu

Guest
This also works for me, but the problem is that is mouse is not in the same position as the draw_sprite(spr_square, 0, _x, _y); For example when the mouse is in the middle, the square is somewhere outside the view.
So if I use the mouse cursor to click the enemies nothing happens, but if I use the square to select an enemy is does work.
Also there seems to be a difference in the offset between fullscreen and windowed.
 
Last edited by a moderator:

LittleAngel

Member
And he had also the 2D to 3D conversion as well...
Code:
// Convert_2d(x,y,xfrom,yfrom,zfrom,view)
// The script returns the 3d x and y coordinates at z = 0

screenx = 2*argument0/view_wview[argument5]-1;
screeny = 1-2*argument1/view_hview[argument5];
mX = dX + uX*screeny + vX*screenx;
mY = dY + uY*screeny + vY*screenx;
mZ = dZ + uZ*screeny + vZ*screenx;

if mZ != 0
begin
    x_3d = argument2 - argument4 * mX / mZ;
    y_3d = argument3 - argument4 * mY / mZ;
end;
else
begin
    x_3d = argument2 - argument4 * mX;
    y_3d = argument3 - argument4 * mY;
end;


Very interesting script. I'm looking for a script to convert a 2d point to 3D coordinates with Game Maker studio 1.4. But could you specify your script ? What are the variables dX, dY, dZ? They are not defined. uX, uY, uZ? Thanks !

I've created a new post here
 
Last edited:
Top