angelwire
Member
GM Version: 1.4.1763
Target Platform: ALL
Download: GMS:1.4 .gmz file
Links: N/A
Summary:
This tutorial is an example of how to take pixel art sprites and convert them to 3d models in real time. This makes the sprites look solid and "voxelized" in the 3d space as opposed to flat planes. Since it's done dynamically in real time there's no need to use any modelling software or do any sort of 3d design.
Disclaimer:
I am by no means a perfect programmer, so let me know if you see any avenues for optimization or know how to make my code better.
Tutorial:
If you're reading this, hopefully you already have your 3d environment set up, but if not here's a quick way to set up a 3d camera object in an empty project (using 1.4)
I'm going to call the object that creates and draws the model "player_object"
Just a note here, Make sure your sprite is checked to be used for 3d
In the player_object CREATE EVENT you're need 2 lines of code:
For the player_object STEP EVENT you need just a few more lines:
Here's the first custom script. It checks to see if the model should be updated. It only works with an image_speed that's smaller than 1. You may need to write this code yourself depending on how you're animating your sprite. Updating the model every step is pretty costly so I strongly suggest only updating the model when the image changes.
SCRIPT: sprite_should_update
Now here's the big script, the one that does all the hard work. I’ll explain what I’m doing with the d3d_model_vertex_normal_texture at the end, but the rest is documented.
SCRIPT: create_model_from_sprite
Here's a rundown of how the d3d_model_vertex_normal_texture is working:
Temp is the model (pretty simple)
The xx and yy values are the current pixel position being read from the buffer (again, the yy value starts at the bottom and works its way to the top)
The normals are currently set so that each individual cube is smoothly shaded
The UV coordinates are placed in a way that allows for a neat trick. (see Extra 1 at the bottom) The top faces are each placed at the top of their respective pixel, the bottom faces at the bottom of that pixel, and the left and right on the left side and right side of the pixel. The front and back faces are in the center This doesn’t change the color of the pixel unless the texture you’re applying to the model is a different size than the sprite you’re converting to a model (which is where the cool thing comes in)
All that’s left now is to draw the model with the proper scale, position, and rotation. Here’s a quick start guide to doing that:
Now that's all it takes to get it up and running. It's really just the start, however.
Here are a couple options and extras things you can do:
Here's another download link to the project file: GMS:1.4 .gmz file
I'll be happy to answer any questions or make improvements that might be necessary.
Target Platform: ALL
Download: GMS:1.4 .gmz file
Links: N/A
Summary:
This tutorial is an example of how to take pixel art sprites and convert them to 3d models in real time. This makes the sprites look solid and "voxelized" in the 3d space as opposed to flat planes. Since it's done dynamically in real time there's no need to use any modelling software or do any sort of 3d design.
Disclaimer:
I am by no means a perfect programmer, so let me know if you see any avenues for optimization or know how to make my code better.
Tutorial:
If you're reading this, hopefully you already have your 3d environment set up, but if not here's a quick way to set up a 3d camera object in an empty project (using 1.4)
camera_object CREATE EVENT:
camera_object DRAW EVENT:
Code:
d3d_start();
d3d_set_hidden(false);
d3d_set_lighting(false);
d3d_set_culling(false);
draw_set_color(c_white);
z = 0;
target_x = player_object.x;
target_y = player_object.y;
target_z = 0;
Code:
d3d_set_projection(x,y,z,target_x,target_y,target_z,0,0,1);
I'm going to call the object that creates and draws the model "player_object"
Just a note here, Make sure your sprite is checked to be used for 3d
In the player_object CREATE EVENT you're need 2 lines of code:
Code:
my_model = -1; //Initiate the model to -1
my_thickness = 2; //Set the thickness of the model
Code:
//Only update the model if the image has changed
if sprite_should_update() //Custom script
{
d3d_model_destroy(my_model); //Destroy the past model
texture_set_interpolation(false); //Interpolation should be turned off
//Create new model from the sprite, image frame, and the thickness that was set in the CREATE EVENT
my_model = create_model_from_sprite(sprite_index, image_index, my_thickness); //Custom script
}
SCRIPT: sprite_should_update
Code:
///sprite_should_update()
return (floor(image_index) == round(image_index));
SCRIPT: create_model_from_sprite
Code:
///create_model_from_sprite(sprite,index,depth)
var sprite = argument0; //The sprite that should be modeled
var index = argument1; //The frame of the sprite that should be modeled
var dd = argument2; //This is for the depth/thickness
var temp = d3d_model_create(); //This is the model that will be returned
var ii = 0; //A loop variable
var bool_value = false; //Whether or not a pixel is visible
var last_bool_value = false; //Whether or not the pixel to the left is visible
var side_offset = .4; //This sets the UV coordinates to the edges of the pixel
var sprite_buffer; //The buffer that holds the sprite contents
var surface_size = sprite_get_width(sprite); //How big the surface is
var xx = 0; //The X position
var yy = surface_size-1; //The Y position (starts from bottom)
var op = 1/(surface_size); //one pixel on the UV map
var op_middle = op*.5; //Half of one pixel, used to place UV coordinates in the middle of a pixel
//Create the surface that will be converted to a buffer and set it as the target
var my_surface = surface_create(surface_size,surface_size);
surface_set_target(my_surface);
//Clear the surface and draw the sprite
draw_clear_alpha(c_black,0.0);
draw_sprite(sprite,index,sprite_get_xoffset(sprite),sprite_get_yoffset(sprite));
surface_reset_target(); //Reset the target
//Create the buffer and convert the surface to a buffer
sprite_buffer = buffer_create((surface_size*surface_size)*4,buffer_fast,1);
buffer_get_surface(sprite_buffer,my_surface,1,0,0);
surface_free(my_surface); //Free the surface from memory
d3d_model_primitive_begin(temp,pr_trianglelist); //Begin the model
while(ii < surface_size*surface_size) //For every pixel in the surface
{
buffer_seek(sprite_buffer,buffer_seek_relative,3); //The RGB values are ignored so skip them
//If the value of the pixel's alpha is larger than 128 count the pixel as visible
bool_value = buffer_read(sprite_buffer,buffer_u8) > 128;
if bool_value = true //If the pixel is visible
{
/*
Currently I haven't figured out a way to selectively draw the needed faces
So right now, all 6 sides are being drawn,
This means even if a pixel is surrounded by other pixels,
It will draw the top, bottom, left, and right sides,
Those sides may not be visible so they are redundant
(there is one exception and you'll see it below)
*/
//Front
d3d_model_vertex_normal_texture(temp,xx,yy,0,-1,-1,1,(op*xx)+(op_middle),(op*yy)+(op_middle));
d3d_model_vertex_normal_texture(temp,xx+1,yy,0,1,-1,1,(op*xx)+(op_middle),(op*yy)+(op_middle));
d3d_model_vertex_normal_texture(temp,xx,yy+1,0,-1,1,1,(op*xx)+(op_middle),(op*yy)+(op_middle));
d3d_model_vertex_normal_texture(temp,xx+1,yy,0,1,-1,1,(op*xx)+(op_middle),(op*yy)+(op_middle));
d3d_model_vertex_normal_texture(temp,xx+1,yy+1,0,1,1,1,(op*xx)+(op_middle),(op*yy)+(op_middle));
d3d_model_vertex_normal_texture(temp,xx,yy+1,0,-1,1,1,(op*xx)+(op_middle),(op*yy)+(op_middle));
//Back
d3d_model_vertex_normal_texture(temp,xx,yy,dd,-1,-1,-1,(op*xx)+(op_middle),(op*yy)+(op_middle));
d3d_model_vertex_normal_texture(temp,xx+1,yy,dd,1,-1,-1,(op*xx)+(op_middle),(op*yy)+(op_middle));
d3d_model_vertex_normal_texture(temp,xx,yy+1,dd,-1,1,-1,(op*xx)+(op_middle),(op*yy)+(op_middle));
d3d_model_vertex_normal_texture(temp,xx+1,yy,dd,1,-1,-1,(op*xx)+(op_middle),(op*yy)+(op_middle));
d3d_model_vertex_normal_texture(temp,xx+1,yy+1,dd,1,1,-1,(op*xx)+(op_middle),(op*yy)+(op_middle));
d3d_model_vertex_normal_texture(temp,xx,yy+1,dd,-1,1,-1,(op*xx)+(op_middle),(op*yy)+(op_middle));
/*
Here's that one exception to what I mentioned earlier. If we know that the pixel to the left
is solid, we don't need to draw the left faces. It only saves a handful of faces,
but every little bit counts!
*/
//Left
if !last_bool_value
{
d3d_model_vertex_normal_texture(temp,xx,yy,0,-1,-1,1,(op*xx)-(op*side_offset)+(op_middle),(op*yy)+(op_middle));
d3d_model_vertex_normal_texture(temp,xx,yy,dd,-1,-1,-1,(op*xx)-(op*side_offset)+(op_middle),(op*yy)+(op_middle));
d3d_model_vertex_normal_texture(temp,xx,yy+1,dd,-1,1,-1,(op*xx)-(op*side_offset)+(op_middle),(op*yy)+(op_middle));
d3d_model_vertex_normal_texture(temp,xx,yy,0,-1,-1,1,(op*xx)-(op*side_offset)+(op_middle),(op*yy)+(op_middle));
d3d_model_vertex_normal_texture(temp,xx,yy+1,dd,-1,1,-1,(op*xx)-(op*side_offset)+(op_middle),(op*yy)+(op_middle));
d3d_model_vertex_normal_texture(temp,xx,yy+1,0,-1,1,1,(op*xx)-(op*side_offset)+(op_middle),(op*yy)+(op_middle));
}
//Right
d3d_model_vertex_normal_texture(temp,xx+1,yy,0,1,-1,1,(op*xx)+(op*side_offset)+(op_middle),(op*yy)+(op_middle));
d3d_model_vertex_normal_texture(temp,xx+1,yy,dd,1,-1,-1,(op*xx)+(op*side_offset)+(op_middle),(op*yy)+(op_middle));
d3d_model_vertex_normal_texture(temp,xx+1,yy+1,dd,1,1,-1,(op*xx)+(op*side_offset)+(op_middle),(op*yy)+(op_middle));
d3d_model_vertex_normal_texture(temp,xx+1,yy,0,1,-1,1,(op*xx)+(op*side_offset)+(op_middle),(op*yy)+(op_middle));
d3d_model_vertex_normal_texture(temp,xx+1,yy+1,dd,1,1,-1,(op*xx)+(op*side_offset)+(op_middle),(op*yy)+(op_middle));
d3d_model_vertex_normal_texture(temp,xx+1,yy+1,0,1,1,1,(op*xx)+(op*side_offset)+(op_middle),(op*yy)+(op_middle));
//Top
d3d_model_vertex_normal_texture(temp,xx,yy,0,-1,-1,1,(op*xx)+(op_middle),(op*yy)-(op*side_offset)+(op_middle));
d3d_model_vertex_normal_texture(temp,xx,yy,dd,-1,-1,-1,(op*xx)+(op_middle),(op*yy)-(op*side_offset)+(op_middle));
d3d_model_vertex_normal_texture(temp,xx+1,yy,dd,1,-1,-1,(op*xx)+(op_middle),(op*yy)-(op*side_offset)+(op_middle));
d3d_model_vertex_normal_texture(temp,xx,yy,0,-1,-1,1,(op*xx)+(op_middle),(op*yy)-(op*side_offset)+(op_middle));
d3d_model_vertex_normal_texture(temp,xx+1,yy,0,1,-1,1,(op*xx)+(op_middle),(op*yy)-(op*side_offset)+(op_middle));
d3d_model_vertex_normal_texture(temp,xx+1,yy,dd,1,-1,-1,(op*xx)+(op_middle),(op*yy)-(op*side_offset)+(op_middle));
//Bottom
d3d_model_vertex_normal_texture(temp,xx,yy+1,0,-1,1,1,(op*xx)+(op_middle),(op*yy)+(op*side_offset)+(op_middle));
d3d_model_vertex_normal_texture(temp,xx,yy+1,dd,-1,1,-1,(op*xx)+(op_middle),(op*yy)+(op*side_offset)+(op_middle));
d3d_model_vertex_normal_texture(temp,xx+1,yy+1,dd,1,1,-1,(op*xx)+(op_middle),(op*yy)+(op*side_offset)+(op_middle));
d3d_model_vertex_normal_texture(temp,xx,yy+1,0,-1,1,1,(op*xx)+(op_middle),(op*yy)+(op*side_offset)+(op_middle));
d3d_model_vertex_normal_texture(temp,xx+1,yy+1,0,1,1,1,(op*xx)+(op_middle),(op*yy)+(op*side_offset)+(op_middle));
d3d_model_vertex_normal_texture(temp,xx+1,yy+1,dd,1,1,-1,(op*xx)+(op_middle),(op*yy)+(op*side_offset)+(op_middle));
}
/*
We're saving the bool_value in another variable so we can check
whether this pixel was solid next time the loop goes around
*/
last_bool_value = bool_value;
xx+=1; //Increment the xx value
if xx >= surface_size //If xx is too far over
{
yy-=1; //Move the yy up one
xx = 0; //Move the xx back to the start
}
ii+=1; //Increment ii so we know how many times we've gone through this
}
d3d_model_primitive_end(temp); //The model has been created so we finalize it
buffer_delete(sprite_buffer); //The buffer is no longer needed so we delete it
return temp; //Return the model
Temp is the model (pretty simple)
The xx and yy values are the current pixel position being read from the buffer (again, the yy value starts at the bottom and works its way to the top)
The normals are currently set so that each individual cube is smoothly shaded
The UV coordinates are placed in a way that allows for a neat trick. (see Extra 1 at the bottom) The top faces are each placed at the top of their respective pixel, the bottom faces at the bottom of that pixel, and the left and right on the left side and right side of the pixel. The front and back faces are in the center This doesn’t change the color of the pixel unless the texture you’re applying to the model is a different size than the sprite you’re converting to a model (which is where the cool thing comes in)
All that’s left now is to draw the model with the proper scale, position, and rotation. Here’s a quick start guide to doing that:
Code:
d3d_transform_set_identity()
d3d_transform_add_scaling(1,1,1);
d3d_transform_add_translation(x,y,0);
d3d_model_draw(my_model,-sprite_width*.5,-sprite_width*.5,-my_thickness*.5,sprite_get_texture(sprite_index,image_index));
d3d_transform_set_identity();
Here are a couple options and extras things you can do:
Thanks to @MaddeMichael for the awesome vertex buffer guide
Here's the custom buffer_vertex_normal_texture script I used, for some reason the normals don't work, I'm hoping someone smarter than me can figure it out.
Code:
///create_vertex_buffer_from_sprite(sprite,index,depth)
var sprite = argument0; //The sprite that should be modeled
var index = argument1; //The frame of the sprite that should be modeled
var dd = argument2; //This is for the depth/thickness
var temp = d3d_model_create(); //This is the model that will be returned
var ii = 0; //A loop variable
var bool_value = false; //Whether or not a pixel is visible
var last_bool_value = false; //Whether or not the pixel to the left is visible
var side_offset = .4; //This sets the UV coordinates to the edges of the pixel
var sprite_buffer; //The buffer that holds the sprite contents
var surface_size = sprite_get_width(sprite); //How big the surface is
var xx = 0; //The X position
var yy = surface_size-1; //The Y position (starts from bottom)
var op = 1/(surface_size); //one pixel on the UV map
var op_middle = op*.5; //The pixel middle on the UV map
//Create the surface that will be converted to a buffer and set it as the target
var my_surface = surface_create(surface_size,surface_size);
surface_set_target(my_surface);
//Clear the surface and draw the sprite
draw_clear_alpha(c_black,0.0);
draw_sprite(sprite,index,sprite_get_xoffset(sprite),sprite_get_yoffset(sprite));
surface_reset_target(); //Reset the target
//Create the buffer and convert the surface to a buffer
sprite_buffer = buffer_create((surface_size*surface_size)*4,buffer_fast,1);
buffer_get_surface(sprite_buffer,my_surface,1,0,0);
surface_free(my_surface); //Free the surface from memory
//Begin defining a format
vertex_format_begin();
vertex_format_add_position_3d();//Add 3D position info
vertex_format_add_colour();//Add colour info
vertex_format_add_textcoord();//Texture coordinate info
//End building the format, and assign the format to the variable "format"
var format = vertex_format_end();
vertex_begin(temp,format);
while(ii < surface_size*surface_size) //For every pixel in the surface
{
buffer_seek(sprite_buffer,buffer_seek_relative,3); //The RGB values are ignored so skip them
//If the value of the pixel's alpha is larger than 128 count the pixel as visible
bool_value = buffer_read(sprite_buffer,buffer_u8) > 128;
if bool_value = true //If the pixel is visible
{
//Front
buffer_vertex_normal_texture(temp,xx,yy,0,-1,-1,1,(op*xx)+(op_middle),(op*yy)+(op_middle));
buffer_vertex_normal_texture(temp,xx+1,yy,0,1,-1,1,(op*xx)+(op_middle),(op*yy)+(op_middle));
buffer_vertex_normal_texture(temp,xx,yy+1,0,-1,1,1,(op*xx)+(op_middle),(op*yy)+(op_middle));
buffer_vertex_normal_texture(temp,xx+1,yy,0,1,-1,1,(op*xx)+(op_middle),(op*yy)+(op_middle));
buffer_vertex_normal_texture(temp,xx+1,yy+1,0,1,1,1,(op*xx)+(op_middle),(op*yy)+(op_middle));
buffer_vertex_normal_texture(temp,xx,yy+1,0,-1,1,1,(op*xx)+(op_middle),(op*yy)+(op_middle));
//Back
buffer_vertex_normal_texture(temp,xx,yy,dd,-1,-1,-1,(op*xx)+(op_middle),(op*yy)+(op_middle));
buffer_vertex_normal_texture(temp,xx+1,yy,dd,1,-1,-1,(op*xx)+(op_middle),(op*yy)+(op_middle));
buffer_vertex_normal_texture(temp,xx,yy+1,dd,-1,1,-1,(op*xx)+(op_middle),(op*yy)+(op_middle));
buffer_vertex_normal_texture(temp,xx+1,yy,dd,1,-1,-1,(op*xx)+(op_middle),(op*yy)+(op_middle));
buffer_vertex_normal_texture(temp,xx+1,yy+1,dd,1,1,-1,(op*xx)+(op_middle),(op*yy)+(op_middle));
buffer_vertex_normal_texture(temp,xx,yy+1,dd,-1,1,-1,(op*xx)+(op_middle),(op*yy)+(op_middle));
//Left
if !last_bool_value
{
buffer_vertex_normal_texture(temp,xx,yy,0,-1,1,0,op*xx,op*yy);
buffer_vertex_normal_texture(temp,xx,yy,dd,-1,1,0,op*xx,op*yy);
buffer_vertex_normal_texture(temp,xx,yy+1,dd,-1,1,0,op*xx,op*yy);
buffer_vertex_normal_texture(temp,xx,yy,0,-1,1,0,op*xx,op*yy);
buffer_vertex_normal_texture(temp,xx,yy+1,dd,-1,1,0,op*xx,op*yy);
buffer_vertex_normal_texture(temp,xx,yy+1,0,-1,1,0,op*xx,op*yy);
}
//Right
buffer_vertex_normal_texture(temp,xx+1,yy,0,-1,1,0,op*xx,op*yy);
buffer_vertex_normal_texture(temp,xx+1,yy,dd,-1,1,0,op*xx,op*yy);
buffer_vertex_normal_texture(temp,xx+1,yy+1,dd,-1,1,0,op*xx,op*yy);
buffer_vertex_normal_texture(temp,xx+1,yy,0,-1,1,0,op*xx,op*yy);
buffer_vertex_normal_texture(temp,xx+1,yy+1,dd,-1,1,0,op*xx,op*yy);
buffer_vertex_normal_texture(temp,xx+1,yy+1,0,-1,1,0,op*xx,op*yy);
//Top
buffer_vertex_normal_texture(temp,xx,yy,0,-1,1,0,op*xx,op*yy);
buffer_vertex_normal_texture(temp,xx,yy,dd,-1,1,0,op*xx,op*yy);
buffer_vertex_normal_texture(temp,xx+1,yy,dd,-1,1,0,op*xx,op*yy);
buffer_vertex_normal_texture(temp,xx,yy,0,-1,1,0,op*xx,op*yy);
buffer_vertex_normal_texture(temp,xx+1,yy,0,-1,1,0,op*xx,op*yy);
buffer_vertex_normal_texture(temp,xx+1,yy,dd,-1,1,0,op*xx,op*yy);
//Bottom
buffer_vertex_normal_texture(temp,xx,yy+1,0,-1,1,0,op*xx,op*yy);
buffer_vertex_normal_texture(temp,xx,yy+1,dd,-1,1,0,op*xx,op*yy);
buffer_vertex_normal_texture(temp,xx+1,yy+1,dd,-1,1,0,op*xx,op*yy);
buffer_vertex_normal_texture(temp,xx,yy+1,0,-1,1,0,op*xx,op*yy);
buffer_vertex_normal_texture(temp,xx+1,yy+1,0,-1,1,0,op*xx,op*yy);
buffer_vertex_normal_texture(temp,xx+1,yy+1,dd,-1,1,0,op*xx,op*yy);
}
/*
We're saving the bool_value in another variable so we can check
whether this pixel was solid next time the loop goes around
*/
last_bool_value = bool_value;
xx+=1; //Increment the xx value
if xx >= surface_size //If xx is too far over
{
yy-=1; //Move the yy up one
xx = 0; //Move the xx back to the start
}
ii+=1; //Increment ii so we know how many times we've gone through this
}
vertex_end(temp); //The model has been created so we finalize it
buffer_delete(sprite_buffer); //The buffer is no longer needed so we delete it
return temp; //Return the vertex buffer
Code:
///buffer_vertex_normal_texture(buffer,x,y,z,normal_x,normal_y,normal_z,tex_x,tex_y)
var temp = argument0;
var xx = argument1;
var yy = argument2;
var zz = argument3;
var nx = argument4;
var ny = argument5;
var nz = argument6;
var tx = argument7;
var ty = argument8;
var col = c_white;
var alpha = 1.0;
vertex_position_3d(temp, xx, yy, zz);
vertex_colour(temp, col, alpha);
vertex_texcoord(temp, tx, ty);
//vertex_normal(temp,nx,ny,nz);
Keep you original sprite, but make a copy. Now scale the copy up to 400% with the "Poor" quality selected (so there's no blurring). Now on each "pixel" of the sprite is made up of a square of 4x4 smaller pixels. Using the UV map image above, you can change the pixel colors for each of the sides of the pixel. For example, you can simulate lighting by making the top faces brighter and the bottom faces darker. Or by making all the side faces black you can give it an outline.
The alpha channel is the only part of the pixel that is being used. So you could pass in a heightmap to the create_model_from_sprite script to control how thick each individual pixel is.
Here's another download link to the project file: GMS:1.4 .gmz file
I'll be happy to answer any questions or make improvements that might be necessary.