Legacy GM Having trouble deforming multiple regions of several surfaces with shader

B

Buzz Killdrin

Guest
GM Version: 1.4
Target Platform: Windows
Download: N/A
Links: N/A

(Sorry for just dumping hundreds of lines of code like this, but I don't know what to narrow it down to so I'm posting the whole thing.)

I'm using fragment a fragment shader to add to the roster of customizations that can be displayed for multiple characters without actually making different art assets for different shaped or colored sprites. It deforms images by bulging them outward in a magnifying effect and then adds white or near white highlights to them - the last part is part of a process whereby I blend monochrome sprites with randomly created colors and then use the final part of the shader's code to render the unblended version of the monochrome sprite with only the bright parts visible and the rest having alpha = 0. Essentially I take 1 colored image and it's gray scale version, deform both and slap the (now also deformed) highlights to the former.

The images that I'm passing through the shader are in fact screen sized surfaces that I draw multiple sprites on, mainly to reduce the number of shader calls, but also because I don't know how to get the shader to not pick whatever the it wants from a texture page. Each surface represents a group of several drawn sprites for one of 3 characters, with a color version and a separate gray scale version.

After I randomize the visual features for each character, I store them into separate arrays that I can use in loops in the draw event. I also store coordinates on where I want the deformation centers to be and how big.

~~I should mention that the deformation kind of worked IF I was doing 1 per shader call, per surface, but that had the problem of also drawing versions of the surface that were not bulged out, with the visual result of having one normal image drawn on top of a deformed image (I only want to draw the deformed image, obviously).

Here's the fragment shader, vertex is vanilla. It's a variation of xygthop3's 'Free Shaders' marketplace submission (sorry, can't post links yet) --- I repeat a lot of the code since I've been told that using loops in a shader is very taxing.

Code:
   varying vec2 v_texcoord;

   uniform vec2 point_pos1;
   uniform vec2 point_pos2;
   uniform vec2 point_pos3;

   uniform vec2 resolution;

   uniform float circleRadiusBase1;
   uniform float circleRadiusBase2;
   uniform float circleRadiusBase3;

   uniform float multiplier1;
   uniform float multiplier2;
   uniform float multiplier3;

   void main()
   {
       float distInside1 = 0.0; //using them as  booleans
       float distInside2 = 0.0;
       float distInside3 = 0.0;

       float pointInside1 = 0.0;
       float pointInside2 = 0.0;
       float pointInside3 = 0.0;

       float newX; //uvs to draw only the distorted pixels at
       float newY;

       vec4 base_texture = texture2D(gm_BaseTexture, v_texcoord);

       vec2 uv1 = v_texcoord;
       uv1.x *= (resolution.x/resolution.y);
       float centre_x1 = (point_pos1.x / resolution.x) * (resolution.x/resolution.y);
       float centre_y1 = point_pos1.y / resolution.y;

       //check if pixel is within square tangent to the circle the distortion will take place in
       float circleRadius1 = circleRadiusBase1 * multiplier1;
       float maxX1 = centre_x1 + circleRadius1; //right horizontal limit of distortion
       float minX1 = centre_x1 - circleRadius1; //left h limit
       float maxY1 = centre_y1 + circleRadius1; //top v limit
       float minY1 = centre_y1 - circleRadius1; //bottom v limit

       if( uv1.x > minX1 && uv1.x < maxX1 && uv1.y > minY1 && uv1.y < maxY1) /
       {
           float relX1 = uv1.x - centre_x1; //x distance of the pixel from the center
           float relY1 = uv1.y - centre_y1; //y dist from the center
           float ang1 =  atan(relY1, relX1); //get angle with arc tangent
           float dist1 = sqrt(relX1*relX1 + relY1*relY1); // hypotenuse

           //gl_FragColor.g = 1.0; //DELETE ---- I used these two in each check for each center to see if it's running this code ---- it doesn't seem to, so the problem could be here or slightly below, I think, but I can't tell why
           //base_texture.g = 1.0; //DELETE

           if( dist1 <= circleRadius1 ) //a more accurate check if the current pixel is within the distortion field
           {   
           float newRad1 = dist1 * ( (0.6 * dist1/circleRadius1) + 0.4 );
           newX = centre_x1 + cos( ang1 ) * newRad1; //newX and newY will be used further down below to replace v_texcoord
           newX *= (resolution.y/resolution.x);
           newY = centre_y1 + sin( ang1 ) * newRad1;
           distInside1 = 1.0;

           //gl_FragColor.g = 1.0; //DELETE      ---- same here
           //base_texture.g = 1.0; //DELETE

           }

       pointInside1 = 1.0; //gl_Frag_Color's v_texcoord will be altered (further down)
       }



       //repeat second version of the code above for the second distortion field
       base_texture = texture2D(gm_BaseTexture, v_texcoord);

       vec2 uv2 = v_texcoord;
       uv2.x *= (resolution.x/resolution.y);
       float centre_x2 = (point_pos2.x / resolution.x) * (resolution.x/resolution.y);
       float centre_y2 = point_pos2.y / resolution.y;

       float circleRadius2 = circleRadiusBase2 * multiplier2;
       float maxX2 = centre_x2 + circleRadius2;
       float minX2 = centre_x2 - circleRadius2;
       float maxY2 = centre_y2 + circleRadius2;
       float minY2 = centre_y2 - circleRadius2;

       if( uv2.x > minX2 && uv2.x < maxX2 && uv2.y > minY2 && uv2.y < maxY2)
       {
           float relX2 = uv2.x - centre_x2;
           float relY2 = uv2.y - centre_y2;
           float ang2 =  atan(relY2, relX2);
           float dist2 = sqrt(relX2*relX2 + relY2*relY2);

           if( dist2 <= circleRadius2 )
           {   
               float newRad2 = dist2 * ( (0.6 * dist2/circleRadius2) + 0.4 );
               newX = centre_x2 + cos( ang2 ) * newRad2;
               newX *= (resolution.y/resolution.x);
               newY = centre_y2 + sin( ang2 ) * newRad2;
               distInside2 = 1.0;

           }

       pointInside2 = 1.0;
       }






       //repeat second version of the code above for the third distortion field
       base_texture = texture2D(gm_BaseTexture, v_texcoord);

       vec2 uv3 = v_texcoord;
       uv3.x *= (resolution.x/resolution.y);
       float centre_x3 = (point_pos3.x / resolution.x) * (resolution.x/resolution.y);
       float centre_y3 = point_pos3.y / resolution.y;

       float circleRadius3 = circleRadiusBase3 * multiplier3;
       float maxX3 = centre_x3 + circleRadius3;
       float minX3 = centre_x3 - circleRadius3;
       float maxY3 = centre_y3 + circleRadius3;
       float minY3 = centre_y3 - circleRadius3;

       if( uv3.x > minX3 && uv3.x < maxX3 && uv3.y > minY3 && uv3.y < maxY3)
       {
           float relX3 = uv3.x - centre_x3;
           float relY3 = uv3.y - centre_y3;
           float ang3 =  atan(relY3, relX3);
           float dist3 = sqrt(relX3*relX3 + relY3*relY3);

           if( dist3 <= circleRadius3 )
           {   
               float newRad3 = dist3 * ( (0.6 * dist3/circleRadius3) + 0.4 );
               newX = centre_x3 + cos( ang3 ) * newRad3;
               newX *= (resolution.y/resolution.x);
               newY = centre_y3 + sin( ang3 ) * newRad3;
               distInside3 = 1.0;

           }

       pointInside3 = 1.0;
       }





   if ( (pointInside1 == 0.0) && (pointInside2 == 0.0) && (pointInside3 == 0.0) ) {
       gl_FragColor = base_texture;
       } 


   if ( (distInside1 == 1.0) || (distInside2 == 1.0) || (distInside3 == 1.0) ) {
       gl_FragColor = texture2D(gm_BaseTexture, vec2(newX, newY) );   
       } else {
       gl_FragColor = base_texture;
       }




         //this is the highlighting bit of code, it only acts on the grayscale surface and this is the only part of the shader that seems to work as intended         
        //I use this because the hilights get muddled when I blend the textures with other colors

          //Hilight
       //##############################
       vec4 tempCol = gl_FragColor;
       if ( tempCol.r > 200.0/255.0 && tempCol.g > 200.0/255.0 && tempCol.b > 200.0/255.0 ) {  //runs on pixels with high channel values only
           gl_FragColor = tempCol;
           } else {
               if ( (tempCol.r == tempCol.g) && (tempCol.r == tempCol.b) && (tempCol.b == tempCol.r) ) { //hides other gray pixels
                   tempCol.a = 0.0;
                   gl_FragColor = tempCol;
                   }
           }
   }




this is a script ran in the create event of the draw object, it initializes the arrays that store coordinates, sprites, surfaces etc and looped in the draw ev:

Code:
{
var _n, _letters, _digits, _barpos, _numstring;
switch room {
   case room1:
       w = 960; h = 540;
       //DEFORMATION COORDINATES
       pekLeftx[1] = 152; pekLefty[1] = 272;   pekRightx[1] = 130; pekRighty[1] = 268;
       biceLeftx[1] = 176; biceLefty[1] = 299;   biceRightx[1] = 153; biceRighty[1] = 322;
       waistx[1] = 152; waisty[1] = 295;
       pekLeftx[2] = 417; pekLefty[2] = 320;   pekRightx[2] = 9;   pekRighty[2] = 9;
       biceLeftx[2] = 500; biceLefty[2] = 360;   biceRightx[2] = 480;  biceRighty[2] = 366;
       waistx[2] = 450; waisty[2] = 351; //moved it from 450, 351
       pekLeftx[3] = 620; pekLefty[3] = 286;   pekRightx[3] = 599; pekRighty[3] = 284;
       biceLeftx[3] = 615; biceLefty[3] = 337;   biceRightx[3] = 583; biceRighty[3] = 340;
       waistx[3] = 600; waisty[3] = 315;
       //RADIUS SIZE IN UV
       _n = 1;
       radPekLeft[_n] = 0.05; radPekRight[_n] = 0.055;                   
       radBiceLeft[_n] = 0.05; radBiceRight[_n] = 0.055; radWaist[_n] = 0.06;
       _n = 2;
       radPekLeft[_n] = 0.04; radPekRight[_n] = 0.001;                   
       radBiceLeft[_n] = 0.04; radBiceRight[_n] = 0.05; radWaist[_n] = 0.1;   
       _n = 3;
       radPekLeft[_n] = 0.06; radPekRight[_n] = 0.06;                   
       radBiceLeft[_n] = 0.03; radBiceRight[_n] = 0.04; radWaist[_n] = 0.045;   


       //THESE ARE RANDOMIZED CUSTOM SPRITES OR PARTS OF THE STRINGS OF THE SPRITE INDEX
       for(iii = 1; iii < 4; iii ++) {

           surf1[iii] = surface_create(w, h);
           surf2[iii] = surface_create(w, h);
           surf1Gray[iii] = surface_create(w, h);
           surf2Gray[iii] = surface_create(w, h);

           itemtype[iii, 0] = top[iii];  itemtype[iii, 1] = bottom[iii];  itemtype[iii, 2] = feet[iii];
           itemtype[iii, 3] = legs[iii];  itemtype[iii, 4] = wrist[iii];  itemtype[iii, 5] = neck[iii];

           bodySprite[iii, 0] = sprBody[iii]; bodySprite[iii, 1] = sprPekLeft[iii]; bodySprite[iii, 2] = sprPekRight[iii];
           bodySprite[iii, 3] = sprHairBG[iii]; bodySprite[iii, 4] = sprHairFG[iii];
           //provisory, considering getting rid of the type sprite affix eventually
           type[iii, 0] = "top"; type[iii, 1] = "bottom"; type[iii, 2] = "feet"; type[iii, 3] = "legs"; type[iii, 4] = "wrist"; type[iii, 5] = "neck";


           }

       uni_resolution = shader_get_uniform(shd_magnify3, "resolution1");

       uni_point_pos1 = shader_get_uniform(shd_magnify3, "point_pos1"); //deform coordiantes
       uni_circleRadiusBase1 = shader_get_uniform(shd_magnify3, "circleRadiusBase1");
       uni_multiplier1 = shader_get_uniform(shd_magnify3, "multiplier1"); //radius values are the same for all types of characters in this room, so I need a multiplier to vary it depending on what properties the character has been randomized with

       uni_point_pos2 = shader_get_uniform(shd_magnify3, "point_pos2");
       uni_circleRadiusBase2 = shader_get_uniform(shd_magnify3, "circleRadiusBase2");
       uni_multiplier2 = shader_get_uniform(shd_magnify3, "multiplier2");

       uni_point_pos3 = shader_get_uniform(shd_magnify3, "point_pos3");
       uni_circleRadiusBase3 = shader_get_uniform(shd_magnify2, "circleRadiusBase3");
       uni_multiplier3 = shader_get_uniform(shd_magnify3, "multiplier3");
       break;

   }

}


The draw event. I draw up to 3 characters, each on a pair of surfaces. Each pair of surfaces is in turn made from several sprites, which I'm doing to save on shader calls -- I tried this with individual surfaces for each sprite in a separate shader call each, and it tanked my framerate 30 fold.

Code:
  var iii, jjj, _r, _g, _b, _itemtype, _sprite, _spritestring;     

   if drawonsurf {


    for(iii = 1; iii < 4; iii++) {     //there are up to 3 characters that need to be drawn on the screen


           //CLEAR ALL IN ONE GO
           surface_set_target(surf1[iii]);
               draw_clear_alpha(c_black, 0);
           surface_reset_target();
           surface_set_target(surf1Gray[iii]);
               draw_clear_alpha(c_black, 0);
           surface_reset_target();
           surface_set_target(surf2[iii]);
               draw_clear_alpha(c_black, 0);
           surface_reset_target();
           surface_set_target(surf2Gray[iii]);
               draw_clear_alpha(c_black, 0);
           surface_reset_target();


           if ( name[iii] != noone ) { //there is a chance the character is not generated, checks for valid name
           //upper
           surface_set_target(surf1[iii]);
               for(jjj = 0; jjj < 5; jjj++) {     
                   if ( (jjj != 1) && (jjj != 2) && sprite_exists(bodySprite[iii, jjj]) ) {
                       draw_sprite_ext(bodySprite[iii, jjj], 0, 480, 270, 1, 1, 0, c_orange, 1); //These are test blend colors
                       }
                   }

               for(jjj = 0; jjj < 6; jjj ++) {
                   if ( itemtype[iii, jjj] != noone ) {
                       _itemtype = string_letters(itemtype[iii, jjj]);
                       _spritestring = "spr_" + type[iii, jjj] + "_" + _itemtype + "_room1_idle" + string(iii) + "_r_";
                       _sprite = asset_get_index(_spritestring); //debug
                       draw_sprite_ext(_sprite, 0, 480, 270, 1, 1, 0, c_green, 1);
                       }
                   }
           surface_reset_target();
           //rest of body
           surface_set_target(surf2[iii]);
               if ( sprite_exists(bodySprite[iii, 1]) && sprite_exists(bodySprite[iii, 2]) ) {
                   draw_sprite_ext(bodySprite[iii, 1], 0, 480, 270, 1, 1, 0, c_yellow, 1);
                   draw_sprite_ext(bodySprite[iii, 2], 0, 480, 270, 1, 1, 0, c_yellow, 1);
                   }
               if ( itemtype[iii, 0] != noone ) {
                   _itemtype = string_letters(itemtype[iii, 0]);
                   _spritestring = "spr_pekleft_" + _itemtype + ntype[iii] + "Left_room1_idle" + string(iii) + "_r_";
                   _sprite = asset_get_index(_spritestring); //debug
                   draw_sprite_ext(_sprite, 0, 480, 270, 1, 1, 0, c_blue, 1);
                   _spritestring = "spr_pekright_" + _itemtype + ntype[iii] + "Right_room1_idle" + string(iii) + "_r_";
                   _sprite = asset_get_index(_spritestring);
                   draw_sprite_ext(_sprite, 0, 480, 270, 1, 1, 0, c_blue, 1);
                   }
           surface_reset_target();


           //##################################################
           // GRAY SCALED SURFACES THAT HOLD CORRECT HGHILIGHT COLORS
           //#######################################################


           //upper
           surface_set_target(surf1Gray[iii]);
               for(jjj = 0; jjj < 5; jjj ++) {     
                   if ( (jjj != 1) && (jjj != 2) && sprite_exists(bodySprite[iii, jjj]) ) {
                       draw_sprite(bodySprite[iii, jjj], 0, 480, 270);
                       }
                   }

               for(jjj = 0; jjj < 6; jjj ++) {
                   if ( itemtype[iii, jjj] != noone ) {
                       _itemtype = string_letters(itemtype[iii, jjj]);
                       _spritestring = "spr_" + type[iii, jjj] + "_" + _itemtype + "_room1_idle" + string(iii) + "_r_";
                       _sprite = asset_get_index(_spritestring); //debug
                       draw_sprite(_sprite, 0, 480, 270);
                       }
                   }
           surface_reset_target();

           //rest
           surface_set_target(surf2Gray[iii]);
               if ( sprite_exists(bodySprite[iii, 1]) && sprite_exists(bodySprite[iii, 2]) ) {
                   draw_sprite(bodySprite[iii, 1], 0, 480, 270);
                   draw_sprite(bodySprite[iii, 2], 0, 480, 270);
                   }
               if ( itemtype[iii, 0] != noone ) {
                   _itemtype = string_letters(itemtype[iii, 0]);
                   _spritestring = "spr_pekleft_" + _itemtype + ntype[iii] + "Left_room1_idle" + string(iii) + "_r_";
                   _sprite = asset_get_index(_spritestring); //debug
                   draw_sprite(_sprite, 0, 480, 270);
                   _spritestring = "spr_pektright_" + _itemtype + ntype[iii] + "Right_room1_idle" + string(iii) + "_r_";
                   _sprite = asset_get_index(_spritestring);
                   draw_sprite(_sprite, 0, 480, 270);
                   }
           surface_reset_target();



               }
           }
           drawonsurf = false; //don't need surface redrawing every step

       }





   for(iii = 1; iii < 4; iii++) { //for each character


       shader_set(shd_magnify3);
           shader_set_uniform_f(uni_point_pos1, biceLeftx[iii], biceLefty[iii]); //3 distortion region centers
           shader_set_uniform_f(uni_point_pos2, biceRightx[iii], biceRighty[iii]);
           shader_set_uniform_f(uni_point_pos3, waistx[iii], waisty[iii]);
           shader_set_uniform_f(uni_resolution, 960, 540);
           shader_set_uniform_f(uni_circleRadiusBase1, radbiceLeft[iii]); //base distortion radius in current room
           shader_set_uniform_f(uni_circleRadiusBase2, radbiceRight[iii]);
           shader_set_uniform_f(uni_circleRadiusBase3, radWaist[iii]);
           shader_set_uniform_f(uni_multiplier1, bice[iii]); //from a script that produces the random values, I don't want all distortion regions to be equal for every character that is generated at one of the 3 positions
           shader_set_uniform_f(uni_multiplier2, bice[iii]);
           shader_set_uniform_f(uni_multiplier3, waist[iii]);

           draw_surface(surf1[iii], 0, 0);
           draw_surface(surf1[iii], 0, 0);
       shader_reset();

       shader_set(shd_magnify3);
           shader_set_uniform_f(uni_point_pos1, pekLeftx[iii], pekLefty[iii]);
           shader_set_uniform_f(uni_point_pos2, pekRightx[iii], pekRighty[iii]);
           shader_set_uniform_f(uni_point_pos3, 9, 9);  //doesn't need this uni in this call, throwaway value
           shader_set_uniform_f(uni_resolution, 960, 540);
           shader_set_uniform_f(uni_circleRadiusBase1, radPekLeft[iii]);
           shader_set_uniform_f(uni_circleRadiusBase2, radPekRight[iii]);
           shader_set_uniform_f(uni_circleRadiusBase3, 0.001); //doesn't need this uni in this call
           shader_set_uniform_f(uni_multiplier1, pek[iii]);
           shader_set_uniform_f(uni_multiplier2, pek[iii]);
           shader_set_uniform_f(uni_multiplier3, 1.0);

           draw_surface(surf2[iii], 0, 0);
           draw_surface(surf2[iii], 0, 0);
       shader_reset();

   }


Whatever you think is off here, anything at all, is something I'd appreciate hearing. Personally, I ran out of clues on this one.
 
B

Buzz Killdrin

Guest
I updated the code a bit by taking this part from the fragment shader at the ' if( dist1,2,3 <= circleRadius1,2,3 )' parts

Code:
newX *= (resolution.y/resolution.x);
and moved it way below, just before this part
Code:
gl_FragColor = texture2D(gm_BaseTexture, vec2(newX, newY) );
because the deformation radii can overlap and so newX can be offset multiple times
Still no dice though, no magnifying effect.
 
Top