How to achieve this effect?

[RocketMonster]

So I want to punch through one layer based on contents on another. Hopefully the attached image will demonstrate. Ideally I want it setup as follow; I have mountains on layer_1, a wall on layer_2 and layer_3 I want to have any pixel that is 255,0,255 to allow layer_1 to show through.

I cant pre-bake these images etc as the game play I have planned will all be random and this is just an example not actual game.

I thought maybe a layer_shader() (am I am new to shaders), so as a test I combined layer_2 and layer_3 ( making the 'wall' and the 'cutouts' into objects) and in the shader set the alpha of any pixel that is 255,0,255 to 0 but that just made the 'cutout' objects transparent and did not affect any of the 'wall' object pixels.

Thinking I might need to push the layer(s) onto a surface or texture and then run the shader? Can anyone point in the right direction?

 

[jo-thijs]

So I want to punch through one layer based on contents on another. Hopefully the attached image will demonstrate. Ideally I want it setup as follow; I have mountains on layer_1, a wall on layer_2 and layer_3 I want to have any pixel that is 255,0,255 to allow layer_1 to show through.

I cant pre-bake these images etc as the game play I have planned will all be random and this is just an example not actual game.

I thought maybe a layer_shader() (am I am new to shaders), so as a test I combined layer_2 and layer_3 ( making the 'wall' and the 'cutouts' into objects) and in the shader set the alpha of any pixel that is 255,0,255 to 0 but that just made the 'cutout' objects transparent and did not affect any of the 'wall' object pixels.

Thinking I might need to push the layer(s) onto a surface or texture and then run the shader? Can anyone point in the right direction?

You can use shaders for this, but you'll have to use texture_set_stage along with it.
This function allows you to pass an aditional texture besides the one you're "drawing" as parameter to shaders.
You would then draw the wall using a shader that has the magenta cutout sprite as parameter.
For every pixel of the wall that is being drawn, you would then check if the corresponding pixel in the magenta cutout sprite is opaque magenta.

An alternative approach would be to use surfaces in combination with blend modes.
You'd first create a surface to hold the "result" before applying it to the background and clear this surface to black with alpha 0.
You'd then use (draw/gpu)_set_blend_mode_ext(bm_one, bm_zero) and draw the wall to the surface.
Afterwards, you would use (draw/gpu)_set_blend_mode_ext(bm_zero, bm_src_alpha) and draw the magenta cutout to the surface.
This assumes that you only use alpha values of 0 and 1 in the cutout sprite and the non-magenta parts have an alpha value of 0.
You can then draw this surface on top of the background.

I didn't go in detail in my answer here,
so if you have any questions, feel free to ask them!