Shaders [SOLVED] Help with NES-like shader

Discussion in 'Programming' started by NeXy, Jan 10, 2019.

  1. NeXy

    NeXy Member

    Joined:
    Oct 8, 2016
    Posts:
    14
    Hi,

    I'm trying to achieve a "true" limited palette effect like it was on the NES devices.
    Notice in the example below how even though the screen is fading, only the colors corresponding to a predefined palette are being drawn:
    [​IMG]
    I would like to achieve the same effect for my game. I'm using Dawnbringer's 16 Color Palette
    [​IMG]
    for all my sprites, but if I draw anything with any opacity through code - whether it's lights, or for example when Player is transitioning between rooms I would draw a black rectangle across the screen which would "fade-in/out" as the transition is happening. As the black rectangle is changing it's opacity, all the game sprites drawn beneath it will blend with the black color at it's current alpha value, and thus produce color that is outside of the range of the above mentioned limited palette.

    Although I have rather limited knowledge when it comes to shaders, through some research I've deducted I would definitely need to use one to achieve the desired effect. Namely, a shader that I can pass a limited palette through which will then, depending on values of colors on-screen, round each one to the nearest color on the given palette.

    I've found this: GameBoyShader on Game Maker marketplace, and was able, with my limited knowledge, to modify it so it accepts more than 4 colors. The end result was still not as desired, as the way the shader is built will round to colors "linearly", meaning we can only round to color from darkest to lightest and not have for example - orange go to yellow then white, while another orange sprite goes to red instead as the blended color over that sprite was closest to the red from the palette at that moment. I probably can't post the code as this is a paid asset, but if I'm wrong maybe someone who has more experience using it can correct me.

    Anyways, I hope my explanation was sufficient, this really was the bane of the progress of my game for quite some time, as I had no luck finding a shader that does this, but if there is one in existence it would be a godsend if you can provide me with a link. Otherwise, any form of help, guidance, instructions or links to tutorials would be greatly appreciated.

    EDIT: I'm using GMS2 if it's relevant.

    -Thanks in advance!
     
    Last edited: Jan 10, 2019
  2. RichHopelessComposer

    RichHopelessComposer Member

    Joined:
    Jun 20, 2016
    Posts:
    1,303
    I think @Carnivius might know something about this. He's a master of retro purism!
     
    NeXy likes this.
  3. Danei

    Danei Member

    Joined:
    Mar 23, 2018
    Posts:
    117
    What you're describing sounds possible with shaders, and maybe someone can help you with it, but in case you don't get it to work, an easier alternative to achieve that effect is to use a more simple palette swapping shader to transition between colors which you have previously set, rather than dynamically rounding them based on some blend. As far as I know, the NES wasn't rounding colors and the screen in CV2 isn't fading out per se, it's drawing the background with different colors to create the appearance of fading out, and that's what I'm describing.
     
    NeXy likes this.
  4. TheouAegis

    TheouAegis Member

    Joined:
    Jul 3, 2016
    Posts:
    5,922
    "Fading" in NES games is just reducing the palette entries by $10. For example, if a sprite had a palette of $3F162532 and you wanted to fade out, you'd subtract $10 from each value up to four times like so (ignore the $3F since that's the transparency):

    Step 1: $061522
    Step 2: $000512
    Step 3: $000002
    Step 4: $000000

    Code:
    if fade != 0 && !(global.timer & 3){
       for(var i=0; i<32; i++)
          palette[i] = max(palette[i]-$10,0);
       fade--;
    }
    If you needed to fade back in, you could store the default palette values, make fade (in the sample above) a range of -4 to +4, and set each palette entry to its default value+$10*fade and adjust fade toward 0.
    Code:
    if fade != 0 && !(global.timer & 3) {
       for(var i=0; i<32; i++)
          palette[i] = max(default_palette[i]+$10*fade,0);
       fade -= sign(fade);
    }
    In NES games you could also simulate nighttime by doing a 1- or 2-step fade and then reducing the red and green color channels. You could simulate scorching sunlight by doing a 1-step reverse fade and reducing the blue channel.
     
    Last edited: Jan 11, 2019
    Zabicka, NeXy and kumitassu like this.
  5. NeXy

    NeXy Member

    Joined:
    Oct 8, 2016
    Posts:
    14
    Thanks! If this thread doesn't provide any substantial solution I will make sure to contact him.

    This does sound like a rougher alternative. Do you perhaps have any code examples to show-off?


    Does this code go into the shader or...? I didn't know that you can manipulate color values in such a way.
    Though, while it might solve the issue of "fading" the screen - the fading was just an example. I might be wrong but this doesn't seem applicable to general use, such as adding spotlights to portions of the screen, highlighting certain sprites...etc.
     
    RichHopelessComposer likes this.
  6. Danei

    Danei Member

    Joined:
    Mar 23, 2018
    Posts:
    117
    Unfortunately not because the palette swapping shader I personally use is a paid asset by Pixelated Pope. This one specifically: https://marketplace.yoyogames.com/assets/1661/retro-palette-swapper. It's one of the featured assets on the marketplace, has worked great for me, and is extremely easy to use, so I recommend it if you have four bucks to burn. But if not, I have seen youTube tutorials for comparable shader-based palette swapping, and hopefully they are similarly effective.

    The basic idea behind these shaders is you set up a series of color sets where each color in each set corresponds to a color in the original sprite, and then the shader simply displays the colors from your chosen color set instead of the original colors, and then by changing which set you're using during runtime, you change the colors in-game.
     
  7. TheouAegis

    TheouAegis Member

    Joined:
    Jul 3, 2016
    Posts:
    5,922
    Assuming you were going for an NES aesthetic, you'd need to store all your color values in an array -- not the palettes, but the set of all possible colors -- organized the same way as the NES, where the first 16 colors are the darkest, then the nexylt 16 are lighter, then another 16, then the brightest. You'd define each palette as the indexes of that array of available colors. Adding or subtracting 16 from each palette entry would be the same result.
     
  8. NeXy

    NeXy Member

    Joined:
    Oct 8, 2016
    Posts:
    14
    Thank you both for suggestions to alternative approaches to this problem. However, I may be wrong but I still see these two as solutions that will only address the "fading transition" issue, but would prove little use when it comes to general use of limiting colors such as having spotlights in the darker levels or highlighting certain sprites...etc.

    I would still need a more general-purpose approach to limit all colors drawn on screen to specified palette.
     
  9. TheouAegis

    TheouAegis Member

    Joined:
    Jul 3, 2016
    Posts:
    5,922
    Limiting colors to a specified palette is opart of my last post. You store all possible colors in an array and only use colors in that array.

    Fake NES spotlights would be fine (i.e., impossible for the NES to do but still look acceptable enough). Have the shader fetch each spotted pixel's color, find it in the array, and add 16 to its index then draw that color. If it is greater than 63, draw color 48 (white).
     
    NeXy likes this.
  10. NeXy

    NeXy Member

    Joined:
    Oct 8, 2016
    Posts:
    14
    Thanks TheouAegis, and sorry for not responding sooner.

    So I think I'm getting the idea of what you are going for - though if I understood correctly, the idea behind adding or removing 16 from/to the index so we switch to another color is based in having the colors we are fetching from the screen already stored in an array, which is nigh impossible in the scenario I'm presenting. I may have not made it clear enough, but I will draw sprites with opacity across the screen at multiple points in the game. From black fading rectangles that will be used for screen transitions, to white transparent circles for different spotlights, to sprite shadows...etc. So as opacity of these elements would be mostly dynamic and change across multiple game-frames, it would be difficult to predict what color of the pixel where these elements are drawn would be so as to have a fixed 16-color array that we can check against and then swap the color with another one from a different array.

    I would need a more general approach where perhaps I would check the color of the pixel on screen, run it trough some math that would round it's rgb values to the rgb values nearest to the closest color from a palette given above, or something along those lines, but am stumped on the best way to approach this.

    Kind regards,
     
  11. Danei

    Danei Member

    Joined:
    Mar 23, 2018
    Posts:
    117
  12. NeXy

    NeXy Member

    Joined:
    Oct 8, 2016
    Posts:
    14
    Thanks again Danei, I checked out the math, but still have trouble applying it to a proper shader. Mostly because I'm very thin on the know-how of this area.
    Anyways, I'll continue studying up on opengl and trying to come-up with a solution.

    If in the meantime, someone would like to chip-in with any help feel free to do so. Oh, and also I found another close example of what I'm attempting to do.
    Check out the lights from Kunai game - (notice how no matter how the levels/sprites are illuminated they are always rendered in 4 specific colors) basically this but with 16 colored sprites:
     
  13. Chaser

    Chaser Member

    Joined:
    Jul 29, 2016
    Posts:
    100
    Why Use a shader? Can you not 'get' what colours are in tiles from code and have them 'removed' independently at intervals via an alarm, or appear in the same way. Interesting post this one, as I'm working on some NES ideas myself.:) hope you get what you need soon.
     
  14. NeXy

    NeXy Member

    Joined:
    Oct 8, 2016
    Posts:
    14
    Hi Chaser,
    I figured the shader is best approach because I need colors to be "rounded" to the ones in a predefined palette at a runtime while objects with transparency / blending for shadows or lights, and/or sprites with alpha are drawn - which would result in a wild destination color. It would be up to a shader to reduce it to the nearest color in the palette list.

    I was thinking of drawing the entire game on a surface, then pass that surface through a shader that does what I described above and viola! -All "wild" colors on screen get rounded to the closest color on a pre-defined palette, leaving me with an authentic "retro" look, while still allowing me to go nuts with blending/opacity/light systems...etc.

    The problem for me is building the shader that does this.
     
  15. Chaser

    Chaser Member

    Joined:
    Jul 29, 2016
    Posts:
    100
    I see, sounds like complicated stuff, I would just draw the sprites tiles and have the tiles animate the effect of a transistion if it was me ;) Not sure you could actually change the 'values' on a NES console with its PPU or attribution table,I'm probably wrong, perhaps Sega's VDP might of done? I don't know, not the best or quickest way I know, but it may be quicker waiting for a shader, and at least you have control of the colour palletes and still keep the retro vibe.:)
     
  16. Danei

    Danei Member

    Joined:
    Mar 23, 2018
    Posts:
    117
    Sorry to continue harping on this, but I bet that Kunai game is not doing this the way you're attempting to (or even using transparent sprites at all, it appears), but actually just doing it with regular palette swapping. In this case it looks like it's swapping, e.g., the lighter blue to tan, the tan to yellow, the yellow to white, and leaving that darker blue alone. It's applying that in a circle around entities and lights, and along the ground at points of contact with the player/lights. And then it's doing a smaller inner circle that turns more things to lighter colors. I could be wrong, I don't know how they did it. But I bet $5, because it'd be so much easier and more efficient that way.
     
    NeXy likes this.
  17. TheouAegis

    TheouAegis Member

    Joined:
    Jul 3, 2016
    Posts:
    5,922
    It sounds like he insists on using a rectangle fade, which is kind of limiting his options.

    You can change palettes whenever you want prior to the scanline being rendered. Although I'm not sure about the attributes of the tiles themselves, I haven't tried messing with those. I just know the palette itself in the PPU is easy to change with a simple write to the PPU.
     
    NeXy likes this.
  18. Bingdom

    Bingdom Googledom

    Joined:
    Jul 1, 2016
    Posts:
    1,655
    Do the fade as normal with the black rectangle overlaying everything, and any other special effects.

    Apply this shader to the application surface:
    • Use uniform array storing all palette colours.
    • Use 3D Pythagorean Theorem to find the difference between the application colour and palette by using a loop to iterate through the palette array.
    • Find the one with the shortest distance (difference) and use that as the final pixel.
    This shader would be pretty intensive, so it may not apply well to low-end gpus.

    Edit: It appears I may have misread the question.
    I suppose you could further expand my implementation using a threshold.
     
    Last edited: Jan 18, 2019 at 12:04 AM
    NeXy and dannyjenn like this.
  19. Chaser

    Chaser Member

    Joined:
    Jul 29, 2016
    Posts:
    100
    Yes your right, you can change the pallet on the fly no problem as you say, what I'm saying is you can't lighten a colour or darken a colour based on hues levels and what not as it didn't have the capabilitys for that(I don't think), what colours the NES has it has, so any colour flashes fancy visuals were nothing more then flipping between colour palettes and banks of attribution data.
     
    NeXy likes this.
  20. TheouAegis

    TheouAegis Member

    Joined:
    Jul 3, 2016
    Posts:
    5,922
    Subtracting 16 from the color is darkening the hue, adding 16 is lightening it. But you would affect the palette entry itself, so all tiles or sprites using that palette would be affected. So it only really works for a global fade/brighten or per scanline on a scanline wipe. It wouldn't work for spotlights or torches. Enemies that flash red or dazzling iFrames were usually palette index swaps (e.g. myPal=++myPal & 3, or myPal^=1). But actual screen fades were full palette color edits. You could also tint all palettes (RGBCMY, I think) per scanline, but that was a FULL palette modification, so it affected everyone. Good for water effects like fish being blue in water and white out of water).

    That's why I said he could make his game a false NES game or whatever like Mega Man games or Bloodstained. lol
     
    NeXy likes this.
  21. dannyjenn

    dannyjenn Member

    Joined:
    Jul 29, 2017
    Posts:
    473
    I don't know whether this would reasonably fit within the limits of the NES, but here's one way you could fade a single sprite in and out: (this is kind of like how you'd do it on the Game Boy...)


    Setting things up:

    - You'll need to design your sprites such that they each have no more than 4 colors (transparency counts as a color, so really no more than 3 colors). The 4 colors you use are arbitrary, not the actual colors that you want the sprite to show up as. But you need to be consistent from sprite to sprite: all sprites should have the same exact 4 colors. What I did in my game was I used pure red, pure blue, pure green, and pure white. The spites look ugly but that doesn't matter, since nobody's going to see them with these colors.
    ts_player.png
    (The reason I used blue, green, red, and white as opposed to four shades of gray is that the gray was giving me problems when I went to write my shader. Not sure why. I think it may have had something to do with casting from int to float or whatever.)

    - Then you'll need to make an array containing the hex values from Dawnbringer's 16 Color Palette.
    Code:
    // game start event:
    global.palette[0] = $1C0C14; // black
    global.palette[1] = $342444; // dark purple
    global.palette[2] = $6D3430; // royal blue
    global.palette[3] = $4E4A4E; // dark gray
    // .
    // .
    // .
    global.palette[13] = $CAC26D; // sky blue
    global.palette[14] = $5ED4DA; // yellow
    global.palette[15] = $D6EEDE; // pale
    - You also need to give each of your instances its own array containing the index numbers of its sprite's three colors, from the 16-color palette.

    e.g. If I wanted my player sprite to be black, sky blue, and pale, I'd do something like this:
    Code:
    // create event:
    color[0] = 0; // because global.palette[0] is black
    color[1] = 13; // because global.palette[13] is sky blue
    color[2] = 15; // because global.palette[15] is pale
    - You'll then need to write a palette-swap shader. And this shader should take (as input) global.palette[color[0]], global.palette[color[1]], and global.palette[color[2]].

    That shader should then change all the pure blue pixels in the original sprite to global.palette[color[0]], all the pure green pixels to global.palette[color[1]], all the pure red to global.palette[color[2]], and it should draw all the pure white to have an alpha of 0.


    The fading:

    - You'll first you need to figure out which colors in the 16-color palette should become which other colors in the 16-color palette. I'd go with maybe three fading levels. The easiest way to do this is probably just to do it by hand in GIMP or something. Here's what I came up with:

    fade.png

    So this can be implemented in various ways (a ds_grid might be easiest). But regardless of how you do it, the basic idea is that it's a function: get_new_color(original_color,fade_level) should output the new color (a value 0 to 15, corresponding to that color's index number in the 16-color palette).

    So I would start by keeping track of the original colors and the fade level:
    Code:
    // create event:
    // .
    // .
    // .
    original_color[0] = color[0];
    original_color[1] = color[1];
    original_color[2] = color[2];
    fade_level = 0;
    That way you can set up an alarm or something, and do
    Code:
    // alarm[0] event:
    if(fade_level<3){
        fade_level += 1;
        color[0] = get_new_color(original_color[0],fade_level);
        color[1] = get_new_color(original_color[1],fade_level);
        color[2] = get_new_color(original_color[2],fade_level);
        alarm[0] = 30;
    }
    And since the shader is taking global.palette[color[whatever]] as input, the sprite should automatically change whenever you change color[whatever], as is the case during the fade out.

    player_fade.png
     
    Last edited: Jan 18, 2019 at 2:28 PM
    NeXy likes this.
  22. NeXy

    NeXy Member

    Joined:
    Oct 8, 2016
    Posts:
    14
    EDIT: Thanks everyone for your help but I managed to solve the problem, so just in case anyone stumbles on this thread looking for a solution I'll post it here.

    What I was looking for is actually called LUT. There is already a shader that does this on Game Maker marketplace found here:

    https://marketplace.yoyogames.com/assets/4505/lut-color-correction

    The trick is to use an "s_lut" texture that comes pre-packaged with this shader and edit it so it only contains the desired colors, then do a very minor edit to the shader's code so it doesn't pass the alpha value to the final color of the pixel - apply this shader to the application surface and you will end up with only exactly defined palette colors on screen at all times, no other color will be shown no matter the usage of sprites with any opacity, blending-modes...etc.

    GIF Proof:
    Example.gif
     
    Last edited: Jan 20, 2019 at 9:08 AM

Share This Page

  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice