L

Misu is talking about smoothing of the normals.What? You have normals or you don't have normals, that's it. I'm not sure i understand what you are asking.

What is your goal here? Are you talking about smooth (aka fragment based) lighting versus flat (aka vertex based) lighting?

For example, if you have a cube model and want to make it have rounded looking edges, the normal direction needs to be averaged across each vertex. Rather than all pointing 90 degrees from each other.

I'm not that great with the math behind it though.

If the model is created in a 3D package like Maya or something, it will allow this to be done. That's how I go about it.

https://stackoverflow.com/questions/1966587/given-3-pts-how-do-i-calculate-the-normal-vector

2) Use that facet normal value for each point. Each will have this normal. that will result is a cheap ass lighting you are complaining about. do this for every facet of your model

3) loop through each point of the model, figure out which points share the same coordinate. add all their normals and average the result and change the normal of each point to that average

I have various lighting normals for models I wrote in a dll for gm8.... you can find the link in my tools page below in sig. the dll is called gmmodelext.dll and the app is called gmmodelfix. the tool is almost unusable now but it still works

simple variations on the method can have a cutoff angle looking at the dot product of 2 normals, to see if their angle is somewhat facing the same direction can allow keeping hard edges and smoothing softer ones

This explanation helpped. Thanks a ton.1) Calculate the normal of the triangle facet using the 3 points that form the triangle with cross product

https://stackoverflow.com/questions/1966587/given-3-pts-how-do-i-calculate-the-normal-vector

2) Use that facet normal value for each point. Each will have this normal. that will result is a cheap ass lighting you are complaining about. do this for every facet of your model

3) loop through each point of the model, figure out which points share the same coordinate. add all their normals and average the result and change the normal of each point to that average

It was a tough job to make this though but i got done finally.