SOLVED I Need more explanation on how "Audio_listener_orientation" works, please

Evanski

Raccoon Lord
Forum Staff
Moderator
so using
audio_listener_orientation(lookat_x, lookat_y, lookat_z, up_x, up_y, up_z);
sets the listeners hearing, so you hear stuff on the right come from the right and left to the left,

I don't understand the values that I'm editing and the picture of a floating head doesn't really help me understand, not to mention the manual doesn't even talk about the "up" values
I understand Im setting a virtual position that a person would hear things in a 3D space, but I don't understand how to modify the position to where I'd want it.


there is a bug in gms that makes right sounds come from the left but that is fixed with
audio_listener_orientation(0, 1, 0, 0, 0, 1);
and that works making the sounds on the left come from the left and the sounds from the right come from the right,
Why does this change it so that the sounds come from the right direction?

another thing I can't figure out is how to get sounds to come from the front of the listener or from behind, If I place an emitter in front of the listener it comes from left and right at the same time, the same for when the audio is from behind.

TL;DR
Explain to me how exactly I'm modifying the position of the listener using the lookat and up values
Why does the values shown fix a bug with the audio being playing from the opposite side its on
How do I get sounds to not only play on the left and right, but forward and behind

I have read the manual, I just don't fully understand what the manual is trying to explain and wish for more insight from others who understand it better then I.

Thanks.
 

kburkhart84

Firehammer Games
there is a bug in gms that makes right sounds come from the left but that is fixed with
audio_listener_orientation(0, 1, 0, 0, 0, 1);
and that works making the sounds on the left come from the left and the sounds from the right come from the right,
Why does this change it so that the sounds come from the right direction?
Actually, I don't think that this is a bug...it's just that the default coord system is upside down(y is positive going down, that is actually kinda strange and not all that common).

Explain to me how exactly I'm modifying the position of the listener using the lookat and up values
The lookat and up vectors don't change the position. They change which way the listener is facing. If I'm facing "forward", a sound to my left comes in my left ear before my right. If I stay in the same place but turn left, then that same sound that was to my left is now forward from me and hits my ears at the same time. If I turn right, the sound is now behind me and so hits my ears at the same time, but a little more muffled.

The reason you also need the up vector to complete the definition is because I could be facing forward, but if I do a barrel roll and am upside down, the left and right sides are now switched. This isn't really a thing a your typical FPS because you don't flip over, but in some games you actually do.

So for full definition of the listener, you need a position in 3d space, a direction it is looking, and the direction that is "up." There are other ways to describe it, including things like euler angles, but the vectors are in some opinion an easy way to consider the thing.

Why does the values shown fix a bug with the audio being playing from the opposite side its on
I think I explained it above...it isn't actually a bug so much as the audio engine defaults to that direction since Y position is actually down instead of up.

How do I get sounds to not only play on the left and right, but forward and behind
This is all determined by the position of the listener and sound(either singly or though an emitter). If the Z position for both listener and sound is always 0(as is default), then you are stuck on that 2d plane(like a 2d platformer). If you want something in front or behind, you need to actually have a sound be offset from that 2d plane on the Z axis. That is what the 3rd number in all those vectors is in all the positions as you are using the 3d sound functions.

**********

I hope that helps you some. If you have more questions, go for it. If you have something more specific to your game project you want to describe, supply the information as well. Finally, I'm willing to give you a few minutes in a Discord voice chat if you think it would help you better(you gave my Jam entry some personal time so you deserve it).
 

Evanski

Raccoon Lord
Forum Staff
Moderator
The lookat and up vectors don't change the position. They change which way the listener is facing.
so if I understand correctly the look at values are position and the up values are direction, relative to where the sound is coming from?

This is all determined by the position of the listener and sound
So i would need and emitter to not only be playing a sound from an x and y but a Z, So how do I go about creating a Z value for the sound to come from?

I'm willing to give you a few minutes in a Discord voice chat if you think it would help you better
I probably wont need to go that far as this isn't an urgent issue just something I want to delve into as I've never messed around with a sound system before, i usually use fire and forget.
But I do absolutely appreciate this offer!
 

kburkhart84

Firehammer Games
so if I understand correctly the look at values are position and the up values are direction, relative to where the sound is coming from?
Imagine the listener is an actual person, somewhere in 3d space.

1. Position: where in 3d space the person is
2. LookAt: which way that person is facing
3. Up: which direction is 'up' based on the "roll" angle

So you are needing 3 actual vectors to define this thing. The lookat and up vectors alone don't define it, you have to have the position as well. This means that you have to use audio_listener_position() along with audio_listener_orientation() to handle it. For some reason Yoyo decided to just keep them as separate, likely because so often the listener in GM games moves without actually changing orientation. I'm guessing this separation is what is confusing you, since you really need two functions to fully define it by the time its done.

Also, note that when defining the listener position and orientation, you don't have to worry about the actual sounds at this time. Just imagine it is a person in 3d space. The sounds will then also be in 3d space somewhere. The system automatically will handle things like what sound is how far and in what direction from the listener. You only tell is where things are and the system handles the rest.

So i would need and emitter to not only be playing a sound from an x and y but a Z, So how do I go about creating a Z value for the sound to come from?
There are two ways to play 3d sounds, one with an emitter, and one without.

1. audio_play_sound_at() - this one is the one-off version. The function takes a position for the sound to be at in 3d space, including x, y, AND z. It also has other stuff for the falloff things that determine how far away the sound can be heard from which is a separate thing from actual volume. The whole falloff thing is another discussion, but imagine a loud sound compared to something like a flying bee. The sounds may be equal volume depending on where you are, but the sound from the bee will die off right away.

2. audio_play_sound_on() - this one uses an emitter. Instead of supplying position to the function, you have the emitter control the position. The emitter is kind of a separate entity that can actually move. If you have a sound on the emitter and you move the emitter, the sound position also moves, which is what you want for a sound that lasts more than a couple seconds, but isn't necessary on a sound that is quick enough to not notice that it moved. Another advantage of emitters is that they allow you to use velocity(which I'll explain below).

So, but simply changing the Z value in these functions, the sound will be moved forward or backward in 3d space, instead of only up, down, left, and right.

I probably wont need to go that far as this isn't an urgent issue just something I want to delve into as I've never messed around with a sound system before, i usually use fire and forget.
But I do absolutely appreciate this offer!
No problem. I always like to pay back favors even if they were given without the intention of the recipient owing the giver. Feel free to PM me if you ever want to take advantage of my offer. It doesn't really have to be about this exact topic either, just on something you happen to think I know about and could help.

Now, about "velocity." If you don't know what the doppler effect is, look it up. It is basically how a sound is different based on if the sound is moving away or towards the listener, and how fast that movement is. You can set the velocity of sounds and the listener and the system will automatically apply the doppler effect to it accordingly. If you really want to make your sound stuff immersive in a game, this can help.

*********

I'm actually still working on my audio system. It is just using the internal functions they already give us, but with adding things. For example, I have a single function that can set the listener position AND orientation all in one call(even if internally I still have to call the two functions). It will also do things like having the listener AND the sound emitters automatically follow instances around without you have to call any functions to do it once you set it up. Then it has the stuff I'm doing with music, like the cross-fading and swap-fading stuff. If you really are interested in getting better sound in your game, my system will be able to help. If you would like to get it for free instead of that Discord time I'm glad to offer(just let me finish it first, it is coming along).
 

Evanski

Raccoon Lord
Forum Staff
Moderator
wow space.png

so using this chart to understand the values better.

If I wanted to have 3D sound
the Audio orientation would be: look at X 0, Look at Y 1, Look at Z 1, | X Up 0, Y up 0, Z up 1

saying the room is a 100x100 space
and the listener is in the center of the room

the audio position would be: x 50, y 50, Z 0

am I correct?

Now, about "velocity." If you don't know what the doppler effect is
I am aware of the doppler effect, my thinking was to have the emitter moving and its velocity be the objects speed, and applying that with its position the audio system should create the doppler effect ( for example a car passing by)
 
Last edited:

kburkhart84

Firehammer Games
If I wanted to have 3D sound
the Audio orientation would be: look at X 0, Look at Y 1, Look at Z 1, | X Up 0, Y up 0, Z up 1
The issue here is that both Y and Z are 1, meaning you would get a 45 degree angle instead of facing straight down an axis. Assuming that The first two pictures is a top down thing, and that they are basically the same thing with different labels, then the lookat would be 1, 0, 0, because you are looking down the X axis. However, the UP axis you have at 0, 0, 1 would be CORRECT because Z is your up/down axis. As far as your audio position, it looks to me like it is at X99, Y 50, Z0, assuming that the point where the 3 axes meet in your drawing is 0, 0, 0. The listener in the exact middle of the cube would be 50, 50, 50, but if that listener is in the middle but on the ground instead of in the air, it is at 50, 50, 0, or maybe 50, 50, 10 if the person is 10 units tall.

Note that your actual axis definitions are a little off, because X is normally left/right, and yours is forward/back(if that pick is top-down). So if I rotated your coordinates to be what the more common definitions are, it would be like your character is facing left. In reality, coordinate systems are just a defined thing and there are all kinds of them, which is why people have issues with 3d models so much because the modeller uses one while the game engine uses another. The fact that Y+ is down on the screen is an example of something strange about a coordinate system.

I'm gonna finish ironing something, and I'll make you a couple pics in Blender to help you out. Or I could screenshare with you over Discord if you want to directly see.

I am aware of the doppler effect, my thinking was to have the emitter moving and its velocity be the objects speed, and applying that with its position the audio system should create the doppler effect ( for example a car passing by)
This part I think you have it. Just changing the emitter position doesn't actually give it velocity though, rather you have to use the actual function to tell the system what the velocity is, and the same applies to the listener. I coded a "multiplier" in my system for this as well because instances move in pixels but the audio world units may be bigger or smaller so it will automatically calculate the velocity when it is having things follow instances. But yes, just setting the velocity will make the sound have doppler all automatically.
 

Evanski

Raccoon Lord
Forum Staff
Moderator
I think the main thing im struggling is figuring out the Z axis and the correct look at points

I'm gonna finish ironing something, and I'll make you a couple pics in Blender to help you out.
These will help out a lot more then my ms paint drawings
 

kburkhart84

Firehammer Games
I think the main thing im struggling is figuring out the Z axis and the correct look at points
I actually think a lot of your issue is that you are thinking of the lookat and up vectors as points. These aren't actually points, they are vectors. These pictures will help explain.
NewMiddle.jpg
Let's say the monkey is in the exactly center of the world, 0, 0, 0. The monkey is facing "down" in GM's space, say let's say that is the positive Y direction. The up direction is the Z positive.
Code:
audio_listener_position(0, 0, 0);
audio_listener_orientation(0, 1, 0, 0, 0, 1);
*****Now look at this second picture.
NewOffset.jpg
You see I shifted the monkey to the right...let's say 5 units. It is now 5 units in the X positive direction.
Code:
audio_listener_position(5, 0, 0);
audio_listener_orientation(0, 1, 0, 0, 0, 1);
Notice how the code for the position changed, but the code for the orientation did NOT change at all. This is because the monkey is looking the exact same direction even though it moved. So the "point" it is looking at might be different, but the DIRECTION is not different.

Vectors can be things that define actual positions, but they can also define directions(with distance/magnitude). So the listener position is a position, while the lookat and up vectors are DIRECTIONS. I could put the monkey literally anywhere in 3d space, but as long as it didn't rotate at all and stays facing the same direction, the orientation would never change. The opposite is also true, I could face the monkey in any direction I wanted(which would change the orientation), but the position wouldn't change. In real life it is usually a combination of the two things, but they are actually separate concepts with each one changing independently.

Another way to think of it, in an FPS, if you strafe without moving the mouse, your lookat vector doesn't change, but if you move the mouse, it does.
 

Evanski

Raccoon Lord
Forum Staff
Moderator
I actually think a lot of your issue is that you are thinking of the lookat and up vectors as points. These aren't actually points, they are vectors. These pictures will help explain.
View attachment 39740
Let's say the monkey is in the exactly center of the world, 0, 0, 0. The monkey is facing "down" in GM's space, say let's say that is the positive Y direction. The up direction is the Z positive.
Code:
audio_listener_position(0, 0, 0);
audio_listener_orientation(0, 1, 0, 0, 0, 1);
*****Now look at this second picture.
View attachment 39741
You see I shifted the monkey to the right...let's say 5 units. It is now 5 units in the X positive direction.
Code:
audio_listener_position(5, 0, 0);
audio_listener_orientation(0, 1, 0, 0, 0, 1);
Notice how the code for the position changed, but the code for the orientation did NOT change at all. This is because the monkey is looking the exact same direction even though it moved. So the "point" it is looking at might be different, but the DIRECTION is not different.

Vectors can be things that define actual positions, but they can also define directions(with distance/magnitude). So the listener position is a position, while the lookat and up vectors are DIRECTIONS. I could put the monkey literally anywhere in 3d space, but as long as it didn't rotate at all and stays facing the same direction, the orientation would never change. The opposite is also true, I could face the monkey in any direction I wanted(which would change the orientation), but the position wouldn't change. In real life it is usually a combination of the two things, but they are actually separate concepts with each one changing independently.

Another way to think of it, in an FPS, if you strafe without moving the mouse, your lookat vector doesn't change, but if you move the mouse, it does.

okay so using that same logic, I want is the monkey to be in the center of the room,
if we are looking at a 2d screen of game maker
I want any audio being played at a Y that is above the monkeys Y to be sounding like its coming from in-front of the monkey
and any audio with a Y lower then the monkey to sound like its coming from behind the monkey
 

kburkhart84

Firehammer Games
okay so using that same logic, I want is the monkey to be in the center of the room,
if we are looking at a 2d screen of game maker
I want any audio being played at a Y that is above the monkeys Y to be sounding like its coming from in-front of the monkey
and any audio with a Y lower then the monkey to sound like its coming from behind the monkey
That is perfectly logical to me, you just need to orient the listener to face that way is all. The following code is for a 500x500 room, and the monkey is currently in the middle.
Code:
audio_listener_position(250, 250, 0);
audio_listener_orientation(0, 1, 0, 0, 0, 1);
Since your gain is constraining to a single plane(just the X/Y plane), you don't have to actually move anything in the Z direction. You just have to rotate the listener the right way. I put the monkey in the middle of the room and stick it on that X/Y plane. The lookat direction is 0, 1, 0, meaning the monkey is looking "down the screen" because y negative is up. Note that from this angle you would see the top of the monkey's head. The camera is basically a bird's eye top-down thing, so you just make the listener follow that. Finally, the up vector is 0, 0, 1, meaning as far as the monkey is concerned, the up direction is pointing the same way your eyes look, INTO the screen depth. So a higher Z value is for further away things(just like with the depth variable and layers). Depth and Z are basically the same thing once you get into 3d space.

Then, you just play sounds wherever on the X/Y plane they are, and they will not be around the monkey as if they are on the ground plane with the monkey instead of being over and under the monkey.

edit*** I fixed a couple things since the monkey is facing down, not up.
 

Evanski

Raccoon Lord
Forum Staff
Moderator
so the monkeys Z up is 1 so its standing at Z 0

So playing a audio with a Z of 1 would make the audio sound as if its coming from above
and a Z of -1 being from below
?
 

kburkhart84

Firehammer Games
so the monkeys Z up is 1 so its standing at Z 0

So playing a audio with a Z of 1 would make the audio sound as if its coming from above
and a Z of -1 being from below
?
DING DING DING!!!!! The catch is that X and Y would have to be pretty close to the same X and Y as the listerner, or it might be more to the side than to the top depending on the distances in each direction.

Also, check this code.
Code:
audio_listener_position(250, 250, 100);
audio_listener_orientation(0, 1, 0, 0, 0, 1);
If you see, I changed the listener position to be 100 in the Z position. So if you have a sound at Z of only 1, it would be 99 units below the listener and so sound like it is below it. If you make the sound at Z of 105, than it would be 5 units over the listener and so sound like it is above.
 

Evanski

Raccoon Lord
Forum Staff
Moderator
DING DING DING!!!!! The catch is that X and Y would have to be pretty close to the same X and Y as the listerner, or it might be more to the side than to the top depending on the distances in each direction.

Also, check this code.
Code:
audio_listener_position(250, 250, 100);
audio_listener_orientation(0, 1, 0, 0, 0, 1);
If you see, I changed the listener position to be 100 in the Z position. So if you have a sound at Z of only 1, it would be 99 units below the listener and so sound like it is below it. If you make the sound at Z of 105, than it would be 5 units over the listener and so sound like it is above.
Awesome I think I understand it a whole world better then I did before, just one thing I just realized

You would need headphones or a surround sound system to hear sound in a 3D sense right?
 

kburkhart84

Firehammer Games
Awesome I think I understand it a whole world better then I did before, just one thing I just realized
Glad to help...these are concepts that elude you until you get it...but suddenly when you do, they kinda just click.

You would need headphones or a surround sound system to hear sound in a 3D sense right?
Not really. Regular PC speakers have left and right sides(and lots of MP3 music takes advantage of it, you just don't notice because it is subtle). But it isn't only left and right though...if I understand it correctly, the system will do much more, even with just those speakers. Sounds that are behind you get a subtle "muffing" effect, since sounds from behind you are slightly muffled due to how your ears are "designed" to hear more clearly from the front. There is also a very slight delay between left and right for sounds that are to the sides. Plus, you get that doppler effect based on velocity that also works regardless of how many speakers you have.
 
Top