GML Generating sounds in real time

GMWolf

aka fel666
Hello everyone.

I have been trying to generate sound waves, and play them back in near-real time using audio buffers.

I came up with two techniques:
The first was to have a number of small buffers (10ms), and queue them up onto a audio queue.
whenever an audio async event was triggered, i would get the buffer ID, generate more data, and push it back onto the queue.

The problem with this technique was that i needed either many buffers, or larger buffers in order for the queue to keep playing. With smaller buffers the queue would simply stop after each buffered had played once.
The problem with larger buffers was that it ended up having too much lag. im looking for sub 20ms, optimally.

Technique 2 was to have two longer buffers, one "active" and one "passive". The two buffers would be queud up, and every async event, i would queue them back onto the queue.
In the step event, I would generate a bit more audio data onto the "active" buffer, in order to be just ahead of the current track position (adjusted to take into account how many bufferes already played).
When generating samples, if i ever exceeded the length of the active buffer, i would switch the two buffers around, and start generating on the other buffer.

This technique seemed to work, but was quite glitchy, It seemed like rather than streaming from the buffers, the queue was taking chunks from the buffer. Those chunks seemed larger than the ammount I would buffer ahead by. This resulted in some of the buffers to either not play or play some leftover garbage data.

From these two tests, It seems to me like the qudio queues would copy a chunk of data from the queue, and start playing it. The size of chunk taken is too long for my usage.

Do any of you know of a way to get near-real time audio generation to work?
If not i guess Ill be building a DLL or something. (but i'd rather not)

[edit] Not sure if important, but i was using s16 formated buffers. Would that make a difference when it comes to minimum buffer sizes?
 
Last edited:
From what you'd said, I'm taking it you are talking about 'Square', 'Triangle' and 'Sine' waves.

What I'd do is keep a list and a pre-generated buffer with a 10ms sample of each type of wave you need, then keep a ds_qeaue, two of them. One will have your audio buffer for which wave type you want, one will have the pitch. It will then reference both while it plays them, to play the wave at the correct pitch.

But that's how I'd do it.


Your second method should work well though, were you using 'buffer_copy' or manually transferring the buffer with read/write? buffer_copy is a memcpy call in c++, so it's VERY fast, and what I use for everything I can. When it comes to your problem with leftover data or not playing buffers, these are the same problem, just different sides of it. Your transferal method probably has the wrong offsets/length, so you are writing to the wrong region of the buffer you're streaming from... Try clearing out the buffer for testing reasons, before you write new data.

A good way to test this is to write the buffer out to a file, 50 or so times for each time new data is placed into the buffer. It will tell you exactly what is going on with your engine, its how I've found problems.

Keep in mind all this comes from theory, not practice. My skill is more with making custom audio-formats, converting other formats etc. I even made an ADPCM decoder for PS1/PS2 games.


A note to you, try and generate your sample waves with a C note, this is 440hz if memory serves me correctly. Most internet documentation is based around this, so you can get a lot of help with it.

s16 is merely better quality audio, it's two bytes of data rather than 1, and can be non positive. Did you generate your data this way?
 

GMWolf

aka fel666
From what you'd said, I'm taking it you are talking about 'Square', 'Triangle' and 'Sine' waves.
Any kind of wave, actually.

What I'd do is keep a list and a pre-generated buffer with a 10ms sample of each type of wave you need, then keep a ds_qeaue, two of them. One will have your audio buffer for which wave type you want, one will have the pitch. It will then reference both while it plays them, to play the wave at the correct pitch.
The idea is to have real time audio generation. Pre-generated buffers won't do.

Your second method should work well though, were you using 'buffer_copy' or manually transferring the buffer with read/write?
I'm writing directly to the buffer that is on the queue. Not much sense in writing to one buffer and then copy...

Your transferal method probably has the wrong offsets/length, so you are writing to the wrong region of the buffer you're streaming from...
I'm fairly certain I'm writing to the correct part of the buffer: I'm always using buffer write, and when I reach the end, I seek to the start of the other buffer.
Since my buffers are continually placed back onto the queue, this should work correctly.
But unlike the first technique, even when buffering a whole second in advance, it doesn't seem to work seamlessly. You will sometimes only get the first section of the buffer that was generated before the buffer was pushed to the queue (that is with 5 second long buffers).

Regardless, my first method should work fine with any size buffers, but it seems like I need around 100ms of buffers ahead of the current play position. When I try around 20 or 50ms of buffers, it simply stops playing, as if I didn't keep queuing data up.
 
Well then the only thing I can suggest is to write out your buffer as it's playing to debug. Have a look if you have random data that shouldn't be there.

With your first technique, having loads of buffers shouldn't be a problem. They're just locations in memory (plus a small amount of overhead to store the seek, size etc), so if they're not that big it won't matter. If keeping track of them is a problem, use a ds_list or something. Heck I use buffer's as a way to have structs in programs I make, so there ends up being more than 200 of them when I'm working with a lot of files.
 

GMWolf

aka fel666
Well then the only thing I can suggest is to write out your buffer as it's playing to debug. Have a look if you have random data that shouldn't be there.
When I generate the data to a single buffer and then push it to the queue the data is fine. Its when I generate the data with not enough ahead time on a buffer that is on the queue that I get intermittent behavior.
With your first technique, having loads of buffers shouldn't be a problem.
It is a problem: I need it to be near-real time. That means 20ms or so ahead of the current play position.
If I use more, or longer buffers, then It is no longer near real time.
It doesn't matter if I use many small buffers, or fewer larger buffers, they always seem to need to total up to ~100ms. Unacceptable for my use.

Btw, I simply let the queue keep track of them.
Whenever I get an async event, I simply overwrote the returned buffer with new data, and push it back onto the queue. Works like a charm with 100+ms buffers zone, simply stops playing with anything smaller.
 

GMWolf

aka fel666
This may just be a problem with audio latency then, as this is starting to sound exactly like that, rather than your code.
I'm not so sure, since just playing the audio buffers can be done in real time if I don't use a queue. (But then I get horrible gaps between the sounds).
I don't remember how I did it so I'll just give you my project file...

https://www.dropbox.com/s/u9kndgivo2gefyj/Project4.gmx.7z?dl=0
Thanks, Ill check it out!

[edit]
Ok, this seems to work very well!
Im suprised playing the sound every step like that isnt giving you breaks in the sound. when i tried something simmilar, i got breaks between each play of the sound.
ill try to implement something simmilar, hopefully itl work :) thanks again, to both of you, this was very helpful :)
 
Last edited:
R

renex

Guest
Im suprised playing the sound every step like that isnt giving you breaks in the sound.
I remember implementing a millisecond counter that would adjust the length of the sound based on delta time. It might be what's missing for your implementation. It's one of the numbers on the debug text.
 
T

ThunkGames

Guest
Something I had commissioned a while back (I'm free to distribute it too):

Code:
///SoundCreate(parameters);

var params = argument0;

var rate = params[?"sampling"];

//General Information
wave_type = params[?"wave_type"];

//Punch
var punch = params[?"punch"];


//Time/Duration Variables
var repeats = params[?"repeats"];
var attacktime = params[?"attack_time"];; //No sound to full
var sustaintime = params[?"sustain_time"];; //Full sound all the way
var decaytime = params[?"decay_time"];; //Fading of the sound

var duration = attacktime+sustaintime+decaytime;

//Frequency Variables
var freq = params[?"start_frequency"];; //The base sound frequency
var fslide = params[?"frequency_slide"];; //Altering the frequency overtime
var dslide = params[?"delta_slide"];; //Altering the altertion of frequency


var freqcut = params[?"frequency_cutoff"];

//Vibrato Variables
var vibrato = params[?"vibrato_depth"];;
var vibratospeed = params[?"vibrato_speed"];;

//Harmonics variables
var harmonics = params[?"harmonics"];;

//Frequency Filters
var low_limiter = params[?"low_limiter"];;
var high_limiter = params[?"high_limiter"];;
var ll_strength = params[?"low_limiter_strength"];
var ll_sweep = params[?"low_limiter_sweep"];
var hl_strength = params[?"high_limiter_strength"];
var hl_sweep = params[?"low_limiter_sweep"];


//Pitch control system
var pitch = 1;
var p1 = params[?"pitch1"];
var p2 = params[?"pitch2"];
var p1_time = params[?"pitch1_time"];
var p2_time = params[?"pitch2_time"];
var pitch_repeats = params[?"pitch_repeat"];
var pitch_time = duration/(pitch_repeats);


//Bit crushing algorithms (quatization)
var bit_crush = params[?"bit_crushing"];
var bc_sweep = params[?"bit_crushing_sweep"];;



var gain = params[?"gain"]; //The volume of the sound

//Create the sound buffer

var sbuffer = buffer_create(rate*duration*4*repeats,buffer_fixed,4);


//Fill the buffer
repeat(repeats){
    for(t = 0; t<duration*rate;t++){
 
        var truet = t/rate; //The true time used for calculations
        var value = 0; //The value that will be written
      
        //Calculate the correct gain of the sound
        var finalgain = gain; //The final volume of the sound
      
        //Tamper with the frequency
      
        fslide+=dslide/rate;
        freq+=fslide/rate;
        freq = clamp(freq,50,20000); //Clamp the frequency to audible amounts
      
      
        //Apply the filters
          
        low_limiter += ll_sweep/rate;
        high_limiter += hl_sweep/rate;
      
        var flt = -1;
        if(freq < low_limiter){
            flt = low_limiter;
            var distance = abs(flt-freq);
            finalgain = max(0,finalgain-distance*ll_strength);
        }
        if(freq > high_limiter){
            flt = high_limiter;
            var distance = abs(flt-freq);
            finalgain = max(0,finalgain-distance*hl_strength);
        }
      
        //Vibrato additions
        finalgain *= (1+vibrato*SoundGetFrequency(0,vibratospeed/2,t,rate));
      
      
        //Cuttof when time ends/starts
        if(truet < attacktime){
            finalgain *= sqr(truet/attacktime);
        }else if(truet > attacktime+sustaintime){
            finalgain *= sqr(1-(truet-attacktime-sustaintime)/decaytime);
        }
      
        //Cuttof when frequency cutter does not allow
        if(freq < freqcut)
            finalgain = 0;
          
        //Pitch changer
      
        pitch = 1;
        var tempt = truet mod pitch_time;
        tempt /= pitch_time;
        if(truet mod pitch_time > duration*p1_time/pitch_repeats)
            pitch*= p1;
        if(truet mod pitch_time > duration*p2_time/pitch_repeats)
            pitch*= p2;
      
        var finalfreq = freq*pitch;
      
        var value;
      
        if(harmonics == 0 || harmonics == 1)
            value += finalgain*(SoundGetFrequency(wave_type,finalfreq,t,rate)*(1+random(punch/100)));
        else{
            var ff = finalfreq;
            for(i = 1; i < harmonics; i++){
                value += finalgain*(SoundGetFrequency(wave_type,ff,t,rate)*(1+random(punch/100)));
              
                ff+=finalfreq;
                //Check and stop if frequencies are geeting too high
                var off = clamp(ff,0,20000);
                if(ff != off){
                    break;
                }
            }
        }
      
        //Bit crushing the value in the end
        bit_crush += bc_sweep/rate;
      
        var s = sign(value);
        value /= bit_crush;
        value = round(abs(value));
        value *= s*bit_crush;
      
        /*var valper = abs(value)/finalgain;
        valper *= finalgain;
        valper /= bit_crush;
        valper = floor(valper);
        valper *= bit_crush;
        valper /= 1000;
        value = sign(value)*valper*finalgain*/
      
 
        buffer_write(sbuffer,buffer_s16,value);
 
    }
}
return audio_create_buffer_sound(sbuffer,buffer_s16,rate,0,rate*duration*4*repeats,1);

Helper Script:
Code:
///get_frequency(type,frequency,time,r);


var type = argument0;
var freq = argument1;
var t = argument2;
var r = argument3;

var period = 1/freq;

var truet = t/r;
var tt = truet/period;


switch(type){
 
    case 0:
        //Sine wave
      
        return cos(2*pi*freq*t/r);
 
    case 1:
        //Rect Wave
        return sign(cos(2*pi*freq*t/r));
      
    case 2:
        //Trig
        if(tt < .25){
            return (tt/.25);
        }else if(tt < .75){
            return 1-2*(tt-.25)/.5;
        }else{
            return -1+(tt-.75)/.25;
        }
    case 3:
        //Saw
        return -1+2*tt;
      
    case 4:
        //Breaker
        return ((2*get_frequency(0,freq,t,r))+sign(get_frequency(1,freq,t,r)))/3;
    case 5:
        //Tan (may rethink this)
        return tan(2*pi*freq*t/r);
    case 6:
        //Whistle
        return cos(2*pi*freq*t/r) + .1*cos(20*2*pi*freq*t/r);
    case 7:
        //White Noise
        return random(abs(cos(2*pi*freq*t/r)));
    case 8:
        //Pink Noise
        return random(abs(cos(pi*freq*t/r)))*sign(cos(pi*freq*t/r))*.9;
      
}

Example:
Code:
params = json_decode('{
                      "wave_type":3,
                      "repeats":1,
                      "punch":0,
                      "attack_time":0.05,
                      "sustain_time":1.6,
                      "decay_time":0.01,
                      "start_frequency":600,
                      "frequency_cutoff":0,
                      "frequency_slide":0,
                      "delta_slide":0,
                      "vibrato_depth":1,
                      "vibrato_speed":3,
                      "harmonics":0,
                      "low_limiter":0,
                      "low_limiter_sweep":0,
                      "low_limiter_strength":0,
                      "high_limiter":20000,
                      "high_limiter_strength":0,
                      "high_limiter_sweep":0,
                      "sampling":44000,
                      "gain":9000,
                      "pitch1":1.5,
                      "pitch1_time":0.5,
                      "pitch2":0.3,
                      "pitch2_time":0.8,
                      "pitch_repeat":1,
                      "bit_crushing":1,
                      "bit_crushing_sweep":0
                      }');     


audio_play_sound(create_sound(params),0,0);

Introduction (copy and pasted):

The 2 scripts: create_sound and get_frequency,
are needed both to run correctly.

They simulate most of the bxfr synthesizer functinality,
However, due to GMS limitations, the effects might be quite altered,
Its important to note that this is not a direct port,
but a sohisticated simulation of the system, which means,
the implementation of the features may, and will most likely,
difer from the original.

Parameters (copied and pasted):

wave_type: 0 to 8
Determines what wave type will be used for the effect,
Different wave types alter the sound a lot.

0 sine wave
1 rect wave
2 trig wave
3 saw wave
4 breaker wave
5 tan wave
6 whistle
7 white noise
8 pink noise

gain: positive

It affects the volume of the sound


attack_time: positive
The time it takes for the track to reach full volume from the start

sustain_time: positive
The time where the sound plays without volume change

decay_time: positive
The time it takes for the sound to reach mute from full volume

repeats: positive
How many times will the sound repeat

sampling: positive
Determines the compression of the sound, more is better. A resonable value is 40000.

start_frequency: positive
Determines the frequency of the sound, which affects the pitch.E.G the note A has a frequency of 440

frequency_cutoff: positive
Limits the possible frequency, If it falls below the frequency_cutoff it will be muted.

frequency_slide: positive or negative
Alters the frequency per second (it adds to it or subtracts)

delta_slide: positive or negative
Alters the frequency_slide parameter per second, to give more flexibility

low_limiter: positive
high_limiter: positive
Those set a limit on the possible frequencies, e.g low_limiter:1000 high_limiter:9000 cuts frequencies outside (1000,9000)

low_limiter_strength: positive
high_limiter_strength: positive
Those determine how harshly will the frequency limiters work respectively,

low_limiter_sweep: positive or negative
high_limiter_sweep: positive or negative
Those 2 alter low_limiter and high_limiter values respectively, every second.
This gives more flexibility to the frequency limiting


pitch1: positive
pitch2: positive
pitch1_time: zero to one
pitch2_time: zero to one
pitch_repeat: positive
These variables allow for calculated pitch swifts within the sound
the pitch variables determine the itensity of the effect e.g pitch1:2 doubles the pitch
The pitch_time variables determine when the pitch change will happen
e.g pitch1_time:0.5 the pitch alteration will happen at the half time of the sound effect
Finally pitch_repeat determines how many times the above user created patern will repeat,
throughout the sound effect.


Now for the effects

punch:positive
Adds artifacts to the sound to resemble an old recording, more means greater effect

harmonics:positive
Interlaces the sound with multiples of its frequency, thus enriching it

vibrato_depth:positive
vibrato_speed:positive
Makes the sound have a vibrating effect by affecting the gain, and those 2 parameters,
affect the speed and the itensity of the effect.
e.g vibrato_depth:1 vibrato_speed:1 makes a vibrating volume that doubles and mutes,
like a wave once a second


bit_crushing:positive
Lowers the "resolution" or sampling rate to match a lower capability system,
which creates interesting artifacts, Higher value increases the effect

bit_crushing_sweep:positive or negative
Alters the bit_crushing parameter (adds or subtracts to it every second),
giving the bit_crushing effect greater flexibility.

Here is the list of the parameters range (also copied and pasted from an email):
wave_type: 0-8 (pretty obvious, each number corresponds to a wave,
I have sent you details before)

gain:
It's the volume of the sound.
This requires some testing to figure it's range,
however I found that 8000 is good enough for the max volume.
So I recommend 0 to 8000 as the range.

attack_time, sustain_time,decay_time:
Each of those is measured in seconds, If you add them you get the
full effect duration. Limiting each to five would be a good option.

repeats: Will repeat the signal as many times as you set it,
If the sound lasts 10 seconds, a repeat of 3 will make it 30 seconds.
So if you allow a high value it may put a big load on the cpu.
I recommend not giving an option to change that, or keep it
below 3 if you don't want large sound effects.

sampling: In exact, it measures how compressed the sound will be.
A typical value used by many is 48.000 and will give an ok sound
setting it too hihg will give a cleaner sound, but may affect preformance,
especially in long sounds 5000-100000 or less will be a good range.

start_frequency, frequency_cutoff,low_limiter,high_limiter:
All those parameters have to do with frequency.
The human ear can distinguish 20-20000 Hertz of frequency,
So limiting it between these values will do.


low_limiter_strength, high_limiter_strength:
Those 2 sound be set with a minimum of 0 and a maximum
same as the gain.


pitch1, pitch2:
They alter the pitch of the sound.
A range of 0-3 for each of them Is fine.
Other pitch arguments are fully described, and giver range,
on the .txt file that I sent.


punch:
This effect is hardcoded to accept a range of 0 to 10.
Anything outside that will cause strange effects.
Putting it near 10 will almost destroy the sound with noise,
so it may not be a bad idea to set the range as 0 to 5.

harmonics:
Accepts integer positive values. 0 and one will do nothing.
2 or more will add frequency multiples to your sound.
I recommend limiting it to 10 or less.

vibrato_depth: Range from 0 to 1, it's hardcoded to work well
on those values, so make this the range


vibrato_speed: Determines the vibrato speed effect.
20 means 20 vibrations per second.
In fact 0 to 20 is a good range for it, but you can make it longer
if you see fit.


bit_crushing:
The range should be the same as the gain, more than it, it does nothing.


All other arguments accept previous arguments per second,
so I cannot recommend a range for these,
however they take positive aswell as negative values,
so 0 should be the default and should be in the middle.
for example -10 to 10, and not -20 to 40.
 

GMWolf

aka fel666
I remember implementing a millisecond counter that would adjust the length of the sound based on delta time. It might be what's missing for your implementation. It's one of the numbers on the debug text.
Yes, THe problem i was having when trying something simmilar was to simply play the sound when the async event triggered. But Async isnt actually async. Its very syncronous indeed! It will only fire during a step. So the sound would stop playing, but only fire the event a little after.
The way you implemented it ensure the sound will be as long as the previous step, so if the frame rate is steady, it works quite well. Unfortunately if there is a lag spike, there is an audible artifact. This is still the best solution yet though, so i think ill be using it.
Something I had commissioned a while back (I'm free to distribute it too):

Code:
///SoundCreate(parameters);

var params = argument0;

var rate = params[?"sampling"];

//General Information
wave_type = params[?"wave_type"];

//Punch
var punch = params[?"punch"];


//Time/Duration Variables
var repeats = params[?"repeats"];
var attacktime = params[?"attack_time"];; //No sound to full
var sustaintime = params[?"sustain_time"];; //Full sound all the way
var decaytime = params[?"decay_time"];; //Fading of the sound

var duration = attacktime+sustaintime+decaytime;

//Frequency Variables
var freq = params[?"start_frequency"];; //The base sound frequency
var fslide = params[?"frequency_slide"];; //Altering the frequency overtime
var dslide = params[?"delta_slide"];; //Altering the altertion of frequency


var freqcut = params[?"frequency_cutoff"];

//Vibrato Variables
var vibrato = params[?"vibrato_depth"];;
var vibratospeed = params[?"vibrato_speed"];;

//Harmonics variables
var harmonics = params[?"harmonics"];;

//Frequency Filters
var low_limiter = params[?"low_limiter"];;
var high_limiter = params[?"high_limiter"];;
var ll_strength = params[?"low_limiter_strength"];
var ll_sweep = params[?"low_limiter_sweep"];
var hl_strength = params[?"high_limiter_strength"];
var hl_sweep = params[?"low_limiter_sweep"];


//Pitch control system
var pitch = 1;
var p1 = params[?"pitch1"];
var p2 = params[?"pitch2"];
var p1_time = params[?"pitch1_time"];
var p2_time = params[?"pitch2_time"];
var pitch_repeats = params[?"pitch_repeat"];
var pitch_time = duration/(pitch_repeats);


//Bit crushing algorithms (quatization)
var bit_crush = params[?"bit_crushing"];
var bc_sweep = params[?"bit_crushing_sweep"];;



var gain = params[?"gain"]; //The volume of the sound

//Create the sound buffer

var sbuffer = buffer_create(rate*duration*4*repeats,buffer_fixed,4);


//Fill the buffer
repeat(repeats){
    for(t = 0; t<duration*rate;t++){
 
        var truet = t/rate; //The true time used for calculations
        var value = 0; //The value that will be written
     
        //Calculate the correct gain of the sound
        var finalgain = gain; //The final volume of the sound
     
        //Tamper with the frequency
     
        fslide+=dslide/rate;
        freq+=fslide/rate;
        freq = clamp(freq,50,20000); //Clamp the frequency to audible amounts
     
     
        //Apply the filters
         
        low_limiter += ll_sweep/rate;
        high_limiter += hl_sweep/rate;
     
        var flt = -1;
        if(freq < low_limiter){
            flt = low_limiter;
            var distance = abs(flt-freq);
            finalgain = max(0,finalgain-distance*ll_strength);
        }
        if(freq > high_limiter){
            flt = high_limiter;
            var distance = abs(flt-freq);
            finalgain = max(0,finalgain-distance*hl_strength);
        }
     
        //Vibrato additions
        finalgain *= (1+vibrato*SoundGetFrequency(0,vibratospeed/2,t,rate));
     
     
        //Cuttof when time ends/starts
        if(truet < attacktime){
            finalgain *= sqr(truet/attacktime);
        }else if(truet > attacktime+sustaintime){
            finalgain *= sqr(1-(truet-attacktime-sustaintime)/decaytime);
        }
     
        //Cuttof when frequency cutter does not allow
        if(freq < freqcut)
            finalgain = 0;
         
        //Pitch changer
     
        pitch = 1;
        var tempt = truet mod pitch_time;
        tempt /= pitch_time;
        if(truet mod pitch_time > duration*p1_time/pitch_repeats)
            pitch*= p1;
        if(truet mod pitch_time > duration*p2_time/pitch_repeats)
            pitch*= p2;
     
        var finalfreq = freq*pitch;
     
        var value;
     
        if(harmonics == 0 || harmonics == 1)
            value += finalgain*(SoundGetFrequency(wave_type,finalfreq,t,rate)*(1+random(punch/100)));
        else{
            var ff = finalfreq;
            for(i = 1; i < harmonics; i++){
                value += finalgain*(SoundGetFrequency(wave_type,ff,t,rate)*(1+random(punch/100)));
             
                ff+=finalfreq;
                //Check and stop if frequencies are geeting too high
                var off = clamp(ff,0,20000);
                if(ff != off){
                    break;
                }
            }
        }
     
        //Bit crushing the value in the end
        bit_crush += bc_sweep/rate;
     
        var s = sign(value);
        value /= bit_crush;
        value = round(abs(value));
        value *= s*bit_crush;
     
        /*var valper = abs(value)/finalgain;
        valper *= finalgain;
        valper /= bit_crush;
        valper = floor(valper);
        valper *= bit_crush;
        valper /= 1000;
        value = sign(value)*valper*finalgain*/
     
 
        buffer_write(sbuffer,buffer_s16,value);
 
    }
}
return audio_create_buffer_sound(sbuffer,buffer_s16,rate,0,rate*duration*4*repeats,1);

Helper Script:
Code:
///get_frequency(type,frequency,time,r);


var type = argument0;
var freq = argument1;
var t = argument2;
var r = argument3;

var period = 1/freq;

var truet = t/r;
var tt = truet/period;


switch(type){
 
    case 0:
        //Sine wave
     
        return cos(2*pi*freq*t/r);
 
    case 1:
        //Rect Wave
        return sign(cos(2*pi*freq*t/r));
     
    case 2:
        //Trig
        if(tt < .25){
            return (tt/.25);
        }else if(tt < .75){
            return 1-2*(tt-.25)/.5;
        }else{
            return -1+(tt-.75)/.25;
        }
    case 3:
        //Saw
        return -1+2*tt;
     
    case 4:
        //Breaker
        return ((2*get_frequency(0,freq,t,r))+sign(get_frequency(1,freq,t,r)))/3;
    case 5:
        //Tan (may rethink this)
        return tan(2*pi*freq*t/r);
    case 6:
        //Whistle
        return cos(2*pi*freq*t/r) + .1*cos(20*2*pi*freq*t/r);
    case 7:
        //White Noise
        return random(abs(cos(2*pi*freq*t/r)));
    case 8:
        //Pink Noise
        return random(abs(cos(pi*freq*t/r)))*sign(cos(pi*freq*t/r))*.9;
     
}

Example:
Code:
params = json_decode('{
                      "wave_type":3,
                      "repeats":1,
                      "punch":0,
                      "attack_time":0.05,
                      "sustain_time":1.6,
                      "decay_time":0.01,
                      "start_frequency":600,
                      "frequency_cutoff":0,
                      "frequency_slide":0,
                      "delta_slide":0,
                      "vibrato_depth":1,
                      "vibrato_speed":3,
                      "harmonics":0,
                      "low_limiter":0,
                      "low_limiter_sweep":0,
                      "low_limiter_strength":0,
                      "high_limiter":20000,
                      "high_limiter_strength":0,
                      "high_limiter_sweep":0,
                      "sampling":44000,
                      "gain":9000,
                      "pitch1":1.5,
                      "pitch1_time":0.5,
                      "pitch2":0.3,
                      "pitch2_time":0.8,
                      "pitch_repeat":1,
                      "bit_crushing":1,
                      "bit_crushing_sweep":0
                      }');    


audio_play_sound(create_sound(params),0,0);

Introduction (copy and pasted):

The 2 scripts: create_sound and get_frequency,
are needed both to run correctly.

They simulate most of the bxfr synthesizer functinality,
However, due to GMS limitations, the effects might be quite altered,
Its important to note that this is not a direct port,
but a sohisticated simulation of the system, which means,
the implementation of the features may, and will most likely,
difer from the original.

Parameters (copied and pasted):

wave_type: 0 to 8
Determines what wave type will be used for the effect,
Different wave types alter the sound a lot.

0 sine wave
1 rect wave
2 trig wave
3 saw wave
4 breaker wave
5 tan wave
6 whistle
7 white noise
8 pink noise

gain: positive

It affects the volume of the sound


attack_time: positive
The time it takes for the track to reach full volume from the start

sustain_time: positive
The time where the sound plays without volume change

decay_time: positive
The time it takes for the sound to reach mute from full volume

repeats: positive
How many times will the sound repeat

sampling: positive
Determines the compression of the sound, more is better. A resonable value is 40000.

start_frequency: positive
Determines the frequency of the sound, which affects the pitch.E.G the note A has a frequency of 440

frequency_cutoff: positive
Limits the possible frequency, If it falls below the frequency_cutoff it will be muted.

frequency_slide: positive or negative
Alters the frequency per second (it adds to it or subtracts)

delta_slide: positive or negative
Alters the frequency_slide parameter per second, to give more flexibility

low_limiter: positive
high_limiter: positive
Those set a limit on the possible frequencies, e.g low_limiter:1000 high_limiter:9000 cuts frequencies outside (1000,9000)

low_limiter_strength: positive
high_limiter_strength: positive
Those determine how harshly will the frequency limiters work respectively,

low_limiter_sweep: positive or negative
high_limiter_sweep: positive or negative
Those 2 alter low_limiter and high_limiter values respectively, every second.
This gives more flexibility to the frequency limiting


pitch1: positive
pitch2: positive
pitch1_time: zero to one
pitch2_time: zero to one
pitch_repeat: positive
These variables allow for calculated pitch swifts within the sound
the pitch variables determine the itensity of the effect e.g pitch1:2 doubles the pitch
The pitch_time variables determine when the pitch change will happen
e.g pitch1_time:0.5 the pitch alteration will happen at the half time of the sound effect
Finally pitch_repeat determines how many times the above user created patern will repeat,
throughout the sound effect.


Now for the effects

punch:positive
Adds artifacts to the sound to resemble an old recording, more means greater effect

harmonics:positive
Interlaces the sound with multiples of its frequency, thus enriching it

vibrato_depth:positive
vibrato_speed:positive
Makes the sound have a vibrating effect by affecting the gain, and those 2 parameters,
affect the speed and the itensity of the effect.
e.g vibrato_depth:1 vibrato_speed:1 makes a vibrating volume that doubles and mutes,
like a wave once a second


bit_crushing:positive
Lowers the "resolution" or sampling rate to match a lower capability system,
which creates interesting artifacts, Higher value increases the effect

bit_crushing_sweep:positive or negative
Alters the bit_crushing parameter (adds or subtracts to it every second),
giving the bit_crushing effect greater flexibility.

Here is the list of the parameters range (also copied and pasted from an email):
wave_type: 0-8 (pretty obvious, each number corresponds to a wave,
I have sent you details before)

gain:
It's the volume of the sound.
This requires some testing to figure it's range,
however I found that 8000 is good enough for the max volume.
So I recommend 0 to 8000 as the range.

attack_time, sustain_time,decay_time:
Each of those is measured in seconds, If you add them you get the
full effect duration. Limiting each to five would be a good option.

repeats: Will repeat the signal as many times as you set it,
If the sound lasts 10 seconds, a repeat of 3 will make it 30 seconds.
So if you allow a high value it may put a big load on the cpu.
I recommend not giving an option to change that, or keep it
below 3 if you don't want large sound effects.

sampling: In exact, it measures how compressed the sound will be.
A typical value used by many is 48.000 and will give an ok sound
setting it too hihg will give a cleaner sound, but may affect preformance,
especially in long sounds 5000-100000 or less will be a good range.

start_frequency, frequency_cutoff,low_limiter,high_limiter:
All those parameters have to do with frequency.
The human ear can distinguish 20-20000 Hertz of frequency,
So limiting it between these values will do.


low_limiter_strength, high_limiter_strength:
Those 2 sound be set with a minimum of 0 and a maximum
same as the gain.


pitch1, pitch2:
They alter the pitch of the sound.
A range of 0-3 for each of them Is fine.
Other pitch arguments are fully described, and giver range,
on the .txt file that I sent.


punch:
This effect is hardcoded to accept a range of 0 to 10.
Anything outside that will cause strange effects.
Putting it near 10 will almost destroy the sound with noise,
so it may not be a bad idea to set the range as 0 to 5.

harmonics:
Accepts integer positive values. 0 and one will do nothing.
2 or more will add frequency multiples to your sound.
I recommend limiting it to 10 or less.

vibrato_depth: Range from 0 to 1, it's hardcoded to work well
on those values, so make this the range


vibrato_speed: Determines the vibrato speed effect.
20 means 20 vibrations per second.
In fact 0 to 20 is a good range for it, but you can make it longer
if you see fit.


bit_crushing:
The range should be the same as the gain, more than it, it does nothing.


All other arguments accept previous arguments per second,
so I cannot recommend a range for these,
however they take positive aswell as negative values,
so 0 should be the default and should be in the middle.
for example -10 to 10, and not -20 to 40.
Wow thanks! This will be super helpful!
 

Mike

nobody important
GMC Elder
I've done this quite a bit in GMS2, my C64 emulator, my Spectrum emulator and my MOD player.

The emulators had to try and "sync" with the video rendering so was a little odd. The framerate was basically 9999 with the audio event setting a flag to say when to fill int the buffer and render the next frame. This worked but is yucky

For "pure" music rendering, you simply sit on the audio event callback and fill buffers as they come in. You have to try and stike a balance between big buffers that will last longer but give you loads of lag, and small ones that mean you have to keep filling them in but are faster to respond to changes.

The MOD player does ALL it's work in the audio event, sample generation, channel mixing - everything. Occasionally you may get a couple of events one game frame, and sometimes none. Because the audio stuff is on a timer it will be out of sync with video (/game...sprites etc) rendering, so the more you can put in the audio event the better. Also make sure to check for having to requeue the buffer, as when you start dragging windows around it can stop. This isn't because window dragging stops it, just simply because the buffers "run out" as nothing has refilled them.

The manual on audio buffers is pretty perfect for streaming stuff, copy and paste and it should be streaming an empty buffer - then you just fill it in.

You can buy the C64 and MOD play stuff on the Marketplace ;)
 

GMWolf

aka fel666
For "pure" music rendering, you simply sit on the audio event callback and fill buffers as they come in. You have to try and stike a balance between big buffers that will last longer but give you loads of lag, and small ones that mean you have to keep filling them in but are faster to respond to changes.
he MOD player does ALL it's work in the audio event, sample generation, channel mixing - everything.
THis was actually the first way i tried doing it. I queue up a couple (5) very small (10ms) buffers, and as the async event triggered, i would take the buffer, dump more data onto them and queue them back up, all in the async event.
It didnt work. I had to increase the buffer size, or use more buffers, before it would work correctly.
This meant too much lag. This isnt just background music, As i want the user to interact with the sound, a bit like if it was a keyboard. So it needs to be very responsive.

Because the audio stuff is on a timer it will be out of sync with video (/game...sprites etc) rendering, so the more you can put in the audio event the better.
I actually had a question about that: Are async events truely async, as in, they will trigger the moment the audio ends? or are they queued up and tirggered the next step?
From one of my tests it seemed like it was doing the later.

I still cant seem to get below a certain "buffer ahead" time when using queues. Is there a limit to how little data ahead of the play position i can put?

You can buy the C64 and MOD play stuff on the Marketplace
I may go check out the MOD tracker :)
 
H

HammerOn

Guest
If there is a limited number of possible sounds, you should use a cache. If there isn't, you should design your project to be like this.
Then copy from the cache to the big buffer or audio queue to avoid lag.
But I'm almost sure your main problem is the audio async event.

I actually had a question about that: Are async events truely async, as in, they will trigger the moment the audio ends? or are they queued up and tirggered the next step?
From one of my tests it seemed like it was doing the later.
The engine may spawn a thread to do the processing (ex. load a file) but the async events are queued callbacks. They aren't called when the thread finishes the work but in the main loop of the game like any other event.
You generate 10ms but a frame takes 16ms (if the fps is 60) plus fluctuations. There will be synchronization issues for sure.
You can try a high framerate to workaround this but... you know... GM is a frame based engine so this is solving a problem with another problem. More workarounds!
 
L

Lycanphoenix

Guest
I've done this quite a bit in GMS2, my C64 emulator, my Spectrum emulator and my MOD player.

The MOD player does ALL it's work in the audio event, sample generation, channel mixing - everything. You can buy the C64 and MOD play stuff on the Marketplace ;)
I may go check out the MOD tracker :)
Somebody said MOD player extension? I have to check that out!
*Is a MOD-Tracker enthusiast*
 

GMWolf

aka fel666
If there is a limited number of possible sounds, you should use a cache. If there isn't, you should design your project to be like this.
That would defeat the entire point of this project.
You generate 10ms but a frame takes 16ms (if the fps is 60) plus fluctuations. There will be synchronization issues for sure.
You can try a high framerate to workaround this but... you know... GM is a frame based engine so this is solving a problem with another problem. More workarounds!
Yes, I'm starting to think GM may not be entirely suitable.
I May have to switch to using a DLL for this.
 

Mike

nobody important
GMC Elder
10ms is less than a frame so you'll need to make sure you have at least a frame - probably 2 - worth of buffer. This shouldn't give too much lag. Events are not threaded, they don't come in "any old time", they are queued for the start of the next game frame. This means if they are <1frame you won't be able to fill it until the next frame - and it'll probably stop playing.

Also... Audio drivers a little funny with creating buffers of small "odd" sizes, so make sure the buffers actually get queued. As I said... 1 or 2 frames should do it, but make sure you queue up 3 or 4 of them so there is some overlap to game frames and giving you time to build and queue the next one.

GML based MOD tracker: https://marketplace.yoyogames.com/assets/4951/mod-tracker
 

GMWolf

aka fel666
10ms is less than a frame so you'll need to make sure you have at least a frame - probably 2 - worth of buffer. This shouldn't give too much lag. Events are not threaded, they don't come in "any old time", they are queued for the start of the next game frame. This means if they are <1frame you won't be able to fill it until the next frame - and it'll probably stop playing.

Also... Audio drivers a little funny with creating buffers of small "odd" sizes, so make sure the buffers actually get queued. As I said... 1 or 2 frames should do it, but make sure you queue up 3 or 4 of them so there is some overlap to game frames and giving you time to build and queue the next one.

GML based MOD tracker: https://marketplace.yoyogames.com/assets/4951/mod-tracker
Thanks. I'll play around with timing a little more. Try to get 4 frames at 60fps or something.

I will check out the MOD tracker :)

Thank you all for your help!
 
Top