Introduction
Many commercial and open source multitracking applications feature some type of beat detection algorithm and/or sample replacement function. The concepts of beat detection for beat stretching, and sample replacement for replacing less than perfect human hits with samples, are powerful tools which can be extended into more artistic applications such as creative techniques for computer music. The following examples are beginning efforts for a user approach to sample replacement in Csound.
I. Sample Replacement using tablewa and tablera
In order to work with sample replacement using Csound opcodes, a simple approach was undertaken employing the precalculation and creation of a beat structured .wav file. A Tambourine.wav file was created using a semi-physical model of a tambourine sound from Csound. The Tambourine.wav example was created at 44.1 KHz, placing a tambourine hit every 0.5 sec. This short .wav file simulates a recorded track with a beat structure by placing an attack every 22050 samples, for a total file length of 4 seconds or 176400 samples (44100 * 4), and a total of 8 attacks. Because the beat contents of the tambourine file are known, in this case, there was no need to pursue beat detection for analysis. Csound has opcodes for transient detection, however Scheirer[1], and also Cheng et al.[2] discuss beat detection in the frequency domain.
A clap sound sample was chosen as the replacement sample to help distinguish replaced samples clearly from the existing tambourine hits. The clap sound was positioned via an audio editor to begin at the beginning of another .wav file, or "0" samples onset, and although the clap sample was only 0.5 sec.(22050 samples) in duration, the total length of clap.wav file was extended with silence to match the duration of the tambourine file (4 seconds) in order to work with the tablewa and tablera opcodes.
Although Csound has many opcodes which may be suitable for working with beat detection and sample replacement such as the opcodes floor, limit, peak, filepeak, trigger, readk, fink, seqtime2, timeinstk, etc., it was decided initially to employ the tablewa and tablera opcodes, since those opcodes are designed to write to sequential locations in tables using table lookup functions. The Tambourine.wav and Clap.wav files were each placed in respective tables of 4 seconds or 176400 samples in length and a UDO (User-defined Opcode) was designed to sequentially replace the even tambourine beats with a clap which should be audible in Example1.mp3.
By design it was decided to use .sco file and pfield entries to list the sequential sample numbers for replacement, allowing the instrument calling the UDO to be employed several times in the .sco, while changing the pfield values for particular sample numbers or beat replacements. Also a procedure was employed to read samples or beats from the tambourine and clap tables, and place them sequentially into a third table of the same initial length (table 101, Ex.1 .csd), which was then used with fout for writing the final .wav file to disk.
Setting ksmps to the size of the table length was based on a-rate wa/ra use inside a k-rate loop. For every max k values or end of loop, tablewa will write ksmps number of samples. All .csd example files and .wav sources can be downloaded here: SampleReplacement.zip.
II. Sample Replacement using vaget and vaset
While writing to sequential locations in tables using tablewa and tablera proved useful for sample replacement, the wa/ra approach was an acid test for using the sample level opcodes vaget and vaset. These opcodes, by Steven Yi, allow sample-by-sample manipulation at the k-rate, accessing values of the current buffer of an a-rate variable by indexing.
The approach shown below is a non-standard use of vaget and vaset. It accomplishes sample replacement via sample level manipulation employing multiple conditional if and elseif statements within a loop_lt based on index or sample numbers. This is a different approach from The Canonical Csound Reference Manual example for vaget which employs the values of those indexes, not the index or sample numbers themselves.
It was discovered after considerable experimentation that all varieties of looping oscillator opcodes would not work with this design. Therefore an approach was used to read a-variables from disk before employing vaget. The k-rate loop gives the same result as writing sequentially using tablewa and tablera by listing the sample or index numbers for replacement using conditional elseif statements.
instr 5a1 init 0a2 init 0a3 init 0a1 diskin "Tambourine.wav", 1, 0, 0, 4a2 diskin "Longclap.wav", 1, 0, 0, 4kval init 0kndx = 0/*beat1 - 0beat2 - 22050beat3 - 44100beat4 - 66150beat5 - 88200beat6 - 110250beat7 - 132300beat8 - 154350EOF - 176400*/loopStart1: if (kndx > 0 && kndx < 22049) then kval vaget kndx,a1 elseif (kndx > 22049 && kndx < 44099) then kval vaget kndx - 22049, a2 elseif (kndx > 44099 && kndx < 66139) then kval vaget kndx,a1 elseif (kndx > 66149 && kndx < 88199) then kval vaget kndx - 66149, a2 elseif (kndx > 88199 && kndx < 110249) then kval vaget kndx,a1 elseif (kndx > 110249 && kndx < 132299) then kval vaget kndx - 110249,a2 elseif (kndx > 132299 && kndx < 154349) then kval vaget kndx,a1 elseif (kndx > 154349) then kval vaget kndx - 154349,a2 endif vaset kval,kndx,a3 if (kndx >176400) goto last; loop_lt kndx, 1, ksmps, loopStart1last: turnoff endin
III. An Esoteric Example of Sample Replacement
The use of vaget and vaset, although non-standard, proved satisfactory for sample replacement, however the .orc code was not concise due to the use of multiple conditional if and elseif statements. While the approach shown above could be simplified by the development of a UDO and made more compact, some thought was given to the overall use of sample replacement in Csound before moving on to that task.
Because Csound is not always used for a beat oriented type music, and may use time as a reference, an audio file was synthesized (Bells.wav) at 48 KHz of 12 seconds duration which had pulses but not a clear metric beat structure as in the previous examples, and it was decided to try to replace samples using a percussive clank sound of 24000 samples which could be clearly distinguished from the existing pulses.
Unlike the commercial type drum kit sample replacement editing where the beat or hit is probably preceded and followed by a brief amount of near silence, this esoteric type sample replacement may occur at a non-zero crossing, and required a fade out of a few samples just before the replaced sample in order to avoid a click. One interesting observation was while working on the sample level, many useful opcodes, such as expseg for envelope generation, are not available and one has to create those functions. Log10 was used in the UDO below with extra scaling to create a 1000 sample fade to "0" before sample replacement.
As in the first example above it was decided to use .sco file and pfield entries to list the sample start locations for replacement, allowing the instrument calling the UDO to be employed several times in the .sco, but changing the pfield values for the point of particular beat replacement. The result should be audible in Example3.mp3.
<CsoundSynthesizer><CsOptions>-s -d -+rtaudio=PortAudio -odac4 -W -b1024 -B16384</CsOptions><CsInstruments>sr=48000ksmps=576000nchnls=1;GLOBAL VARIABLESgitmp1 ftgen 100, 0, -576000, -10, 1gitmp3 ftgen 200, 0, -576000, -10, 1gitmp4 ftgen 300, 0, -576000, -10, 1gabells init 0gabells2 init 0gaclank init 0gilen1 filelen "Bells.wav"gilen2 filelen "Clank.wav"gisamples1 init 0gisamples2 init 0;=======================;UDOopcode SampleReplace, a, iiiiiii;init local variablesa1 init 0a2 init 0a3 init 0kval init 0kndx = 0clear gabellsistart, ifade, icut, ismplstart, ismplend, icont, iend, xin/* ;ck values, etc.printf "start %d\n", 1, istartprintf "fade %d\n", 1, ifadeprintf "cut %d\n", 1, icutprintf "smplRplstart > %d\n", 1, ismplstartprintf "smplRplend < %d\n", 1, ismplendprintf "cont %d\n", 1, icont*/a1 diskin "Samplereplace.wav", 1, 0, 0, 4a2 diskin "clankfile.wav", 1, 0, 0, 4loopStart1: ;initial file if (kndx > istart && kndx < ifade) then kval vaget kndx,a1 ;SLIGHT FADE OUT within intial file elseif (kndx > ifade && kndx < icut) then kval vaget kndx,a1 ;LOGARITHMIC w xtra scaling kval = kval * (log10(10 - (10*((kndx-ifade)/1110)))) ;INSERT THE FIRST SAMPLE REPLACEMENT elseif (kndx > ismplstart && kndx < ismplend) then kval vaget kndx - ismplstart, a2 ;CONTINUE AFTER SAMP REPLACEMENT WITH ORIGINAL FILE elseif (kndx > icont) then kval vaget kndx, a1 endif vaset kval,kndx,a3 if (kndx > iend) goto last; loop_lt kndx, 1, ksmps, loopStart1last: gabells = gabells + a3 printf "UDO finished at %d\n", 1, kndx clear a1, a2, a3 xout a3endop;====================;WRITE DISKIN FILES TO EQUAL LENGTH TABLES instr 1;GET/WRITE Bells.wav to table(12 sec., no real beats, but pulsing snds);Load into a table(100) print gilen1gisamples1 = gilen1 * srprint gisamples1andx line 0, gilen1, gisamples1asig1 diskin "Bells.wav", 1, 0, 0, 4tablew asig1, andx, 100 endin;==============;GET/WRITE CLANK wav to table instr 2andx line 0, gilen1, gisamples1 ;using length from aboveasig2 diskin "Clank.wav", 1, 0, 0, 4tablew asig2, andx, 200 endin;======================;SAVE THE TABLES TO .WAV ON DISK instr 3andx line 0, gilen1, gisamples1 ;using length from abovea1 table andx, 100gabells= gabells + a1 endin;================== instr 4fout "Samplereplace.wav", 2, gabells endin;================== instr 5clear gabellsandx line 0, gilen1, gisamples1 ;using length from abovea1 table andx, 200gaclank= gaclank + a1 endin;================== instr 6fout "clankfile.wav", 2, gaclank endin;=================;READ FILES FROM DISK AND DO VAGET/VASET instr 7;SAMPLE REPLACER;istart, ifade, icut, ismplstart, ismplend, icont, iend, xinibeat = p4 ;beat to replaceistart = 0ifade = (ibeat - 1000)-1icut = ibeat -1ismplstart = ibeat -1 ismplend = (ibeat - 1) + (gilen2 * sr)icont = (ibeat - 1) + (gilen2 * sr)iend = gisamples1 asig SampleReplace istart, ifade, icut, ismplstart, ismplend, icont, iend endin/*SR = 48000start = 02 sec. = 960004 sec. = 1920008 sec. = 38400010 sec. = 48000012 sec. = EOF 576000*/;================= ;WRITE THE MODIFIED FILE TO DISK instr 8fout "Samplereplace.wav", 2, gabells endin;=================</CsInstruments><CsScore>f1 0 -576000 1 "Bells.wav" 0 4 0 f2 0 -24000 1 "Clank.wav" 0 4 0 i1 0 12si2 0 12si3 0 12si4 0 12si5 0 12si6 0 12si7 0 12 96000si8 0 12si7 0 12 192000si8 0 12si7 0 12 384000si8 0 12si7 0 12 480000si8 0 12e</CsScore></CsoundSynthesizer>
IV. Conclusions
It is known that disk operations are slow compared to those in memory, however the use of diskin proved more useful than looping oscillator opcodes when working with vaget and vaset for sample replacement. Commercial applications such as Drumagog and Digidesign's SoundReplacer, as well as others, are sophisticated and complex applications which have found usefulness in audio editing as plugins[3]. The basic idea of sample replacement can be employed for more esoteric applications, such as computer music techniques, and there is room for more opcode development in Csound which would support beat detection and sample replacement.
References
[1]Scheirer, Eric D. "Tempo and beat analysis of acoustic musical signals".The Journal of the Acoustical Society of America, Vol. 103, No. 1. (1998), pp. 588-601.
[2] Beat This. A Beat Synchronization Project. http://www.owlnet.rice.edu/~elec301/Projects01/beat_sync/beatalgo.html (8 Nov 2009)
[3]Digidesign. Drum Hit and Sound Replacement AudioSuite Plug-in. http://www.digidesign.com/index.cfm?navid=115&itemid=1059 (8 Nov 2009)