For full credit, show your work.
Suppose Banana Audio Company asks you to design a resampling algorithm for their MP3 product (which runs under a fixed 44100 Hz sample rate). The algorithm converts the sound of sample rate f HZ to any other sample rate f'.
Suppose you have the following ready-made functions
[n,d] = frac(f, g);
// returns the minimal integer n and d such that n / d = f / g.
S = down_sample(a, X);
// it samples X to S at the rate of the integer a. For example, if a = 2 and the signal X = [1 2 3 4 5 6 7], then S = [1 3 5 7].
S = up_sample(b, X);
// it up-samples X to S by inserting b-1 zeros between each pair of samples in X. For example, if a = 3, and X = [1 2 3 1], then S = [1 0 0 2 0 0 3 0 0 1].
S = lp(X, f)
// a low pass filter. It filters out the frequency components above f HZ from X and returns signal S.
S = hp(X, f)
// a high pass filter. It filters out the frequency components between 0 and f HZ from X and returns the new signal S.
The goal of your design is to ensure that the resampled sound is free from distortion (e.g. making the new sound higher or lower) and noise (e.g. aliasing).
(1.1.1) Suppose the original signal X has sample rate of f = 88200 HZ and the target sample rate is f' = 44100 HZ. Your teammate thinks we can use function S = down_sample(2, X) to get the result. Is there any problem?
(1.1.2) After doing some case studies, you found that many customers have files where f = 48000 HZ, which is not a multiple or divisor of f' = 44100 HZ. Your teammate thinks we can use the following scheme: First, calculate the integer ratio of f' / f = n / d, which will give us 147/160. Then, down_sample the original signal at the rate 160. Then use up_sample to raise the sample rate by a factor of 147. I.e. Y = up_sample(147, down_sample(160, X)). Finally, use a low pass filter to remove aliasing: S = lp(Y, 44100 * 147 / 160); What's the problem? Hint: up_sample() followed by a low-pass filter is much like a DAC followed by a reconstruction filter; a low-pass filter followed by down_sample is much like an anti-aliasing filter followed by an ADC. Think of this problem as like converting from digital (sample rate f) to analog and back to digital (new sample rate f').
1.2.1. We've learned that envelopes are actually represented as sounds in Nyquist, so that using an envelope to shape a sound is equivalent to multiplying two sounds together. Please run the following codes and briefly describe and explain what you hear:
play (hzosc(439) + hzosc(441)) ~ 2
play (hzosc(440) * hzosc(1)) ~ 2
play (hzosc(150) * hzosc(350)) ~ 2
play (hzosc(500) + hzosc(200)) ~ 2
You should expect 2 pairs of similar sounds, yet all 4 sounds are created quite differently. Explain in terms of frequency spectra why the similar sounds are similar.
1.2.2. Assume that the frequency of a carrier signal is 1100 Hz and the frequency of a modulation signal is 800 Hz. What are the frequencies of two sidebands in Ring Modulation?
1.3.1. What is the relationship between modulation index and amplitude? Will amplitude vary a lot if we change modulation index?
1.3.2. When will FM synthesis generate inharmonic spectra?
1.3.3. As a rule of thumb, what is the number of significant sideband pairs in FM?
1.4. Nyquist
1.4.1. What does the score-sustain function in Nyquist help in achieving?
1.4.2. Explain the difference between fn1() and fn2() in the following:
variable *fn* = pwl(0.5,1,1)
function fn1() return pwl(0.5,1,1)
function fn2() return *fn*
In particular, describe the results of the following expressions:
fn1() ~ 2
fn2() ~ 2
fn1() @ 3
fn2() @ 3
1.4.3. Give one reason you might want to add a “pseudo-sound-expression” score-begin-end event in a score.
In FM Synthesis, the depth of modulation (D) and the frequency of modulation (M) control the number of generated significant sideband pairs together, using the following formula: IM = D / M. We can measure depth of modulation in Hz, which means how many Hz does the carrier frequency deviates. Thus, the result of D / M is a unit-less number, called the Index of Modulation, or IM. When D = 0, there is no sideband generated (see Roads pp-229 for detail).
2.1. Create an FM instrument that takes the following keyword parameters:
pitch: the carrier frequency in steps (a number)
im: a time-varying index of modulation (of type sound)
You may use additional parameters to control loudness (vel:), vibrato, C:M ratio, envelope parameters, etc. The stretch factor should allow you to control the duration of the instrument sound.
An example that plays the instrument is:
play my-fm-instrument(pitch: g4, im: const(0, 1)) ~ 2
In this example, the IM parameter is the constant zero with a duration of 1, so this is expected to play a boring sine tone at pitch G4.
2.2. Demonstrate the instrument by using PWL to create an interesting Index of Modulation. In other words, replace const(0, 1) in the previous example with an interesting, time-varying IM. You can use the envelope editor in the Nyquist IDE to draw an envelope if you like, and you can apply scale factors and offsets to the PWL function to fine tune your sonic result.
2.3. Finally, for a (hopefully) more interesting composition, we are going to analyze a real sound, extract the time-varying spectral centroid, and use that to control the Index of Modulation of one or more FM sounds.
Read the documentation on spectral centroids in the accompanying files, and try out the testcode that is provided. To summarize, you call spectral-centroid() on an input sound (or filename), and it returns an envelope that tracks the spectral centroid of the input sound.
The idea here is that when the input sound gets brighter, the spectral centroid goes up. If we use spectral centroid to control the Index of Modulation, an increasing spectral centroid should cause the Index of Modulation to increase, and therefore the FM sound should get brighter. Thus, there should be a connection between spectral variation of the source sound analyzed with spectral-centroid() and the spectral variation of your synthesized sound.
Your technical task is to make this connection between input sound and output sound. This is a rare case where we are going to suggest assigning the spectral centroid (sound) to a global variable. If you do that, then any note can reference the spectral centroid. For example:
set *sc* = 0.003 * spectral-centroid(...)
play seq(my-fm-instrument(pitch: c4, im: *sc*),
my-fm-instrument(pitch: c4, im: *sc*) ~ 2)
This plays two notes. The first runs nominally from 0s to 1s, and it will use the first second of the spectral centroid sound to control its IM. The second note runs nominally from 1 to 3s (the duration is 2 because of the stretch operator ~), and this note will use the spectral centroid from 1 to 3s. It is important to note that the second note does not begin "reading" the *sc* variable from the beginning.
Note: You could align the beginning of the spectral centroid with the beginning of the note by replacing *sc* with cue(*sc*). The cue behavior shifts its sound parameter to start at the time specified in the environment. However, we think it will be more interesting to let the IM evolve throughout your piece and let each note access the current state of the spectral centroid as the piece is being playing.
Note 2: Why did we multiply spectral-centroid(...) by 0.003? The point is that spectral-centroid is measured in Hz and will probably range into the thousands, but reasonable values of IM are probably going to be in the single digits. Is 0.003 the right scale factor? No, it is just a rough guess, and you should adjust it.
Note 3: You can use a score here rather than SEQ.
Your musical task is to create something interesting, with a duration of at least 30 seconds. We do not want to box you in to a specific procedure, but we are looking for a result that shows interesting spectral variation driven by the spectral centroid of a source sound. Some possibilities include:
Make a melody and use the spectral centroid to control modulation.
Use your voice to produce a sound for analysis by spectral centroid.
If the spectral centroid is not as smoothly varying as you want it to be, consider using a low-pass filter, e.g. lowpass8(*sc*, 10) will remove most frequencies in *sc* above 10 Hz, leaving you with a smoother control function. (See the project3-demo.sal file.)
Rather than a melody, use long, overlapping tones to create a thick texture.
Use a mix of centroid-modulated FM tones and other tones that are not modulated. Maybe the modulated tones are the foreground, and unmodulated tones form a sustained background. Maybe the modulated tones are a sustained "chorus" and an unmodulated melody or contrasting sound is in the foreground.
The FM depth of modulation can be the product of a local envelope (to give some characteristic shape to every tone) multiplied by the global spectral centroid (to provide some longer-term evolution and variety).
Add some reverb, stereo panning (see the PAN function), or other effects.
Experiment with different source sounds. Rock music is likely to have a fairly constant high centroid. Random sounds like traffic noise will probably have low centroids punctuated by high moments. Piano music will likely have centroid peaks every time a note is struck. Choose sources and modify the result with a smoothing lowpass filter (see 3 above) if needed to get the most interesting results
9) You may use an audio editor for extending/processing/mixing your piece, but we must be able to hear clearly your instrument, score, and spectral-centroid-based modulation.
Grading will be based on meeting the technical specifications of the assignment: the code should work, it should be readable, and it should have well-chosen parameters that indicate you understand what all the parameters mean. Regurgitating minor variations of the sample code does not indicate understanding and may result in large grading penalties. It's best to envision what you want to do, write it down in comments, then implement your design. You can refer to the sample code, but should avoid using that as a template for your project -- make this your own creation. In addition, your composition for part 3 should be at least 30 seconds long, the FM index of modulation control should be clearly audible and clearly related to the sound that you analyzed. Up to 10% of the grade may be based on musical results, effective FM instrument design, novel control and composition strategies, mixing quality, and general effort. Bonus points may also be awarded for exceptional work.
Code for part 1 and 2, named andrewid-p3-fm-instrument.sal
Code for part 3, named andrewid-p3-fm-sc-comp.sal
Note: Code should be clearly commented in case we want to run it.
Audio for part 2, named andrewid-p3-fm-instrument.wav
Audio for part 3, named andrewid-p3-fm-sc-comp.wav
Input sound analyzed for part 3, named andrewid-p3-sound.wav
A short statement of your intention of the composition, named
andrewid-p3-sc-comp.txt
Answer for the short questions, named andrewid-p3.txt
Please do NOT use .doc or .docx for the short questions.