Concatenative Synthesis Using Score-Aligned Transcription
MIDI files are automatically aligned to audio files to provide a
transcription of the audio. The audio file can then be segmented at
note onsets and offsets to create a sequence of
sound units. Each unit is labeled with the set of
pitches that are sounding. This database can then be used to synthesize
a MIDI file. The MIDI file is segmented in a similar way to obtain a
sequence of chords. The database is searched for matching chords which
are output.
The result preserves the approximate harmony and pitch content (to the
extent that good matches are found in the database), but the actual
timbres depend on what is in the database. The juxtaposition of sound
units can be pretty chaotic but strangely in tune with the original
harmonic progression.
Examples:
Bach Chorale Synthesized from Beethoven's Fifth
In this example, a Bach
chorale, performed on piano is recreated with sounds from Beethoven's
Fifth Symphony,
Movement 1. The two versions are cross-faded back and forth for
contrast and
comparison.
More Bach Synthesized from Beethoven
In this example,
a piece by Bach is synthesized from Beethoven's Fifth Symphony. The
melody from the Bach is synthesized
with a conventional synthesizer and mixed in with the concatenative
synthesis output to from a rather strangely accompanied melody. (Music
scholars: yes, the Bach was originally in B minor. I transposed to C
minor to maximize the chords in common with Beethoven.)