Abstract
The problem of extracting singing voice from music recordings has received increasing research interest in recent years. Many proposed decomposition techniques are based on one of the following two strategies. The first approach is to directly decompose a given music recording into one component for the singing voice and one for the accompaniment by exploiting knowledge about specific characteristics of singing voice. Procedures following the second approach disassemble the recording into a large set of fine-grained components, which are classified and reassembled afterwards to yield the desired source estimates. In this paper, we propose a novel approach that combines the strengths of both strategies. We first apply different audio decomposition techniques in a cascaded fashion to disassemble the music recording into a set of mid-level components. This decomposition is fine enough to model various characteristics of singing voice, but coarse enough to keep an explicit semantic meaning of the components. These properties allow us to directly reassemble the singing voice and the accompaniment from the components. Our objective and subjective evaluations show that this strategy can compete with state-of-the-art singing voice separation algorithms and yields perceptually appealing results.
Audio Examples
In the following we present audio examples to demonstrate the capabilities of our proposed procedure. You can choose an example from the list and click on the stylized spectrograms to listen to the audio files.
SiSEC: Bearlin
Original
recording
Harmonic
component
Percussive
component
Residual
component
Harmonic portion
singing voice
Harmonic portion
accompaniment
Fricatives
singing voice
Instrument onsets
accompaniment
Vibrato & formants
singing voice
Diffuse instr. sounds
accompaniment
Estimate
singing voice
Estimate
accompaniment
SiSEC: Tamy
Original
recording
Harmonic
component
Percussive
component
Residual
component
Harmonic portion
singing voice
Harmonic portion
accompaniment
Fricatives
singing voice
Instrument onsets
accompaniment
Vibrato & formants
singing voice
Diffuse instr. sounds
accompaniment
Estimate
singing voice
Estimate
accompaniment
SiSEC: Another Dreamer
Original
recording
Harmonic
component
Percussive
component
Residual
component
Harmonic portion
singing voice
Harmonic portion
accompaniment
Fricatives
singing voice
Instrument onsets
accompaniment
Vibrato & formants
singing voice
Diffuse instr. sounds
accompaniment
Estimate
singing voice
Estimate
accompaniment
SiSEC: Fort Minor
Original
recording
Harmonic
component
Percussive
component
Residual
component
Harmonic portion
singing voice
Harmonic portion
accompaniment
Fricatives
singing voice
Instrument onsets
accompaniment
Vibrato & formants
singing voice
Diffuse instr. sounds
accompaniment
Estimate
singing voice
Estimate
accompaniment
SiSEC: Ultimate NZ Tour
Original
recording
Harmonic
component
Percussive
component
Residual
component
Harmonic portion
singing voice
Harmonic portion
accompaniment
Fricatives
singing voice
Instrument onsets
accompaniment
Vibrato & formants
singing voice
Diffuse instr. sounds
accompaniment
Estimate
singing voice
Estimate
accompaniment
Classical Opera: Weber - Der Freischütz
Original
recording
Harmonic
component
Percussive
component
Residual
component
Harmonic portion
singing voice
Harmonic portion
accompaniment
Fricatives
singing voice
Instrument onsets
accompaniment
Vibrato & formants
singing voice
Diffuse instr. sounds
accompaniment
Estimate
singing voice
Estimate
accompaniment
Romantic Piano Music: Schubert - Gute Nacht (Winterreise)
Original
recording
Harmonic
component
Percussive
component
Residual
component
Harmonic portion
singing voice
Harmonic portion
accompaniment
Fricatives
singing voice
Instrument onsets
accompaniment
Vibrato & formants
singing voice
Diffuse instr. sounds
accompaniment
Estimate
singing voice
Estimate
accompaniment
Choire: Sunshine Garcia Band - For I am the Moon
Original
recording
Harmonic
component
Percussive
component
Residual
component
Harmonic portion
singing voice
Harmonic portion
accompaniment
Fricatives
singing voice
Instrument onsets
accompaniment
Vibrato & formants
singing voice
Diffuse instr. sounds
accompaniment
Estimate
singing voice
Estimate
accompaniment
Pop: Mystery Mike - Mike Senior
Original
recording
Harmonic
component
Percussive
component
Residual
component
Harmonic portion
singing voice
Harmonic portion
accompaniment
Fricatives
singing voice
Instrument onsets
accompaniment
Vibrato & formants
singing voice
Diffuse instr. sounds
accompaniment
Estimate
singing voice
Estimate
accompaniment
Metal: Acheronita - Human
Original
recording
Harmonic
component
Percussive
component
Residual
component
Harmonic portion
singing voice
Harmonic portion
accompaniment
Fricatives
singing voice
Instrument onsets
accompaniment
Vibrato & formants
singing voice
Diffuse instr. sounds
accompaniment
Estimate
singing voice
Estimate
accompaniment
MIREX: Train01
Original
recording
Harmonic
component
Percussive
component
Residual
component
Harmonic portion
singing voice
Harmonic portion
accompaniment
Fricatives
singing voice
Instrument onsets
accompaniment
Vibrato & formants
singing voice
Diffuse instr. sounds
accompaniment
Estimate
singing voice
Estimate
accompaniment
MIREX: Train03
Original
recording
Harmonic
component
Percussive
component
Residual
component
Harmonic portion
singing voice
Harmonic portion
accompaniment
Fricatives
singing voice
Instrument onsets
accompaniment
Vibrato & formants
singing voice
Diffuse instr. sounds
accompaniment
Estimate
singing voice
Estimate
accompaniment
MIREX: Train05
Original
recording
Harmonic
component
Percussive
component
Residual
component
Harmonic portion
singing voice
Harmonic portion
accompaniment
Fricatives
singing voice
Instrument onsets
accompaniment
Vibrato & formants
singing voice
Diffuse instr. sounds
accompaniment
Estimate
singing voice
Estimate
accompaniment
Audio Files from the Objective and Subjective Evaluation
You can download all audio files which were used in the objective and subjective evaluation here. Partially, these files were obtained from the SiSEC 2013 website.