This is the accompanying website for the paper "Extracting Singing Voice from Music Recordings by Cascading Audio Decomposition Techniques" by Jonathan Driedger and Meinard Müller [pdf][bib].

Abstract

The problem of extracting singing voice from music recordings has received increasing research interest in recent years. Many proposed decomposition techniques are based on one of the following two strategies. The first approach is to directly decompose a given music recording into one component for the singing voice and one for the accompaniment by exploiting knowledge about specific characteristics of singing voice. Procedures following the second approach disassemble the recording into a large set of fine-grained components, which are classified and reassembled afterwards to yield the desired source estimates. In this paper, we propose a novel approach that combines the strengths of both strategies. We first apply different audio decomposition techniques in a cascaded fashion to disassemble the music recording into a set of mid-level components. This decomposition is fine enough to model various characteristics of singing voice, but coarse enough to keep an explicit semantic meaning of the components. These properties allow us to directly reassemble the singing voice and the accompaniment from the components. Our objective and subjective evaluations show that this strategy can compete with state-of-the-art singing voice separation algorithms and yields perceptually appealing results.

Audio Examples

In the following we present audio examples to demonstrate the capabilities of our proposed procedure. You can choose an example from the list and click on the stylized spectrograms to listen to the audio files.

SiSEC: Bearlin


Original
recording

Harmonic
component

Percussive
component

Residual
component

Harmonic portion
singing voice

Harmonic portion
accompaniment

Fricatives
singing voice

Instrument onsets
accompaniment

Vibrato & formants
singing voice

Diffuse instr. sounds
accompaniment

Estimate
singing voice

Estimate
accompaniment

SiSEC: Tamy


Original
recording

Harmonic
component

Percussive
component

Residual
component

Harmonic portion
singing voice

Harmonic portion
accompaniment

Fricatives
singing voice

Instrument onsets
accompaniment

Vibrato & formants
singing voice

Diffuse instr. sounds
accompaniment

Estimate
singing voice

Estimate
accompaniment

SiSEC: Another Dreamer


Original
recording

Harmonic
component

Percussive
component

Residual
component

Harmonic portion
singing voice

Harmonic portion
accompaniment

Fricatives
singing voice

Instrument onsets
accompaniment

Vibrato & formants
singing voice

Diffuse instr. sounds
accompaniment

Estimate
singing voice

Estimate
accompaniment

SiSEC: Fort Minor


Original
recording

Harmonic
component

Percussive
component

Residual
component

Harmonic portion
singing voice

Harmonic portion
accompaniment

Fricatives
singing voice

Instrument onsets
accompaniment

Vibrato & formants
singing voice

Diffuse instr. sounds
accompaniment

Estimate
singing voice

Estimate
accompaniment

SiSEC: Ultimate NZ Tour


Original
recording

Harmonic
component

Percussive
component

Residual
component

Harmonic portion
singing voice

Harmonic portion
accompaniment

Fricatives
singing voice

Instrument onsets
accompaniment

Vibrato & formants
singing voice

Diffuse instr. sounds
accompaniment

Estimate
singing voice

Estimate
accompaniment

Classical Opera: Weber - Der Freischütz


Original
recording

Harmonic
component

Percussive
component

Residual
component

Harmonic portion
singing voice

Harmonic portion
accompaniment

Fricatives
singing voice

Instrument onsets
accompaniment

Vibrato & formants
singing voice

Diffuse instr. sounds
accompaniment

Estimate
singing voice

Estimate
accompaniment

Romantic Piano Music: Schubert - Gute Nacht (Winterreise)


Original
recording

Harmonic
component

Percussive
component

Residual
component

Harmonic portion
singing voice

Harmonic portion
accompaniment

Fricatives
singing voice

Instrument onsets
accompaniment

Vibrato & formants
singing voice

Diffuse instr. sounds
accompaniment

Estimate
singing voice

Estimate
accompaniment

Choire: Sunshine Garcia Band - For I am the Moon


Original
recording

Harmonic
component

Percussive
component

Residual
component

Harmonic portion
singing voice

Harmonic portion
accompaniment

Fricatives
singing voice

Instrument onsets
accompaniment

Vibrato & formants
singing voice

Diffuse instr. sounds
accompaniment

Estimate
singing voice

Estimate
accompaniment

Pop: Mystery Mike - Mike Senior


Original
recording

Harmonic
component

Percussive
component

Residual
component

Harmonic portion
singing voice

Harmonic portion
accompaniment

Fricatives
singing voice

Instrument onsets
accompaniment

Vibrato & formants
singing voice

Diffuse instr. sounds
accompaniment

Estimate
singing voice

Estimate
accompaniment

Metal: Acheronita - Human


Original
recording

Harmonic
component

Percussive
component

Residual
component

Harmonic portion
singing voice

Harmonic portion
accompaniment

Fricatives
singing voice

Instrument onsets
accompaniment

Vibrato & formants
singing voice

Diffuse instr. sounds
accompaniment

Estimate
singing voice

Estimate
accompaniment

MIREX: Train01


Original
recording

Harmonic
component

Percussive
component

Residual
component

Harmonic portion
singing voice

Harmonic portion
accompaniment

Fricatives
singing voice

Instrument onsets
accompaniment

Vibrato & formants
singing voice

Diffuse instr. sounds
accompaniment

Estimate
singing voice

Estimate
accompaniment

MIREX: Train03


Original
recording

Harmonic
component

Percussive
component

Residual
component

Harmonic portion
singing voice

Harmonic portion
accompaniment

Fricatives
singing voice

Instrument onsets
accompaniment

Vibrato & formants
singing voice

Diffuse instr. sounds
accompaniment

Estimate
singing voice

Estimate
accompaniment

MIREX: Train05


Original
recording

Harmonic
component

Percussive
component

Residual
component

Harmonic portion
singing voice

Harmonic portion
accompaniment

Fricatives
singing voice

Instrument onsets
accompaniment

Vibrato & formants
singing voice

Diffuse instr. sounds
accompaniment

Estimate
singing voice

Estimate
accompaniment

Audio Files from the Objective and Subjective Evaluation

You can download all audio files which were used in the objective and subjective evaluation here. Partially, these files were obtained from the SiSEC 2013 website.