International Audio Laboratories Erlangen

Common Fate Model for Unison Source Separation

Abstract

In this paper we present a novel source separation method aiming to overcome the difficulty of modeling non-stationary signals. The method can be applied to mixtures of musical instruments with frequency and/or amplitude modulation, e.g.\ typically caused by vibrato. It is based on a signal representation that divides the complex spectrogram into a grid of patches of arbitrary size. These complex patches are then processed by a two-dimensional discrete Fourier transform, forming a tensor representation which reveals spectral and temporal modulation textures. Our representation can be seen as an alternative to modulation transforms computed on magnitude spectrograms. An adapted factorization model allows to decompose different time-varying harmonic sources based on their particular common modulation profile: hence the name Common Fate Model. The method is evaluated on musical instrument mixtures playing the same fundamental frequency (unison), showing improvement over other state-of-the-art methods.

Common Fate Transform

cft

The Common Fate Transform is based on a signal representation that divides a complex spectrogram into a grid of patches of arbitrary size. These complex patches are then processed by a two-dimensional discrete Fourier transform, forming a tensor representation which reveals spectral and temporal modulation textures.

Common Fate Model

cfm

An adapted factorization model similar to the PARAFAC/CANDECOMP factorisation allows to decompose the common fate transform tensor into different time-varying harmonic sources based on their particular common modulation profile: hence the name Common Fate Model.

Demo

Viola + Flute

42-73_mix

Saxofone + Flute

66-73_mix

Dataset

Python Implementation

A package is a python implementation of the Common Fate Transform and Model to be used for audio source separation is available by Antoine Liutkus.

Further Information

Cite

@inproceedings{stoeter2016icassp,
  author    = {F.-R. St{\"o}ter, A. Liutkus, R. Badeau, B. Edler and P. Magron},
  title     = {Common Fate Model for Unison Source Separation},
  booktitle = {International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
  address   = {Shanghai, China},
  year      = {2016},
}