International Audio Laboratories Erlangen

Unifying Local and Global Methods for Harmonic-Percussive Source Separation

This is the accompanying website for the following paper:

  1. Christian Dittmar, Patricio López-Serrano, and Meinard Müller
    Unifying Local and Global Methods for Harmonic-Percussive Source Separation
    In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2018. Demo DOI 
    @inproceedings{DittmarLM18_HPSS_KAM_NMF_ICASSP,
author    = {Christian Dittmar and Patricio L{\'o}pez-Serrano and Meinard M{\"u}ller},
title     = {Unifying Local and Global Methods for Harmonic-Percussive Source Separation},
booktitle = {Proceedings of the {IEEE} International Conference on Acoustics, Speech, and Signal Processing ({ICASSP})},
address   = {Calgary, Canada},
month     = {April},
year      = {2018},
doi       = {https://ieeexplore.ieee.org/document/8462119},
url-demo  = {https://www.audiolabs-erlangen.de/resources/MIR/2018-ICASSP-HPSS_KAM_NMF},
}

Abstract

This paper addresses the separation of drums from music recordings, a task closely related to harmonic-percussive source separation (HPSS). In previous works, two families of algorithms have been prominently applied to this problem. They are based either on local filtering and diffusion schemes or on global low-rank models. In this paper, we propose to combine the advantages of both paradigms. To this end, we use a local approach based on Kernel Additive Modeling (KAM) [3,4] to extract an initial guess for the percussive and harmonic components. Subsequently, we use constrained Non-Negative Matrix Factorization (NMF) [6] as a global approach to jointly enhance both estimates.
On this companion webpage, we present excerpts from real-world music recordings as well as examples from our test to illustrate the benefits and limitations of our proposed method and comparison algorithms that have been previously proposed by other authors.
Each example page below delivers a short description of the music and comments on the separation quality. Furthermore, we show an animation how the the NMF activations are iteratively refined with our proposed method. Finally, an audio player using trackswitch.js allows to directly compare the separation quality attained by the different HPSS algorithms.
Let there be drums
This figure shows an idealized example of how the NMF activations are iteratively refined with our proposed method [1]. For the sake of clarity, we used score-informed initialization of the NMF components as described in [2]. On the audio example pages linked below, we are going to present similar animations for each item. In contrast to this animation, the NMF components will be initialized randomly, without using score-information.

Audio Examples

Excerpts from real-world music recordings:
[I Got You] [Feeling Good] [He Ain't Heavy] [Good Bait] [Tak dej se k nám a projdem svět]
[Caravan] [Yègellé tezeta] [Unwind Yourself] [Svlíkám lásku] [Schreib es mir in den Sand]
Excerpts from our test corpora:
[Test item 10] [Test item 23] [Test item 28] [Test item 70] [Test item 73]

References

  1. Christian Dittmar, Patricio López-Serrano, and Meinard Müller
    Unifying Local and Global Methods for Harmonic-Percussive Source Separation
    In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2018. 
    @inproceedings{DittmarLM18_HPSS_KAM_NMF_ICASSP,
author    = {Christian Dittmar and Patricio L{\'o}pez-Serrano and Meinard M{\"u}ller},
title     = {Unifying Local and Global Methods for Harmonic-Percussive Source Separation},
booktitle = {Proceedings of the {IEEE} International Conference on Acoustics, Speech, and Signal Processing ({ICASSP})},
address   = {Calgary, Canada},
month     = {April},
note      = {accepted for publication},
year      = {2018},
}
  2. Sebastian Ewert, Bryan Pardo, Meinard Müller, and Mark Plumbley
    Score-Informed Source Separation for Musical Audio Recordings: An overview
    IEEE Signal Processing Magazine, 31(3): 116–124, 2014. PDF 
    @article{EwertPMP14_SourceSeparation_IEEE-SPS,
author    = {Sebastian Ewert and Bryan Pardo and Meinard M{\"u}ller and Mark Plumbley},
title     = {Score-Informed Source Separation for Musical Audio Recordings: An overview},
journal   = {IEEE Signal Processing Magazine},
volume    = {31},
number    = {3},
year      = {2014},
month     = {April},
pages     = {116--124},
url-pdf   = {http://ieeexplore.ieee.org/document/6784086/},
}
  3. Derry FitzGerald, Antoine Liutkus, Zafar Rafii, Bryan Pardo, and Laurent Daudet
    Harmonic/Percussive Separation Using Kernel Additive Modelling
    In Irish Signals and Systems Conference (IET): 35–40, 2014. PDF 
    @inproceedings{FitzgeraldLRPD14_KAMHP_IET,
title     = {Harmonic/Percussive Separation Using {K}ernel {A}dditive {M}odelling},
author    = {Derry FitzGerald and Antoine Liutkus and  Zafar Rafii and Bryan Pardo and Laurent Daudet},
booktitle = {Irish Signals and Systems Conference ({IET})},
pages     = {35--40},
address   = {Limerick, Ireland},
year      = {2014},
url-pdf   = {http://ieeexplore.ieee.org/document/6912726/},
}
  4. Christian Dittmar, Jonathan Driedger, Meinard Müller, and Jouni Paulus
    An Experimental Approach to Generalized Wiener Filtering in Music Source Separation
    In Proceedings of the European Signal Processing Conference (EUSIPCO), 2016. PDF 
    @inproceedings{DittmarDMP16_WienerFiltering_EUSIPCO,
author    = {Christian Dittmar and Jonathan Driedger and Meinard M{\"u}ller and Jouni Paulus},
title     = {An Experimental Approach to Generalized Wiener Filtering in Music Source Separation},
booktitle = {Proceedings of the European Signal Processing Conference ({EUSIPCO})},
address   = {Budapest, Hungary},
year      = {2016},
pages     = {},
url-pdf   = {http://ieeexplore.ieee.org/document/7760547/},
}
  5. Derry FitzGerald
    Harmonic/Percussive Separation Using Median Filtering
    In Proceedings of the International Conference on Digital Audio Effects (DAFx): 246–253, 2010. PDF 
    @inproceedings{Fitzgerald10_HarmPercSep_DAFX,
author      = {Derry FitzGerald},
title       = {Harmonic/Percussive Separation Using Median Filtering},
booktitle   = {Proceedings of the International Conference on Digital Audio Effects ({DAFx})},
address     = {Graz, Austria},
year        = {2010},
pages       = {246--253},
url-pdf     = {http://dafx10.iem.at/papers/DerryFitzGerald_DAFx10_P15.pdf},
}
  6. Jeongsoo Park, Jaeyoung Shin, and Kyogu Lee
    Exploiting Continuity/Discontinuity of Basis Vectors in Spectrogram Decomposition for Harmonic-Percussive Sound Separation
    IEEE/ACM Transactions on Audio, Speech, and Language Processing, 25(5): 1061–1074, 2017. PDF 
    @Article{ParkSL17_HPSS_NMF_IEEE-TASLP,
author={Jeongsoo Park and Jaeyoung Shin and Kyogu Lee},
journal={{IEEE/ACM} Transactions on Audio, Speech, and Language Processing},
title={Exploiting Continuity/Discontinuity of Basis Vectors in Spectrogram Decomposition for Harmonic-Percussive Sound Separation},
year={2017},
volume={25},
number={5},
pages={1061--1074},
month={May},
url-pdf = {http://ieeexplore.ieee.org/document/7876808/},
}
  7. Antoine Liutkus and Roland Badeau
    Generalized Wiener filtering with fractional power spectrograms
    In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP): 266–270, 2015. PDF 
    @inproceedings{LiutkusB15_WienerFilter_ICASSP,
author = {Antoine Liutkus and Roland Badeau},
booktitle = {Proceedings of the {IEEE} International Conference on Acoustics, Speech and Signal Processing ({ICASSP})},
title = {Generalized Wiener filtering with fractional power spectrograms},
year = {2015},
month = {April},
pages = {266--270},
address = {Brisbane, Australia},
url-pdf = {http://ieeexplore.ieee.org/document/7177973/},
}