International Audio Laboratories Erlangen

Weakly Supervised Multi-Pitch Estimation Using Cross-Version Alignment

This is the accompanying website for the following paper:

1. Michael Krause, Sebastian Strahl, and Meinard Müller
   Weakly Supervised Multi-Pitch Estimation Using Cross-Version Alignment
   In Proceedings of the International Society for Music Information Retrieval Conference (ISMIR), 2023. Bibtex

   @inproceedings{KrauseSM23_WeaklySupervisedMPE_ISMIR,
   author    = {Michael Krause and Sebastian Strahl and Meinard M{\"u}ller},
   title     = {Weakly Supervised Multi-Pitch Estimation Using Cross-Version Alignment},
   booktitle = {Proceedings of the International Society for Music Information Retrieval Conference ({ISMIR})},
   address   = {Milano, Italy},
   year      = {2023}
   }

Abstract

Multi-pitch estimation (MPE), the task of detecting active pitches within a polyphonic music recording, has garnered significant research interest in recent years. Most state-of-the-art approaches for MPE are based on deep networks trained using pitch annotations as targets. The success of current methods is therefore limited by the difficulty of obtaining large amounts of accurate annotations. In this paper, we propose a novel technique for learning MPE without any pitch annotations at all. Our approach exploits multiple recorded versions of a musical piece as surrogate targets. Given one version of a piece as input, we train a network to minimize the distance between its output and time-frequency representations of other versions of that piece. Since all versions are based on the same musical score, we hypothesize that the learned output corresponds to pitch estimates. To further ensure that this hypothesis holds, we incorporate domain knowledge about overtones and noise levels into the network. Overall, our method replaces strong pitch annotations with weaker and easier-to-obtain cross-version targets. In our experiments, we show that our proposed approach yields viable multi-pitch estimates and outperforms two baselines.

Code and Trained Models

On Github

Acknowledgements

This work was supported by the German Research Foundation (DFG MU 2686/7-2, MU 2686/11-2). The International Audio Laboratories Erlangen are a joint institution of the Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU) and Fraunhofer Institut für Integrierte Schaltungen IIS. The authors gratefully acknowledge the compute resources and support provided by the Erlangen Regional Computing Center (RRZE).

References

1. Marco Cuturi and
   Soft-DTW: a Differentiable Loss Function for Time-Series
   In Proceedings of the International Conference on Machine Learning (ICML): 894–903, 2017. Bibtex

   @inproceedings{CuturiB17_SoftDTW_ICML,
   author    = {Marco Cuturi and
   Mathieu Blondel},
   title     = {Soft-{DTW}: a Differentiable Loss Function for Time-Series},
   booktitle = {Proceedings of the International Conference on Machine Learning ({ICML})},
   address   = {Sydney, NSW, Australia},
   pages     = {894--903},
   year      = {2017},
   url       = {http://proceedings.mlr.press/v70/cuturi17a.html}
   }

2. Christof Weiß and Geoffroy Peeters
   Learning Multi-Pitch Estimation From Weakly Aligned Score-Audio Pairs Using a Multi-Label CTC Loss
   In Proceedings of the IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA): 121–125, 2021. Bibtex

   @inproceedings{WeissP21_MultiPitchMCTC_WASPAA,
   author    = {Christof Wei{\ss} and Geoffroy Peeters},
   title     = {Learning Multi-Pitch Estimation From Weakly Aligned Score-Audio Pairs Using a Multi-Label {CTC} Loss},
   booktitle = {Proceedings of the {IEEE} Workshop on Applications of Signal Processing to Audio and Acoustics ({WASPAA})},
   address   = {New Paltz, USA},
   pages     = {121--125},
   year      = {2021}
   }

3. Michael Krause, Christof Weiß, and Meinard Müller
   Soft Dynamic Time Warping for Multi-Pitch Estimation and Beyond
   In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2023. Bibtex

   @inproceedings{KrauseWM23_SoftDTW_ICASSP,
   author    = {Michael Krause and Christof Wei{\ss} and Meinard M{\"u}ller},
   title     = {Soft Dynamic Time Warping for Multi-Pitch Estimation and Beyond},
   booktitle = {Proceedings of the {IEEE} International Conference on Acoustics, Speech and Signal Processing ({ICASSP})},
   pages     = {},
   address   = {Rhodes Island, Greece},
   year      = {2023}
   }

4. Taylor Berg-Kirkpatrick, Jacob Andreas, and Dan Klein
   Unsupervised Transcription of Piano Music
   In Proceedings of Advances in Neural Information Processing Systems (NIPS): 1538–1546, 2014. Bibtex

   @inproceedings{Berg-KirkpatrickAK14_PianoTranscriptionUnsup_NeurIPS,
   author    = {Taylor Berg{-}Kirkpatrick and Jacob Andreas and Dan Klein},
   title     = {Unsupervised Transcription of Piano Music},
   booktitle = {Proceedings of Advances in Neural Information Processing Systems ({NIPS})},
   address   = {Montr{\'e}al, Canada},
   pages     = {1538--1546},
   year      = {2014},
   url       = {https://proceedings.neurips.cc/paper/2014/hash/3b5dca501ee1e6d8cd7b905f4e1bf723-Abstract.html}
   }

5. Jesse Engel, Lamtharn Hantrakul, Chenjie Gu, and Adam Roberts
   DDSP: Differentiable Digital Signal Processing
   In Proceedings of the International Conference on Learning Representations (ICLR), 2020. Bibtex

   @inproceedings{EngelHGR20_DifferentiableDSP_ICLR,
   title     = {{DDSP}: Differentiable Digital Signal Processing},
   author    = {Jesse Engel and Lamtharn Hantrakul and Chenjie Gu and Adam Roberts},
   booktitle = {Proceedings of the International Conference on Learning Representations ({ICLR})},
   year      = {2020},
   address   = {Virtual},
   url       = {https://openreview.net/forum?id=B1x1ma4tDr},
   }