International Audio Laboratories Erlangen

MTD: A Multimodal Dataset of Musical Themes for MIR Research

This website presents the Musical Theme Dataset (MTD), which is described in the following paper:

  1. Frank Zalkow, Stefan Balke, Vlora Arifi-Müller, and Meinard Müller
    MTD: A Multimodal Dataset of Musical Themes for MIR Research
    Transactions of the International Society for Music Information Retrieval (TISMIR), 3(1): 180–192, 2020. PDF Details Demo DOI 
    @article{ZalkowBAM20_MTD_TISMIR,
title     = {{MTD}: A Multimodal Dataset of Musical Themes for {MIR} Research},
author    = {Frank Zalkow and Stefan Balke and Vlora Arifi-M{\"{u}}ller and Meinard M{\"{u}}ller},
journal   = {Transactions of the International Society for Music Information Retrieval ({TISMIR})},
volume    = {3},
number    = {1},
year      = {2020},
pages     = {180--192},
doi       = {10.5334/tismir.68},
url-demo  = {https://www.audiolabs-erlangen.de/resources/MIR/MTD},
url-details = {https://transactions.ismir.net/articles/10.5334/tismir.68/},
url-pdf   = {https://www.audiolabs-erlangen.de/fau/assistant/zalkow/publications/2020_ZalkowBAM20_MTD_TISMIR.pdf}
}

Abstract

Musical themes are essential elements in Western classical music. In this paper, we present the Musical Theme Dataset (MTD), a multimodal dataset inspired by “A Dictionary of Musical Themes” by Barlow and Morgenstern from 1948. For a subset of 2048 themes of the printed book, we created several digital representations of the musical themes. Beyond graphical sheet music, we provide symbolic music encodings, audio snippets of music recordings, alignments between the symbolic and audio representations, as well as detailed metadata on the composer, work, recording, and musical characteristics of the themes. In addition to the data, we also make several parsers and web-based interfaces available to access and explore the different modalities and their relations through visualizations and sonifications. These interfaces also include computational tools, bridging the gap between the original dictionary and music information retrieval (MIR) research. The dataset is of relevance for various subfields and tasks in MIR, such as cross-modal music retrieval, music alignment, optical music recognition, music transcription, and computational musicology.

Access and Tools

We provide access to the MTD in three different ways. On our overview website, we present all themes and modalities of the MTD. We also offer a Jupyter notebook containing Python code for parsing, visualizing, and sonifying the data. Finally, we provide a ZIP archive for downloading that contains the raw data. In the list below, we link an HTML export of the Jupyter notebook. The executable notebook is included in the ZIP archive.

MTD overview website Jupyter notebook (HTML export) Raw data (ZIP file, 2.7 GB)

Furthermore, we also provide a custom tool for aligning the symbolic and audio representations in the MTD. This web tool is not a general-purpose product, but its source code may be useful for the MTD users who want to further refine the alignments or add further themes to the dataset.

Alignment tool

Acknowledgements

This work was supported by the German Research Foundation (DFG MU 2686/11-1, DFG MU 2686/12-1). The International Audio Laboratories Erlangen are a joint institution of the Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU) and Fraunhofer Institut für Integrierte Schaltungen IIS. We thank Lena Krauß, Lukas Lamprecht, Anna-Luisa Römling, and Quirin Seilbeck for helping us with the annotations.

References

  1. Harold Barlow and Sam Morgenstern
    A Dictionary of Musical Themes
    Crown Publishers, Inc., 1975. 
    @book{BarlowM75_MusicalThemes_BOOK,
Author    = {Harold Barlow and Sam Morgenstern},
Edition   = {Revised edition Third Printing},
Publisher = {Crown Publishers, Inc.},
Title     = {A Dictionary of Musical Themes},
Year      = {1975}
}
  2. Stefan Balke, Sanu Pulimootil Achankunju, and Meinard Müller
    Matching Musical Themes based on Noisy OCR and OMR Input
    In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP): 703–707, 2015. 
    @inproceedings{BalkePM15_MatchingMusicalThemes_ICASSP,
author    = {Stefan Balke and Sanu Pulimootil Achankunju and Meinard M{\"u}ller},
title     = {Matching Musical Themes based on Noisy {OCR} and {OMR} Input},
booktitle = {Proceedings of the {IEEE} International Conference on Acoustics, Speech, and Signal Processing ({ICASSP})},
address   = {Brisbane, Australia},
year      = {2015},
pages     = {703--707}
}
  3. Stefan Balke, Vlora Arifi-Müller, Lukas Lamprecht, and Meinard Müller
    Retrieving Audio Recordings Using Musical Themes
    In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP): 281–285, 2016. 
    @inproceedings{BalkeALM16_BarlowRetrieval_ICASSP,
author    = {Stefan Balke and Vlora Arifi-M{\"u}ller and Lukas Lamprecht and Meinard M{\"u}ller},
title     = {Retrieving Audio Recordings Using Musical Themes},
booktitle = {Proceedings of the {IEEE} International Conference on Acoustics, Speech, and Signal Processing ({ICASSP})},
address   = {Shanghai, China},
year      = {2016},
pages     = {281--285},
}
  4. Frank Zalkow, Stefan Balke, and Meinard Müller
    Evaluating Salience Representations for Cross-Modal Retrieval of Western Classical Music Recordings
    In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP): 311–335, 2019. 
    @inproceedings{ZalkowBM19_SalienceRetrieval_ICASSP,
author      = {Frank Zalkow and Stefan Balke and Meinard M{\"u}ller},
title       = {Evaluating Salience Representations for Cross-Modal Retrieval of {W}estern Classical Music Recordings},
booktitle   = {Proceedings of the {IEEE} International Conference on Acoustics, Speech, and Signal Processing ({ICASSP})},
address     = {Brighton, United Kingdom},
year        = {2019},
pages       = {311--335},
}
  5. Frank Zalkow and Meinard Müller
    Using Weakly Aligned Score—Audio Pairs to Train Deep Chroma Models for Cross-Modal Music Retrieval
    In Proceedings of the International Conference on Music Information Retrieval (ISMIR): 184–191, 2020. 
    @inproceedings{ZalkowM20_BarlowCTC_ISMIR,
author      = {Frank Zalkow and Meinard M{\"u}ller},
title       = {Using Weakly Aligned Score--Audio Pairs to Train Deep Chroma Models for Cross-Modal Music Retrieval},
booktitle   = {Proceedings of the International Conference on Music Information Retrieval ({ISMIR})},
address     = {Montr{\'e}al, Canada},
year        = {2020},
pages       = {184--191}
}