International Audio Laboratories Erlangen

Computational Analysis of Traditional Georgian Vocal Music (GVM)

Logo_DFG Teaser_GVM Logo_UniPotsdam Logo_FAU

The core mission of the GVM project was to advance ethnomusicological research with a focus on traditional Georgian vocal music (GVM) by using computational methods from audio signal processing and music information retrieval (MIR). This interdisciplinary project was funded by the German Research Foundation. On this website, we summarize the project's main objectives and provide links to project-related resources (data, demonstrators, websites) and publications.

Project Description

Computational Analysis of Traditional Georgian Vocal Music

Teaser_GVM_Erkomaishvili

Georgia has a rich cultural heritage. Its traditional polyphonic vocal music, which has been acknowledged as Intangible Cultural Heritage by the UNESCO in 2001, is one of the most prominent examples. Being an orally transmitted culture, most of the sources are available as field recordings (often with rather poor audio quality). Musicological research using these sources has usually been conducted on the basis of notated musical scores, which were obtained by manually transcribing the audio material. Such approaches are problematic since important tonal cues and performance aspects are likely to get lost in the transcription process. Furthermore, previous studies often suffer from subjectivity and reproducibility issues. In the GVM project, our main objective was to advance ethnomusicological research focusing on traditional Georgian vocal music by employing computational methods from audio signal processing and music information retrieval (MIR). To this end, we considered three main objectives.

  • Our first objective was to improve the understanding of traditional Georgian vocal music by analyzing existing and newly created corpora of field recordings.

  • In the second objective, we aimed at developing novel computational tools for processing and analyzing field recordings of polyphonic singing. Considering the tonal analysis of traditional Georgian vocal music as a concrete application scenario, we explored their potential for corpus-driven research in the humanities.

  • By systematically processing and annotating multimodal collections of field recordings and implementing tools for accessing and analyzing this data using web-based technologies, our third objective was to contribute to the preservation of the rich Georgian musical heritage.

Projektbeschreibung

Computergestützte Analyse traditioneller georgischer Vokalmusik (GVM)

Teaser_GVM_Erkomaishvili

Georgien hat ein reiches kulturelles Erbe. Insbesondere gehört hierzu georgische, mehrstimmig gesungene Vokalmusik, die 2001 von der UNESCO als immaterielles Kulturerbe anerkannt wurde. Aufgrund der mündlich überlieferten Musiktraditionen sind die meisten der historischen Quellen nur in Form von Feldaufnahmen (oft mit eher schlechter Klangqualität) verfügbar. Auf diesen Quellen basierende musikwissenschaftliche Forschung stützt sich oft auf symbolisch notierten Notentexten, die durch manuelle Transkription des Audiomaterials erhalten wurden. Solche Ansätze sind problematisch, da im Transkriptionsprozess für die Aufführungspraxis wichtige (mikro-)tonale Strukturen verloren gehen können. Darüber hinaus leiden frühere Studien oft unter Problemen einer gewissen Subjektivität und fehlenden Reproduzierbarkeit. Vor diesem Hintergrund bestand das allgemeine Ziel des GVM-Projekts darin, musikethnologische Forschung mit Schwerpunkt auf traditionelle georgische Vokalmusik durch Einsatz rechnergestützter Methoden aus den Bereichen der Audiosignalverarbeitung und des Music Information Retrieval (MIR) voranzutreiben. Das Projekt bestand insbesondere aus den folgenden drei Hauptzielen:

  • Erstens haben wir einen neu geschaffenen Korpus von (qualitativ hochwertigen) Feldaufnahmen analysiert und hierdurch ein tieferes Verständnis traditioneller georgischer Vokalmusik entwickelt.

  • Zweitens haben wir durch die Entwicklung neuartiger Werkzeuge und deren Einsatz in einem konkreten Anwendungskontext das Potential computergestützter Methoden für reproduzierbare, Korpus-basierte Forschung innerhalb der Geisteswissenschaften ausgelotet.

  • Unser drittes Ziel bestand darin, durch die systematische Aufarbeitung einer einzigartigen multimodalen Sammlung von Feldaufnahmen sowie durch die Implementierung webbasierter Werkzeuge für den Zugriff und die Analyse dieser Daten zur Erhaltung und Verbreitung des musikalischen Erbes von Georgien beizutragen.

Projected-Related Resources and Publications

In view of reproducible and sustainable research, we have made all relevant data, annotations, and research results publicly available under open-source licenses. Furthermore, for most publications, we provided accompanying websites with additional material in the form of freely available audio samples, visualizations, and sonifications. For demonstration purposes, some websites also integrate web-based interfaces, which allow users to access, navigate, comprehend, and evaluate the data and the results. Furthermore, we have published relevant research code for the GVM project in the form of well-documented toolboxes and Jupyter notebooks under open-source licenses. Subsequently, we provide links to the resources and publications of the GVM project organized along the following four subtopics.

Computational Analysis of Erkomaishvili Corpus

As a major contribution of our project, we examined the tonal organization of a series of recordings of liturgical chants, sung in 1966 by the Georgian master chanter Artem Erkomaishvili. There is consensus among ethnomusicologists that this corpus is of outstanding importance for understanding the tuning principles of traditional Georgian vocal music.

  • Erkomaishvili dataset and web-based interface with score-following functionality
  • F0-annotations of Erkomashvili recordings
  • Erkomaishvili dataset on Zenodo
  • YouTube videos of Erkomaishvili recordings with score-following functionality
  • Publications
    1. Frank Scherbaum, Nana Mzhavanadze, Simha Arom, Sebastian Rosenzweig, and Meinard Müller
      Tonal Organization of the Erkomaishvili Dataset: Pitches, Scales, Melodies and Harmonies
      Universitätsverlag Potsdam, 2020. PDF Details DOI 
      @book{ScherbaumMARM20_Erkomaishvili_UniPotsdam,
author      = {Frank Scherbaum and Nana Mzhavanadze and Simha Arom and Sebastian Rosenzweig and Meinard M{\"u}ller},
title       = {Tonal Organization of the {E}rkomaishvili Dataset: {P}itches, Scales, Melodies and Harmonies},
publisher   = {Universit{\"a}tsverlag Potsdam},
year        = {2020},
doi         = {10.25932/publishup-47614},
url-details = {https://publishup.uni-potsdam.de/frontdoor/index/index/docId/47614},
url-pdf     = {https://publishup.uni-potsdam.de/opus4-ubp/frontdoor/deliver/index/docId/47614/file/catgvm01.pdf}
}
    2. Sebastian Rosenzweig, Frank Scherbaum, David Shugliashvili, Vlora Arifi-Müller, and Meinard Müller
      Erkomaishvili Dataset: A Curated Corpus of Traditional Georgian Vocal Music for Computational Musicology
      Transactions of the International Society for Music Information Retrieval (TISMIR), 3(1): 31–41, 2020. PDF Demo DOI 
      @article{RosenzweigSSAM20_Erkomaishvili_TISMIR,
author    = {Sebastian Rosenzweig and Frank Scherbaum and David Shugliashvili and Vlora Arifi-M{\"u}ller and Meinard M{\"u}ller},
title     = {{E}rkomaishvili {D}ataset: {A} Curated Corpus of Traditional {G}eorgian Vocal Music for Computational Musicology},
journal   = {Transactions of the International Society for Music Information Retrieval ({TISMIR})},
volume    = {3},
number    = {1},
pages     = {31--41},
year      = {2020},
doi       = {https://doi.org/10.5334/tismir.44},
url-demo  = {https://www.audiolabs-erlangen.de/resources/MIR/2019-GeorgianMusic-Erkomaishvili},
url-pdf   = {2020_RosenzweigEtAl_Erkomaishvili_TISMIR_ePrint.pdf}
}
    3. Frank Scherbaum, Meinard Müller, and Sebastian Rosenzweig
      Rechnergestützte Musikethnologie am Beispiel historischer Aufnahmen mehrstimmiger georgischer Vokalmusik
      In Proceedings of the GI Jahrestagung: 163–175, 2017. PDF DOI 
      @inproceedings{ScherbaumMR17_Georgien_GI,
author    = {Frank Scherbaum and Meinard M{\"u}ller and Sebastian Rosenzweig},
title     = {{R}echnergest{\"u}tzte {M}usikethnologie am {B}eispiel historischer {A}ufnahmen mehrstimmiger georgischer {V}okalmusik},
booktitle = {Proceedings of the GI Jahrestagung},
address   = {Chemnitz, Germany},
year      = {2017},
pages     = {163--175},
doi       = {10.18420/in2017\_11},
url-pdf   = {https://dl.gi.de/bitstream/handle/20.500.12116/3870/B1-10.pdf}
}
    4. Frank Scherbaum, Meinard Müller, and Sebastian Rosenzweig
      Analysis of the Tbilisi State Conservatory Recordings of Artem Erkomaishvili in 1966
      In Proceedings of the International Workshop on Folk Music Analysis (FMA), 2017. PDF 
      @inproceedings{ScherbaumMR17_Erkomaishvili_FMA,
author    = {Frank Scherbaum and Meinard M{\"u}ller and Sebastian Rosenzweig},
title     = {Analysis of the {T}bilisi {S}tate {C}onservatory Recordings of {A}rtem {E}rkomaishvili in {1966}},
booktitle = {Proceedings of the International Workshop on Folk Music Analysis ({FMA})},
address   = {M{\'a}laga, Spain},
year      = {2017},
url-pdf   = {/resources/MIR/2017-GeorgianMusic-Erkomaishvili/2017_ScherbaumMR_ErkomaishviliAnalysis_FMA.pdf}
}
    5. Meinard Müller, Sebastian Rosenzweig, Jonathan Driedger, and Frank Scherbaum
      Interactive Fundamental Frequency Estimation with Applications to Ethnomusicological Research
      In Proceedings of the AES Conference on Semantic Audio, 2017. PDF Details Demo 
      @inproceedings{MuellerRDS17_EthnoMusicF0_AES,
author      = {Meinard M{\"u}ller and Sebastian Rosenzweig and Jonathan Driedger and Frank Scherbaum},
title       = {Interactive Fundamental Frequency Estimation with Applications to Ethnomusicological Research},
booktitle   = {Proceedings of the {AES} Conference on Semantic Audio},
address     = {Erlangen, Germany},
year        = {2017},
url-demo    = {https://www.audiolabs-erlangen.de/resources/MIR/2017-GeorgianMusic-Erkomaishvili},
url-details = {http://www.aes.org/e-lib/browse.cfm?elib=18777},
url-pdf     = {https://www.aes.org/tmpFiles/elib/20220711/18777.pdf}
}

Computational Analysis of Georgian Funeral Songs (Zär) Corpus

As a second central contribution towards the understanding of Georgian vocal music, we conducted a comprehensive interdisciplinary study of three-voiced funeral songs from Svaneti in North-West Georgia (also referred to as Zär in Svan). This study is based on a new set of field recordings, which is part of the GVM Corpus.

  • Zär dataset with videos and audio switching interface for multitrack recordings
  • Publications
    1. Sebastian Rosenzweig, Frank Schwerbaum, and Meinard Müller
      Computer-Assisted Analysis of Field Recordings: A Case Study of Georgian Funeral Songs
      ACM Journal on Computing and Cultural Heritage (JOCCH), 16(1): 1–16, 2023. PDF Demo DOI 
      @article{RosenzweigSM23_GeorgianFuneralSongs_ACM-JOCCH,
author   = {Sebastian Rosenzweig and Frank Schwerbaum and Meinard M{\"u}ller},
title    = {Computer-Assisted Analysis of Field Recordings: {A} Case Study of {G}eorgian Funeral Songs},
journal  = {{ACM} Journal on Computing and Cultural Heritage ({JOCCH})},
volume   = {16},
number   = {1},
pages    = {1--16},
year     = {2023},
doi      = {doi.org/10.1145/3551645},
url      = {https://doi.org/10.1145/3551645},
url-pdf = {2023_RosenzweigSM_FuneralSongs_ACM-JOCCH_ePrint.pdf},
url-demo = {https://www.audiolabs-erlangen.de/resources/MIR/2022-GeorgianMusic-Zaer}
}
    2. Nana Mzhavanadze and Frank Scherbaum
      Svan Funeral Dirges (Zär): Cultural Context
      Musicologist, 5: 133–165, 2021. PDF DOI 
      @article{MzhavanadzeS21_ZarCulture_Musicologist,
title       = {Svan Funeral Dirges ({Z}{\"a}r): {C}ultural Context},
author      = {Nana Mzhavanadze and Frank Scherbaum},
journal     = {Musicologist},
volume      = {5},
issue       = {2},
publisher   = {Trabzon University},
year        = {2021},
pages       = {133--165},
doi         = {10.33906/musicologist.906765},
url-pdf = {https://dergipark.org.tr/en/download/article-file/1673889}
}
    3. Frank Scherbaum and Nana Mzhavanadze
      Svan Funeral Dirges (Zär): Language-Music Relation and Phonetic Properties
      Musicologist, 5: 66–82, 2021. PDF DOI 
      @article{ScherbaumM21_ZarLanguage_Musicologist,
title       = {Svan Funeral Dirges ({Z}{\"a}r): {L}anguage-Music Relation and Phonetic Properties},
author      = {Frank Scherbaum and Nana Mzhavanadze},
journal     = {Musicologist},
volume      = {5},
issue       = {1},
publisher   = {Trabzon University},
year        = {2021},
pages       = {66--82},
doi         = {10.33906/musicologist.875348},
url-pdf = {https://dergipark.org.tr/en/download/article-file/1559232}
}
    4. Nana Mzhavanadze and Frank Scherbaum
      Svan Funeral Dirges (Zär): Musicological Analysis
      Musicologist, 4: 168–197, 2020. PDF DOI 
      @article{MzhavanadzeS20_ZarMusicology_Musicologist,
title       = {Svan Funeral Dirges ({Z}{\"a}r): {M}usicological Analysis},
author      = {Nana Mzhavanadze and Frank Scherbaum},
journal     = {Musicologist},
eissn       = {2618-5652},
volume      = {4},
issue       = {2},
publisher   = {Trabzon University},
year        = {2020},
pages       = {168--197},
doi         = {10.33906/musicologist.782185},
url-pdf = {https://dergipark.org.tr/en/download/article-file/1246319}
}
    5. Frank Scherbaum and Nana Mzhavanadze
      Svan Funeral Dirges (Zär): Musical Acoustical Analysis of a New Collection of Field Recordings
      Musicologist, 4: 138–167, 2020. PDF DOI 
      @article{ScherbaumM20_ZarMusicAcoustic_Musicologist,
title     = {Svan Funeral Dirges ({Z}{\"a}r): {M}usical Acoustical Analysis of a New Collection of Field Recordings},
author    = {Frank Scherbaum and Nana Mzhavanadze},
journal   = {Musicologist},
eissn     = {2618-5652},
volume    = {4},
issue     = {2},
publisher = {Trabzon University},
year      = {2020},
pages     = {138--167},
doi       = {10.33906/musicologist.782094},
url-pdf = { https://dergipark.org.tr/en/download/article-file/1246012}
}

Computational Analysis of GVM Corpus

The GVM (Georgian Vocal Music) multimedia material stems from a research expedition in Georgia led and carried out by Frank Scherbaum and Nana Mzhavanadze in 2016. In our project, we created a curated version of the preprocessed tracks of the GVM data, comprising the original multitrack audio and video material as well as descriptive material related to the individual recording sessions. The resulting GVM corpus is permanently stored within a long-term archive and is accessible through web-based interfaces for research and other non-commercial purposes.

  • GVM dataset on LaZAR
  • GVM dataset with videos and multitrack audio switching interface
  • Throat microphone demonstrator with videos (shown for five selected songs)
  • GVM interface (shown for five selected songs)
  • Publications
    1. Frank Scherbaum, Nana Mzhavanadze, Sebastian Rosenzweig, and Meinard Müller
      Tuning Systems of Traditional Georgian Singing Determined from a New Corpus of Field Recordings
      Musicologist, 6(2): 142–168, 2022. PDF Demo DOI 
      @article{ScherbaumMRM22_GVM_Musicologist,
author = {Frank Scherbaum and Nana Mzhavanadze and Sebastian Rosenzweig and Meinard M{\"u}ller},
title = {Tuning Systems of Traditional Georgian Singing Determined from a New Corpus of Field Recordings},
journal = {Musicologist},
year={2022},
volume={6},
number={2},
pages={142--168},
doi = {10.33906/musicologist.1068947},
url-pdf = {2022_ScherbaumMRM_TuningGVM_Musicologist_ePrint.pdf},
url-demo = {https://www.audiolabs-erlangen.de/resources/MIR/2017-GeorgianMusic-Scherbaum}
}
    2. Frank Scherbaum, Nana Mzhavanadze, Sebastian Rosenzweig, and Meinard Müller
      Multi-media recordings of traditional Georgian vocal music for computational analysis
      In Proceedings of the International Workshop on Folk Music Analysis (FMA): 1–6, 2019. PDF 
      @InProceedings{ScherbaumMRM19_MultimediaRecordings_FMA,
author    = {Frank Scherbaum and Nana Mzhavanadze and Sebastian Rosenzweig and Meinard M{\"u}ller},
title     = {Multi-media recordings of traditional Georgian vocal music for computational analysis},
booktitle = {Proceedings of the International Workshop on Folk Music Analysis ({FMA})},
address   = {Birmingham, UK},
year      = {2019},
pages     = {1--6},
url-pdf   = {Scherbaum_etal_FMA2019.pdf}
}
    3. Frank Scherbaum, Nana Mzhavanadze, and Elguja Dadunashvili
      A web-based, long-term archive of audio, video, and larynx-microphone field recordings of traditional Georgian singing, praying and lamenting with special emphasis on Svaneti
      International Symposium on Traditional Polyphony (ISTP), 2018. PDF 
      @article{ScherbaumMD18_FieldRecordings_ISTP,
author  = {Frank Scherbaum and Nana Mzhavanadze and Elguja Dadunashvili},
title   = {A web-based, long-term archive of audio, video, and larynx-microphone field recordings of traditional {G}eorgian singing, praying and lamenting with special emphasis on {S}vaneti},
journal = {International Symposium on Traditional Polyphony {(ISTP)}},
address = {Tbilisi, Georgia},
year    = {2018},
url-pdf = {2018_ScherbaumMD_LongTermArchive_ISTP.pdf}
}
    4. Frank Scherbaum, Sebastian Rosenzweig, Meinard Müller, Daniel Vollmer, and Nana Mzhavanadze
      Throat Microphones for Vocal Music Analysis
      In Demos and Late Breaking News of the International Society for Music Information Retrieval Conference (ISMIR), 2018. PDF Demo 
      @inproceedings{ScherbaumRMVM18_ThroatMics_ISMIR-LBD,
author    = {Frank Scherbaum and Sebastian Rosenzweig and Meinard M\"uller and Daniel Vollmer and Nana Mzhavanadze},
title     = {Throat Microphones for Vocal Music Analysis},
booktitle = {Demos and Late Breaking News of the International Society for Music Information Retrieval Conference ({ISMIR})},
address   = {Paris, France},
year      = {2018},
url-pdf   = {2018_ScherbaumRMVM_GVM_ISMIR.pdf},
url-demo  = {https://www.audiolabs-erlangen.de/resources/MIR/2018-ISMIR-LBD-ThroatMics}
}

Computational Tools

In the context of the GVM project, we developed computational signal processing and music information retrieval (MIR) tools with a specific focus on analyzing multitrack singing voice recordings. With reproducible research in mind, we have created well-documented and user-friendly toolboxes that integrate many of these computational tools under an open-source license and provide reference implementations. For an overview and further applications, see also the Ph.D. thesis by Sebastian Rosenzweig.

  • Detecting Stable Regions in F0-trajectories
  • F0 Toolbox on GitHub, F0 Toolbox on Zenodo
  • GVM Toolbox on GitHub, GVM Toolbox on Zenodo
  • Publications
    1. Sebastian Rosenzweig, Frank Scherbaum, and Meinard Müller
      Reliability Assessment of Singing Voice F0-Estimates using Multiple Algorithms
      In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP): 261–265, 2021. PDF DOI 
      @inproceedings{RosenzweigSM21_F0Reliability_ICASSP,
author    = {Sebastian Rosenzweig and Frank Scherbaum and Meinard M{\"u}ller},
title     = {Reliability Assessment of Singing Voice {F0}-Estimates using Multiple Algorithms},
booktitle = {Proceedings of the {IEEE} International Conference on Acoustics, Speech, and Signal Processing ({ICASSP})},
pages     = {261--265},
doi       = {10.1109/ICASSP39728.2021.9413372},
address   = {Toronto, Canada},
year      = {2021},
url-pdf   = {https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9413372}
}
    2. Sebastian Rosenzweig, Frank Scherbaum, and Meinard Müller
      Detecting Stable Regions in Frequency Trajectories for Tonal Analysis of Traditional Georgian Vocal Music
      In Proceedings of the International Society for Music Information Retrieval Conference (ISMIR): 352–359, 2019. PDF Demo DOI 
      @inproceedings{RosenzweigSM19_StableF0_ISMIR,
author    = {Sebastian Rosenzweig and Frank Scherbaum and Meinard M{\"u}ller},
title     = {Detecting Stable Regions in Frequency Trajectories for Tonal Analysis of Traditional Georgian Vocal Music},
booktitle = {Proceedings of the International Society for Music Information Retrieval Conference ({ISMIR})},
pages     = {352--359},
address   = {Delft, The Netherlands},
year      = {2019},
url-demo  = {https://www.audiolabs-erlangen.de/resources/MIR/2019-ISMIR-StableF0},
url-pdf   = {2019_RosenzweigSM_StableF0_ISMIR.pdf},
doi       = {10.5281/zenodo.3527816}
}

Project-Related Presentations

  • Frank Scherbaum and Meinard Müller (2022). Togetherness in Traditional Georgian Singing: From Tuning Adjustments to Synchronisation of Heartbeat Variability. Presentation at the Musical Togetherness Symposium (MTS-22), 13-15 July 2022, University of Music and Performing Arts Vienna, Austria.
    Presenter: Frank Scherbaum
    YouTube Link

  • Frank Scherbaum, Nana Mzhavanadze, Simha Arom, Sebastian Rosenzweig, and Meinard Müller (2021). Analysis of Tonal Organisation and Intonation Practice in the Tbilisi State Conservatory Recordings of Artem Erkomaishvili of 1966. Presentation at the 6th International Conference on Analytical Approaches to World Music, 12 June 2021 (Special Session in Honor of Simha Arom).
    Presenter: Frank Scherbaum
    YouTube Link

  • Nana Mzhavanadze and Frank Scherbaum (2020). Zär: Polyphonic Group Laments from Svaneti/Georgia. Presentation at the Annual Meeting of the Society of Ethnomusicology, Ottawa, 30 October 2020.
    Presenter: Nana Mzhavanadze and Frank Scherbaum
    YouTube Link

Projected-Related Ph.D. Thesis

  1. Sebastian Rosenzweig
    Interactive Signal Processing Tools for Analyzing Multitrack Singing Voice Recordings
    PhD Thesis, Friedrich-Alexander-Universität Erlangen-Nürnberg, 2022. PDF Details 
    @phdthesis{Rosenzweig22_Singing_PhD,
author      = {Sebastian Rosenzweig},
year        = {2022},
title       = {Interactive Signal Processing Tools for Analyzing Multitrack Singing Voice Recordings},
school      = {Friedrich-Alexander-Universit{\"a}t Erlangen-N{\"u}rnberg},
url-details = {https://opus4.kobv.de/opus4-fau/frontdoor/index/index/docId/19750},
url-pdf     = {2022_Rosenzweig_Singing_ThesisPhD.pdf}
}