{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Date: Jul 2019<br>Programmer:  Patricio López-Serrano, Yiğitcan Özer\n",
    "This demo illustrates decomposition of a loop-based electronic dance music track, following [1].\n",
    "\n",
    "\n",
    "#### The notebook proceeds in the following steps:\n",
    "<br>1. Load audio files for the complete, downmixed track, as well as for the individual loops that the track contains.\n",
    "<br>2. Compute STFTs for all audio data.\n",
    "<br>3. Each loop becomes a fixed template (\"page\") in the tensor W.\n",
    "<br>The track spectrogram is the target to approximate, V.\n",
    "<br>We wish to learn the activation matrix H, which answers the question\n",
    "<br>\"Where was each loop activated throughout the track?\"\n",
    "<br>4. Visualize results.\n",
    "\n",
    "\n",
    "#### References\n",
    "<br>[1] Patricio López-Serrano, Christian Dittmar, Jonathan Driedger, and\n",
    "<br>Meinard Müller.\n",
    "<br>Towards modeling and decomposing loop-based electronic music.\n",
    "<br>In Proceedings of the International Conference\n",
    "<br>on Music Information Retrieval (ISMIR), pages 502–508,\n",
    "<br>New York City, USA, August 2016.\n",
    "\n",
    "\n",
    "#### If you use the 'NMF toolbox' please refer to:\n",
    "[2] Patricio López-Serrano, Christian Dittmar, Yiğitcan Özer, and Meinard Müller\n",
    "<br>NMF Toolbox: Music Processing Applications of Nonnegative Matrix Factorization\n",
    "<br>In Proceedings of the International Conference on Digital Audio Effects (DAFx), 2019.\n",
    "\n",
    "#### License:\n",
    "This file is part of 'NMF toolbox'. 'NMF toolbox' is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. 'NMF toolbox' is distributed in the hope that it will be useful, but ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.\n",
    "\n",
    "#### You should have received a copy of the GNU General Public License along<br>with 'NMF toolbox'. If not, see http://www.gnu.org/licenses/."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Initialization"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import numpy as np\n",
    "import scipy.io.wavfile as wav\n",
    "import IPython.display as ipd\n",
    "from copy import deepcopy\n",
    "\n",
    "from NMFtoolbox.forwardSTFT import forwardSTFT\n",
    "from NMFtoolbox.inverseSTFT import inverseSTFT\n",
    "from NMFtoolbox.logFreqLogMag import logFreqLogMag\n",
    "from NMFtoolbox.initActivations import initActivations\n",
    "from NMFtoolbox.NMFD import NMFD\n",
    "from NMFtoolbox.visualizeComponentsNMF import visualizeComponentsNMF\n",
    "from NMFtoolbox.alphaWienerFilter import alphaWienerFilter\n",
    "from NMFtoolbox.utils import make_monaural, pcmInt16ToFloat32Numpy\n",
    "\n",
    "inpPath = '../data/'\n",
    "outPath = 'output/'\n",
    "\n",
    "filename = 'LSDDM_EM_track.wav'\n",
    "\n",
    "filenameFX = 'LSDDM_EM_Effects.wav'\n",
    "filenameBass = 'LSDDM_EM_bass.wav'\n",
    "filenameMelody = 'LSDDM_EM_melody.wav'\n",
    "filenameDrums = 'LSDDM_EM_drums.wav'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. Load the audio signal"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "fs, xTr = wav.read(os.path.join(inpPath, filename))\n",
    "\n",
    "fsFX, xFX = wav.read(os.path.join(inpPath, filenameFX))\n",
    "fsBass, xBass = wav.read(os.path.join(inpPath, filenameBass))\n",
    "fsMelody, xMelody = wav.read(os.path.join(inpPath, filenameMelody))\n",
    "fsDrums, xDrums = wav.read(os.path.join(inpPath, filenameDrums))\n",
    "\n",
    "# make monaural if necessary\n",
    "xTr = make_monaural(xTr)\n",
    "xFX = make_monaural(xFX)\n",
    "xBass = make_monaural(xBass)\n",
    "xMelody = make_monaural(xMelody)\n",
    "xDrums = make_monaural(xDrums)\n",
    "\n",
    "# int16 -> float32 conversion\n",
    "xTr = pcmInt16ToFloat32Numpy(xTr)\n",
    "xFX = pcmInt16ToFloat32Numpy(xFX)\n",
    "xBass = pcmInt16ToFloat32Numpy(xBass)\n",
    "xMelody = pcmInt16ToFloat32Numpy(xMelody)\n",
    "xDrums = pcmInt16ToFloat32Numpy(xDrums)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Compute STFT"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# spectral parameters\n",
    "paramSTFT = dict()\n",
    "paramSTFT['blockSize'] = 4096;\n",
    "paramSTFT['hopSize'] = 2048;\n",
    "paramSTFT['winFunc'] = np.hanning(paramSTFT['blockSize'])\n",
    "paramSTFT['reconstMirror'] = True\n",
    "paramSTFT['appendFrame'] = True\n",
    "paramSTFT['numSamples'] = len(xTr)\n",
    "\n",
    "# STFT computation\n",
    "XTr, ATr, PTr = forwardSTFT(xTr, paramSTFT);\n",
    "\n",
    "# get dimensions and time and freq resolutions\n",
    "numBinsTr, numFramesTr = XTr.shape\n",
    "deltaT = paramSTFT['hopSize'] / fs\n",
    "deltaF = fs / paramSTFT['blockSize']\n",
    "\n",
    "# get logarithmically-spaced frequency axis version for visualization\n",
    "logFreqLogMagA, logFreqAxis = logFreqLogMag(ATr, deltaF)\n",
    "numLogBins = len(logFreqAxis)\n",
    "\n",
    "# repeat for FX loop ---------------------------------------------------\n",
    "paramSTFT['numSamples'] = len(xFX);\n",
    "\n",
    "# STFT computation\n",
    "XFX, AFX, _ = forwardSTFT(xFX, paramSTFT)\n",
    "XBass, ABass, _ = forwardSTFT(xBass, paramSTFT)\n",
    "XMelody, AMelody, _ = forwardSTFT(xMelody, paramSTFT)\n",
    "XDrums, ADrums, _ = forwardSTFT(xDrums, paramSTFT)\n",
    "numBinsBass, numFramesBass = XBass.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. Apply NMF variants to STFT magnitude"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# set common parameters\n",
    "numComp = 4\n",
    "numIter = 30\n",
    "numTemplateFrames = deepcopy(numFramesBass)\n",
    "\n",
    "initW = list()\n",
    "initW.append(ADrums)\n",
    "initW.append(AMelody)\n",
    "initW.append(ABass)\n",
    "initW.append(AFX)\n",
    "\n",
    "# generate initial activations\n",
    "paramActivations = dict()\n",
    "paramActivations['numComp'] = numComp\n",
    "paramActivations['numFrames'] = numFramesTr\n",
    "initH = initActivations(paramActivations, 'uniform')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "80d5789b4d104353a9f30532c3bd2cb6",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, description='Processing', max=30, style=ProgressStyle(description_width='i…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "# NMFD parameters\n",
    "paramNMFD = dict()\n",
    "paramNMFD['numComp'] = numComp\n",
    "paramNMFD['numFrames'] = numFramesTr\n",
    "paramNMFD['numIter'] = numIter\n",
    "paramNMFD['numTemplateFrames'] = numTemplateFrames\n",
    "paramNMFD['initW'] = initW\n",
    "paramNMFD['initH'] = initH\n",
    "paramNMFD['numBins'] = numBinsTr\n",
    "paramNMFD['fixW'] = True\n",
    "\n",
    "# NMFD core method\n",
    "nmfdW, nmfdH, nmfdV, _, _ = NMFD(ATr, paramNMFD)\n",
    "\n",
    "# alpha-Wiener filtering\n",
    "nmfdA, _ = alphaWienerFilter(ATr, nmfdV, 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 527448 characters]\n",
      "text/plain": [
       "<Figure size 1440x1440 with 6 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# visualize\n",
    "paramVis = dict()\n",
    "paramVis['deltaF'] = deltaF\n",
    "paramVis['deltaT'] = deltaT\n",
    "paramVis['fontSize'] = 14\n",
    "paramVis['logComp'] = 1e5\n",
    "fh1, _ = visualizeComponentsNMF(ATr, nmfdW, nmfdH, nmfdV, paramVis)\n",
    "fh1.savefig(os.path.join(outPath, 'LSDDM_EM.png'))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
