![\"FMP\"](\"../data/FMP_Teaser_Cover.png\")
\n",
"![\"AudioLabs\"](\"../data/Logo_AudioLabs_Long.png\"){kind=logo}
\n",
"\n",
"\n",
"![\"C8\"](\"../data/C8_nav.png\")
\n",
"\n", "Following Section 8.3.1 [Müller, FMP, Springer 2015], we cover in this notebook a technique known as nonnegative matrix factorization (NMF). The formal definition of the NMF problem and an iterative learning procedure for computing a factorization in practice was first described by Lee and Seoung. Implementations and applications for NMF can also be found in the NMF Toolbox. \n", "\n", "
![\"FMP_C8_F20b\"](\"../data/C8/FMP_C8_F20b.png\")
\n",
"\n",
"Typically, the matrices $W$ and $H$ are required to have a much lower rank than the original matrix $V$. This factorization is interpreted as follows: The columns of $V$ are regarded as data vectors. The underlying assumption is that these vectors can be represented as a weighted superposition of a relatively small number of **template** vectors. The columns of $W$ correspond to these templates. Furthermore, the rows of $H$—called **activations**—indicate where these templates occur in $V$. The nonnegativity constraints often lead to a decomposition that allows for a semantically meaningful interpretation of the coefficients. However, in most cases, the resulting factorization problem has no exact solution, thus requiring optimization procedures for finding suitable numerical approximations. In the following, to build up some intuition, we give a factorization example form the music domain. Subsequently, we formally introduce the NMF problem and discuss an iterative learning procedure. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Example: Spectrogam Factorization \n",
"\n",
"As illustration of NMF, we show how this technique can be used to factorize a spectrogram of a music recording into musically meaningful components. Let us consider the first measures of the Prélude Op. 28, No. 4 by Frédéric Chopin. The following figure shows the musical score as well as a piano-roll representation of the score synchronized to an audio recording of a performance. For illustration purposes, all information related to the note number $p=71$ is highlighted by the red rectangular frames.\n",
"\n",
"![\"FMP_C8_F21a-b\"](\"../data/C8/FMP_C8_F21a-b.png\")
\n",
"\n",
"![\"FMP_C8_F21c-e\"](\"../data/C8/FMP_C8_F21c-e.png\")
\n",
"\n",
"In this case, each template specified by the matrix $W$ reflects how a note of a certain pitch is spectrally realized in $V$, and the activation matrix $H$ looks similar to the piano-roll representation of the score. In practice, however, it is often hard to predict which of the signal's properties are ultimately captured by the learned factors. \n",
"To better control this factorization, we will show how additional score information can be used to [constrain NMF](../C8/C8S3_NMFAudioDecomp.html) to yield a musically meaningful decomposition. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Formal Definition of NMF\n",
"\n",
"A matrix with real-valued coefficients is called **nonnegative** if all the coefficients are either zero or positive. Let $V \\in \\mathbb{R}_{\\ge 0}^{K \\times N}$ be such a nonnegative matrix having $K\\in\\mathbb{N}$ rows and $N\\in\\mathbb{N}$ columns. The dimensions $K$ and $N$ of the matrix $V$ are usually thought to be large. Given a number $R\\in\\mathbb{N}$ smaller than both $K$ and $N$, the goal of NMF is to find two nonnegative matrices $W \\in \\mathbb{R}_{\\ge 0}^{K \\times R}$ and $H \\in \\mathbb{R}_{\\ge 0}^{R \\times N}$ such that \n",
"\n",
"\\begin{equation}\n",
" V \\approx W \\cdot H.\n",
"\\end{equation}\n",
"\n",
"As said above, the columns of $V$ are regarded as $K$-dimensional data vectors, where $N$ is the number of data vectors. This matrix is then approximately factorized into a $(K \\times R)$ matrix $W$ and an $(R \\times N)$ matrix $H$. The parameter $R$, which is referred to as the **rank** of the factorization, is usually chosen to be much smaller than $K$ and $N$. Therefore, the number of coefficients in $W$ and $H$ is typically much smaller than the total number in $V$ (i.e., $KR+RN \\ll KN$), and the product $WH$ can be thought of as a compressed version of the original matrix $V$. As already mentioned before, the column vectors of $W$ are also referred to as **template vectors**, whereas the weights specified by $H$ are called **activations**. As opposed to arbitrary linear combinations as known from linear algebra, the linear combinations occurring in the NMF context only involve nonnegative weights of nonnegative template vectors. As a result, there are no effects such as destructive interferences, where a (positive) component can be canceled out by adding a kind of inverse (negative) component. Instead, the data vectors need to be explained in a purely constructive fashion only involving positive components.\n",
"\n",
"To find an approximate factorization $V \\approx W \\cdot H$, we need to specify a distance function that quantifies the \n",
"quality of the approximation. There are many ways for defining such a distance function, leading to different NMF variants. In the following, we only consider one of these variants, which is based on the **Euclidean distance**. Let $A,B\\in\\mathbb{R}^{K \\times N}$ be two matrices with coefficients $A_{kn}$ and $B_{kn}$\n",
"for $k\\in[1:K]$ and $n\\in[1:N]$. Then, the square of the Euclidean distance between $A$ and $B$ is defined by\n",
"\n",
"\\begin{equation}\n",
" \\|A-B\\|^2:= \\sum_{k=1}^{K}\\sum_{n=1}^{N}(A_{kn}-B_{kn})^2.\n",
"\\end{equation}\n",
"\n",
"Based on this distance measure, we can formalize our NMF problem as follows: Given a nonnegative matrix $V\\in\\mathbb{R}_{\\ge 0}^{K \\times N}$ and a rank parameter $R$, minimize\n",
"\n",
"\\begin{equation}\n",
" \\|V-WH\\|^2\n",
"\\end{equation}\n",
"\n",
"with respect to $W \\in \\mathbb{R}_{\\ge 0}^{K \\times R}$ and $H \\in \\mathbb{R}_{\\ge 0}^{R \\times N}$. In other words, regarding $\\|V-WH\\|^2$ as a joint function of $W$ and $H$, the objective is to find a minimum under the nonnegativity constraint for $W$ and $H$. For general matrices, this is a hard computational problem due to several reasons. First, it is in general difficult to enforce hard constraints such as nonnegativity on the solution of an optimization problem. Second, the joint optimization over both matrices $W$ and $H$ leads to computational challenges. In fact, when regarding $\\|V-WH\\|^2$ as a function of $W$ only or $H$ only, one can show that the resulting functions satisfy a strong property referred to as **convexity**. This property, which implies that any local minimum must be a global minimum, makes it possible to apply powerful tools from the field of convex analysis. However, $\\|V-WH\\|^2$ is **not convex** in both matrices together. Therefore, it is unrealistic to expect an algorithm that can solve this problem in the sense of finding a global minimum. However, there are many techniques from numerical optimization that can be applied to find local minima."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Multiplicative Update Rules\n",
"\n",
"Lee and Seung introduced in their seminal paper a powerful algorithm based on multiplicative update rules to iteratively learn a nonnegative matrix factorization. In the following, we only summarize their algorithm and refer to their paper or Section 8.3.1 of [Müller, FMP, Springer 2015] for a proof. The idea is based on the standard gradient descent approach, which is applied to our problem of minimizing $\\|V-WH\\|^2$ as a function of $W$ and $H$. Since the joint optimization is a very hard problem, one idea is to first fix the factor $W$ and to optimize with regard to $H$, and then to fix the learned factor $H$ and to optimize with regard to $W$. This process is then iterated, where the role of $W$ and $H$ is interchanged after each step. In standard gradient descent the update rules are additive, where a parameter needs to be chosen to control the **step size** towards the direction of the negative gradient. The main trick in the NMF optimization algorithm is that this step size parameter can be set in a specific way so that the **additive** update rules become **multiplicative** update rules. The following table shows the iterative algorithm for learning an NMF decomposition., where the multiplicative update rules are given in matrix notation. The operator $\\odot$ denotes pointwise multiplication and the operator $\\oslash$ pointwise division. \n",
"\n",
"![\"FMP_C8_T01\"](\"../data/C8/FMP_C8_T01.png\")
\n",
"\n",
"The multiplicative update rules and their properties have a number of remarkable implications.\n",
"\n",
"* The first implication is that the matrix sequences $W^{(0)},W^{(1)},W^{(2)},\\ldots$ and $H^{(0)},H^{(1)},H^{(2)},\\ldots$ converge. Denoting the limit matrices by $W^{(\\infty)}$ and $H^{(\\infty)}$, the stationarity property implies that these matrices form a local minimum of the distance function $\\|V-WH\\|^2$.\n",
"* Second, the multiplicative update rules are extremely easy to implement. \n",
"* Third, in practice, the convergence turns out to be relatively fast in comparison with many other methods.\n",
"* Fourth, one major benefit of using multiplicative update rules is that the nonnegativity constraints are enforced automatically.\n",
"\n",
"Indeed, starting with nonnegative matrices $V$, $W^{(0)}$, and $H^{(0)}$, all entries in the equations (1) and (2) of the above table are also nonnegative. Since all operations are multiplicative (either multiplication or division), also the matrices $W^{(\\ell)}$ and $H^{(\\ell)}$ remain nonnegative throughout the iteration.\n",
"\n",
"In practice, the iteration is performed until a specified stop criterion is fulfilled. For example, one may perform a certain number of iterations $\\ell=0,1,2,\\ldots, L$ for some user-specified parameter $L\\in\\mathbb{N}$. As another stop criterion, one may look at the distances between two subsequently computed template matrices and activation matrices. Specifying a threshold $\\varepsilon>0$, the iteration may be stopped when $\\|H^{(\\ell+1)}-H^{(\\ell)}\\|^2\\leq \\varepsilon$ and $\\|W^{(\\ell+1)}-W^{(\\ell)}\\|^2\\leq \\varepsilon$. Note that this stop criterion does not say anything about the quality of the approximation $V\\approx WH$ achieved by the procedure. Even in the limit case and even when converging to the global minimum (and not to a local one), the distance $\\|V-WH\\|^2$ may still be large. In particular, this may happen if the rank parameter $R$ is chosen too small."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Implementation\n",
"\n",
"In the following code cell, we provide a basic implementation that closely follows the NMF procedure described above. Here are some important notes:\n",
"\n",
"* All operation are performed using matrix-based operations.\n",
"* To avoid devision by zero, a small value (machine epsilon) is added to the denominators in the multiplicative update rules.\n",
"* In the factorization $V\\approx WH$, there is a degree of freedom with regard to columns and rows of $W$ and $H$, respectively. For example, one may divide a column of $W$ by a factor $\\alpha\\in\\mathbb{R}$ and multiply the correspond row of $H$ by the same factor without changing the product $WH$. In the following code, setting the parameter `norm=True`, we use this fact to enforce that all columns of the final template matrix $W$ are normalized with regard to the [maximum norm](../C3/C3S1_FeatureNormalization.html).\n",
"* If not specified, the matrices $W$ and $H$ are initialized randomly. The power of NMF lies in the possibility of guiding the decomposition process by [initializing the matrices in a musically informed way](../C8/C8S3_NMFAudioDecomp.html)."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"execution": {
"iopub.execute_input": "2024-02-15T08:44:32.013624Z",
"iopub.status.busy": "2024-02-15T08:44:32.013295Z",
"iopub.status.idle": "2024-02-15T08:44:35.641172Z",
"shell.execute_reply": "2024-02-15T08:44:35.640539Z"
}
},
"outputs": [],
"source": [
"import os, sys\n",
"import numpy as np\n",
"import scipy\n",
"#import scipy.spatial\n",
"import matplotlib.pyplot as plt\n",
"import IPython.display as ipd\n",
"import librosa, librosa.display\n",
"#from matplotlib import gridspec\n",
"from numba import jit\n",
"\n",
"sys.path.append('..')\n",
"import libfmp.b\n",
"import libfmp.c2\n",
"import libfmp.c6\n",
"%matplotlib inline\n",
"\n",
"@jit(nopython=True)\n",
"def nmf(V, R, thresh=0.001, L=1000, W=None, H=None, norm=False, report=False):\n",
" \"\"\"NMF algorithm with Euclidean distance\n",
"\n",
" Notebook: C8/C8S3_NMFbasic.ipynb\n",
"\n",
" Args:\n",
" V (np.ndarray): Nonnegative matrix of size K x N\n",
" R (int): Rank parameter\n",
" thresh (float): Threshold used as stop criterion (Default value = 0.001)\n",
" L (int): Maximal number of iteration (Default value = 1000)\n",
" W (np.ndarray): Nonnegative matrix of size K x R used for initialization (Default value = None)\n",
" H (np.ndarray): Nonnegative matrix of size R x N used for initialization (Default value = None)\n",
" norm (bool): Applies max-normalization of columns of final W (Default value = False)\n",
" report (bool): Reports errors during runtime (Default value = False)\n",
"\n",
" Returns:\n",
" W (np.ndarray): Nonnegative matrix of size K x R\n",
" H (np.ndarray): Nonnegative matrix of size R x N\n",
" V_approx (np.ndarray): Nonnegative matrix W*H of size K x N\n",
" V_approx_err (float): Error between V and V_approx\n",
" H_W_error (np.ndarray): History of errors of subsequent H and W matrices\n",
" \"\"\"\n",
" K = V.shape[0]\n",
" N = V.shape[1]\n",
" if W is None:\n",
" W = np.random.rand(K, R)\n",
" if H is None:\n",
" H = np.random.rand(R, N)\n",
" V = V.astype(np.float64)\n",
" W = W.astype(np.float64)\n",
" H = H.astype(np.float64)\n",
" H_W_error = np.zeros((2, L))\n",
" ell = 1\n",
" below_thresh = False\n",
" eps_machine = np.finfo(np.float32).eps\n",
" while not below_thresh and ell <= L:\n",
" H_ell = H\n",
" W_ell = W\n",
" H = H * (W.transpose().dot(V) / (W.transpose().dot(W).dot(H) + eps_machine))\n",
" W = W * (V.dot(H.transpose()) / (W.dot(H).dot(H.transpose()) + eps_machine))\n",
"\n",
" # H+1 = H *p ((W^T * V) / p (W^T * W * H))\n",
" # H = np.multiply(H, np.divide(np.matmul(np.transpose(W), V),\n",
" # np.matmul(np.matmul(np.transpose(W), W), H)))\n",
" # W+1 = W *p ((V * H^T) / p (W * H * H^T))\n",
" # W = np.multiply(W, np.divide(np.matmul(V, np.transpose(H)),\n",
" # np.matmul(np.matmul(W, H), np.transpose(H))))\n",
"\n",
" H_error = np.linalg.norm(H-H_ell, ord=2)\n",
" W_error = np.linalg.norm(W - W_ell, ord=2)\n",
" H_W_error[:, ell-1] = [H_error, W_error]\n",
" if report:\n",
" print('Iteration: ', ell, ', H_error: ', H_error, ', W_error: ', W_error)\n",
" if H_error < thresh and W_error < thresh:\n",
" below_thresh = True\n",
" H_W_error = H_W_error[:, 0:ell]\n",
" ell += 1\n",
" if norm:\n",
" for r in range(R):\n",
" v_max = np.max(W[:, r])\n",
" if v_max > 0:\n",
" W[:, r] = W[:, r] / v_max\n",
" H[r, :] = H[r, :] * v_max\n",
" V_approx = W.dot(H)\n",
" V_approx_err = np.linalg.norm(V-V_approx, ord=2)\n",
" return W, H, V_approx, V_approx_err, H_W_error"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Toy Example\n",
"\n",
"We illustrate the functioning of the NMF procedure by means of a small toy example $V \\in \\mathbb{R}_{\\ge 0}^{K \\times N}$ with $K=4$ and $N=8$. The rank parameter is set to $R=2$."
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"execution": {
"iopub.execute_input": "2024-02-15T08:44:35.644808Z",
"iopub.status.busy": "2024-02-15T08:44:35.644543Z",
"iopub.status.idle": "2024-02-15T08:44:40.754312Z",
"shell.execute_reply": "2024-02-15T08:44:40.753665Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Matrix V and randomly initialized matrices W and H\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAz4AAACNCAYAAAB2fa+eAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAkKUlEQVR4nO3df7QdZX3v8fcnJwk/E/mRqEDCDyW1aiqgEXRhFXXJBUSoV5aLFHHBtc3VW7hwa6torfZ67VXbLkVExQiICIXbpahcVxDpohSrF0oSfmgMSIzRnASaBA0ECIknfO8fMztuTvaPmX1m75k9+/Naay/Onpn95HsOz/fZzzPzzDyKCMzMzMzMzOpsWtkBmJmZmZmZ9ZsHPmZmZmZmVnse+JiZmZmZWe154GNmZmZmZrXngY+ZmZmZmdWeBz5mZmZmZlZ7HviYmZmZmVnteeBjZmZmZma154GPmZmZmVWKpE9KurjsOOpE0r9LennZcZTJA58BkTQmabukhS323SDpq2XEZTYIku6X9J+b3h8jKSS9u2nbyyVtk3RAKUGamVkmOdv0A9N9R00q4/OSvtKm/LnAu4Ev9+t3GCRJF0haLmmHpGsm7dtL0lWSfpn+ve6VdGqX8s6WtFrSU5J+LukPm/a9VNLtkh6XtEbS25s++g/Axwv95YaMBz4DEhG7gAeB54y0JS0CTgf+qoy4zAZkK/C8pvcXAY9P2nYhcG1EbB1cWGZm1oOtZGzTgWOBJ4F1k8pYCNzXpvzzgGURsT1vYJKmZ9mWt4wp2gh8Ari6xb7pwHrgDSR/v78G/knSkW1iewvwaeB8YBbwemBtU9zfAb4LHAQsAa6T9Hvpx28G3ijpkEJ+qyHkgc9g/QR42aRtfw/8Q0RsLCEes0F5HJgNu8/kvQ34KumXZHqV5xzg8yXFZ2Zm2eVp048FfhoRMamMlwP3tin/VOBfmzdIOlTSNyVtlvQLSf+9ad86SR+U9ADwlKTpbba9VNIdkrZKWiXpjE5l9Pi32UNE3BQR3wYea7HvqYj4m4hYFxHPRsR3gV8Ar2pT3P8EPh4Rd6XHb4iIDem+3wcOBT4bEbsi4nbgh8C56b/1DLACOLmo323YeOAzWKtouuIj6W3AApLBj1md7f6SBP4r8H+AXzVt+y/ADyPiwRJiMzOzfPK06ceRnPjdTdILgDnAA23K/wPgoabjpwH/F7gfOAx4M3CxpP/U9JnFwFuBAyJiYvI2QGkZ3weeT3JF6npJL+lSRnPc300HTa1e323zu+SS/m1+j6TPOHnfGLAImJtOYxuXdLmkfRqHtCqS5Opaw2rgmCJiHUYe+AzW7is+aeX9FPBXEfF0qVGZ9d/jwGxJM4D3kpwFfAJ4XvqF9mfA50qMz8zMssvTph8LLJa0pfEimfr/cEQ82ab8A4BtTe9fDcyNiI9HxM6IWAt8BTi76ZjLImL9pOlxzdteA+wPfCot43aSKWGLu5SxW0ScHhEHtHmd3vEvlkH697we+FqbE4EvAGYAZwF/SPK3PQ74SLr/QWAT8JeSZkg6mWQK3b5NZWwj+fuOJA98BmsVsCCt2O8BniGZ/2pWd42zg2cB90fEQ6RfkiRn1yaA75UXnpmZ5ZCpTZe0F/BSkmlvxza9vkT7+3sAfkNy/0rDEcChzVdYgA+TDAQa1rcop3nbocD6iHi2adsvSa4gdSpjINIB49eBncAFbQ5rDMg+HxGPRMQW4DPAaQAR8Vvgj0j+HzwKvB/4J2C8qYxZJPdojaSib96yztaRDHaOA/4GWNxizqtZHT0OvIjkBtiPptueIPnivJCkEXcumJkNh0xtevok22nAbc1XdyQtILnXpJ0HSKZ73ZO+Xw/8IiIWdPhMq++Q5m0bgfmSpjUNfg4HftaljN0k3UJypaWVH0REx6exdShXwFUkA7nT0gHMHiLiN5LGO8UZEQ+QXOVplP0j4GtNh7wUuK6XOOvAV3wGKO3YrQauAO6OiH/t8hGzunic5MtidkR8P932BMmZv1cD15QTlpmZ9SBrm34cyYMNJk9pezWdr/gso6nzDvw78ET68IF9lCwRslDSq3PEfDfwFPCBdBrYSSQPZbgxawERcWpE7N/m1XbQkz5YYW9gDBiTtPekhyd8iWRA8rYMT7L7KnChpOdLOhC4mGTKXuPfekVa/r6S/gI4hPT/R3oF7lXAbVl/57rxwGfwfkJyk9kHyg7EbIAeJ5lO0HwfzxOkDXKHed5mZlY9Wdv0Y/ndVRtg91PgjqDzwOda4LTGTfvpkiBvS8v7BbAFuJLnPj67o4jYCZxB8sS4LcAXgXcP6KE6HyGZpnYJ8K70548ASDqC5AERxwKPSnoyfZ2T7r9F0oebyvpfJH/Tn5GcTL8X+Num/ecCj5Dc6/Nm4C0RsSPddwZwxyg/SVieXWJmZmZmVSLpfwObIuLSsmOpC0l3A++JiJ90PbimPPAxMzMzM7Pa81Q3Mxtpkq6WtElSyzNgSlyWrpnwgKRXNu07RdJD6b5LBhe1mZmZ5eWBj5mNumuAUzrsP5VkoeEFwBKSm1Aba3F9Id3/MpJ1Kl7W10jNzMysZx74mNlIi4g7gV93OORM4NpI3AUcIOkQ4HhgTUSsTW+avTE91szMzCrIAx8zs84O47mL2o2n29ptNzMzswrqywKmkgp5YsJee+1VRDGFlrX33nsXUg4UF1ORf6eifr+iYlq3bh1btmxRlmNb1LtbI6LTFCbLaZ999onZs2dPuZx58+YVEE1i5cqVWyJiLrRte1aRLBzcsDQilub4J1rVv+iw3czaaJOjbqsH6KCDDor58+dPuZynn366gGgSM2fOLKScHTt2dD8oo6L6Q7/9bcu1SHsya9asQsopKqbx8XEee+yxoeqj9WXgU5QiErNhwYJOi/1md/TRRxdSDjimLBYtWpTr+OnTf1elJyYm5hQShO02e/ZszjnnnCmX83d/93cFRJOYMWPGL5vfN9cBgImJiWciIl9Feq5xoLkxmkeyAvjMNtvNrIMWOeq2eoDmz5/PLbfcMuVyVq5cWUA0iaL6ez//+c8LKQdg4cKFhZSzYcOGQsoBeMMb3tD9oAw2b95cSDknn3xyruOr0EfzVDerDUlMmzZt98tGz+Q6UFA9uBl4d/p0t9cAj0fEIyQLyC2QdJSkmcDZ6bFm1kafctTMKq4qfbRKX/Exy2tsbKzsEKxkeeuApBuAk4A5ksaBjwEzACLiCmAZcBqwBngaOD/dNyHpAuBWYAy4OiJWFfNbmNWX22mz0VSF3PfAx2pDUiWSysrTSx2IiMVd9gfwZ232LSMZGJlZBm6nzUZTVXI/88AnXbNiObAhIk7vX0hmvalKUg2bOuW264BZtTlH+6NO7bjVU1VyP88ku4uA1f0KxKwIVZg/OoRqldu+f8Cs2pyjfVGrdtzqKU/eS7pa0iZJP2mz/yRJj0u6L319NFMMWQ6SNA94K3BlluPNytA4m9B4WXd1y+3JdcD1wKxanKPFq1s7bvXUQx/tGqDb465/EBHHpq+PZyk061S3S4EPAMU8QNysT/wlmtul1Cy3XQfMqs05WrhLqVk7bvWUJ/cj4k5JRxYdQ9crPpJOBzZFxIouxy2RtFzS8sKiM8uhlys+kg6Q9A1JD0paLem1fQ6zMnrJ7e3btw8out74bLJZtTlHi9VLO/7YY48NKDqz3+nTrJzXSrpf0i2SXp7lA1mmup0InCFpHXAj8CZJ100+KCKWRsSiKS4UaDYlPSTV54DvRcTvA8cwWnOkc+f2PvvsM+gYc3OnylqRdKCkkHTUpO2fl/SVsuIaRc7RQuVuxw8++OBBx2gG7NFHm9MYjKevJTmLWwkcERHHAJ8Hvp3lQ10HPhHxoYiYFxFHkizQd3tEvCtncGZ9l3dxLEmzgdcDVwFExM6I2NrfKKujjrntxRGtg2OBJ4F1k7YvBO4bcCwjyzlarDq241ZPLfpoWxqD8fS1NE95EfFERDyZ/rwMmCFpTrfPucWxWsl5FvFFwGbgq5LulXSlpP36G6H1m88mWxvHAj9N12Vq9nLg3sGHM7qco2ajqci8l/RCSUp/Pp5kTNN1HmeuBUwj4g7gjh7iM+u7Fs+InzPpnrOlk84oTAdeCVwYEXdL+hxwCfDX/Y+2WuqS21VZJ8Aq6TjgOY9FlfQCYA7wQCkRjSDnaP/UpR23esqb+5JuAE4i6cuNAx8DZgBExBXAWcD7JE0A24GzW5zY2kOugY9ZlbVIqi1d7jkbB8Yj4u70/TdIBj42pNypsg6OBY6WdGbTtjHg4cZ0Ces/56jZaMqb+xGxuMv+y4HL88bhgY/VSp754hHxqKT1kl4SEQ8BbwZ+2rfgbCB8z4BNJmkv4KXAO4F7mnb9N+DFpQQ1wpyjZqOpCrnvgY/VRo9nEi8Erpc0E1gLnF94YDYwPptsbSwkmf99W/PVHUkLgI6PAbZi9ZKjkk4heQLnGHBlRHxq0v6/BM5J304nGeTOjYhfp0872wbsAib85FmzclTl+9kDH6uVvEkVEfcB/iKskSo0rFY5x5E82GDylLZXkz7V0QYn5zz/MeALwFtIpiffI+nmiNh9dT4i/h74+/T4twH/IyJ+3VTMGyNiSxGxm1nvqvD9XOmBz8MPP1x2CDZEqnI2oc42bdrEZz/72SmXs3nz5gKi2ZPrgLVxLM+d4oakucAR+FHWA9VDjh4PrImItennbwTOpP205MXADVMKsuYee+wxvva1r025nGuvvbaAaBJvf/vbCynn1ltvLaQcgPPOO6+Qcn74wx8WUg5AUWvp7dixo5BynnnmmczHVuX7udIDH7O8qjB/1MrVSx3wVJp6i4gLWmzbDKiEcEZezhw9DFjf9H4cOKHVgZL2BU4Bmv9/B/B9SQF8Oe9aIWZWnCr00TzwsdqoytkEK0+P9w94Ko3ZgLTJ0U5LD7QanLZ7ZO3bgB9Oys0TI2KjpOcDt0l6MCLu7Cl4M+tZVfpoHvhYrVQhqaxcPdQBT6UxG6AWOdpp6YFxYH7T+3nAxjbHns2k3IyIjel/N0n6Fkm+e+BjVoIq9NHKv+ZkVpDG2QSvBj66JteBjPWg1VSaw9qU35hK882mzY2pNCskLekxdLOR0EOO3gMskHRU+vTNs4GbW5T7POANwHeatu0naVbjZ+BkJi1ia2aDUZU+mq/4WG1U5TKqlaeHaTTgqTRmA9PDIoYTki4AbiW5B+/qiFgl6b3p/ivSQ98OfD8inmr6+AuAb0mCpL/zjxHxvQJ+DTPLqSp9NA98rFaqcOOclatFHeg0jQY8lcZsoPK20xGxDFg2adsVk95fA1wzadta4JheYjSz4lWhj+aBj9VGVc4mWHl6rAO7p9IAG0gGN3/couzGVJp3NW3bD5gWEduaptJ8vMfwO5o+fXrMmDGj0DJ37dpVaHkABx54YOFlTkxMFF7mvvvuW3iZADt37iy8zIh2FyB7d9BBBxVa3qOPPsrWrVu7PiXP7bTZaKpK7nvgY7VShaSycvWwiO1QTKWZMWMGL37xiwstc+vWrYWWB/DOd76z8DK3bCn+gXmveMUrCi8TYP369d0Pyum3v/1t4WUuXry40PL+9E//NPOxbqfNRlMVct8DH6uNqpxNsPL0Wgc8lcZsMNxOm42mquS+Bz5WK1WYP2rlch0wqzbnqNloqkLue+BjtVGVswlWHtcBs2pzjpqNpqrkvgc+VhtVSSorj+uAWbU5R81GU1Vy3wMfq5UqJJWVy3XArNqco2ajqQq574GP1UZVziZYeYalDki6Gjgd2BQRC8uOx2xQhiVHzaxYVcn98u8yMivQtGnTdr9sNDXXgQrXg2uAU8oOwqwMQ5KjZlawKuS9WxyrjcbZhMbLRs/kOlDVehARdwK/LjsOs0Eblhw1s2Ll7aNJulrSJkk/abNfki6TtEbSA5JemSUOD3ysVvxlanXqVElaImm5pOW7du0qOxyzQtQpR80su5x5fw2dZ0acCixIX0uAL2UptC/3+LzqVa9i+fLl/SjarK1e549KGgOWAxsi4vTCA6sRSey1115TLuf6668vIJo9VWUOcVEiYimwFGCfffaJksMxm7K65egw2r59O6tXr55yOQ8++GAB0SQ2btxYSDkbNmwopByAzZs3F1LOI488Ukg5AL/61a8KKWfWrFmFlPPss89mPjZv7kfEnZKO7HDImcC1ERHAXZIOkHRIRHT8g/vhBlYrPc4bvQhYDcwuNhorg+8ZMKs256jZaJqU+3MkNV8lWZqe7MvqMGB90/vxdJsHPjYaejmTKGke8Fbgb4E/70dcNjg+m2xWbc5Rs9HUIve3RMSiqRTZYlvXmRE+7WK10eLGuTmN+yPS15IWH7sU+ACQ/XqtVdaw3Dgt6Qbg/wEvkTQu6T1lx2Q2CMOSo2ZWrD48gGocmN/0fh7Qdc6kBz5WK5OSaktELGp6PecSqqTGOiorSgnW+mIYOlURsTgiDomIGRExLyKuKjsms0HJm6OSTpH0UPr0pkta7D9J0uOS7ktfH836WTMbnIK/m28G3p0+3e01wOPd7u8BT3WzGulhCsWJwBmSTgP2BmZLui4i3tWXAK3vPI3GrNry5mj68JkvAG8hOcN7j6SbI+Knkw79weSH0+T4rJn1WQ+5fwNwEsnsnXHgY8AMgIi4AlgGnAasAZ4Gzs9SbtcrPpLmS/oXSaslrZJ0UeaozQYsz+JYEfGh9Gz7kcDZwO2jNuipY373sjiizyibDU7OHD0eWBMRayNiJ3AjydOcspjKZ4dCHdtwq6+cfbQ9ZkZExBXpoIdI/FlEvDgi/iAiMj1OOssVnwng/RGxUtIsYIWk23zGxKrGZ/t7Uqv87vEBFz6jbDYgPeRoqyc3ndDiuNdKup9kjv9fRMSqHJ8dZrVqw62+qtJH6zrwSefLPZL+vE3SapLGxEllldNrUkXEHcAdRcYyDOqY3z3Ugd1nhQEkNc4KZ/kbTOWzZiOpRY52eqxtlic3rQSOiIgn06nL3yZZ1LCnpz4Nkzq24VZfQzHwaZYuJHQccHdfojGbgqqcTRhWdcjvHuuAzyibDUibHO30WNuuT26KiCeafl4m6YuS5mT5bJ3UoQ23+qpKHy3zwEfS/sA3gYubG5mm/UuAJQCHH354YQGa5eGF8XrTKb+bc3sYtKgD3RZJG4ozyvPmzeOTn/xkoWUuWVL8/9Zdu3YVXubXv/71wsv89Kc/XXiZAD/60Y8KL/Ouu+4qvMx//ud/LrS8PCu452yn7wEWSDoK2EByP+YfNx8g6YXAf0RESDqe5P7lx4Ct3T5bF3n6aPvtt9+AozNLVKGPlmngI2kGSUJdHxE3tTom7UgsBVi0aFGtLiXbcKjK2YRh0y2/m3N72rRplc7tHs4mg88omw1M3nY6IiYkXQDcCowBV0fEKknvTfdfAZwFvE/SBLAdODsiAmj52WJ/o/Ll7aMdfPDBlW7HrZ6q0kfrOvCRJOAqYHVEfKb/IZn1pipJNUzqlt891gGfUTYbkF5yNCKWkTy6tnnbFU0/Xw5cnvWzdVK3Ntzqqyp9tCxXfE4EzgV+LOm+dNuH08bErFKqkFRDpnb53UOnymeUzQbI7XShateGW31VIfezPNXt32g9j92sUqpyNmGY1C2/e60DPqNsNhhup4tVtzbc6qsquZ/rqW5mVVeFG+esXK4DZtXmHDUbTVXIfQ98rDaqcjbByuM6YFZtzlGz0VSV3C9/6GVWoLGxsd0vG03NdaDK9UDSKZIekrRG0iVlx2M2KMOSo2ZWrCrkva/4WG1U5WyClWdY6oCkMeALwFtIHol9j6SbI8KrrVutDUuOmlmxqpL7lR74rFmzprCyHn744ULKKTKmohx99NGFlbVgwYJCyikypjyqMH+0ziKCZ555ZsrlLFtW3LMATjvttOe8H5I6cDywJiLWAki6ETgT8MDHam9IcrS2DjzwQN7xjndMuZwdO3YUEE3ihBNOKKScbdu2FVIOwMKFCwspZ/v27YWUA/C6172ukHIeffTRQsqZMWNGruOrkPuVHviY5VGVswlWniGqA4cB65vejwN7fPM3r7Y+d+7cwURm1kdDlKNmVqCq5L4HPlYbVUkqK88Q1YFWj5/dYzX15tXWFyxY4NXWbegNUY6aWYGqkvse+FitVCGprFxDUgfGgflN7+cBG0uKxWyghiRHzaxgVch9D3ysNiRVYv6olWeI6sA9wAJJRwEbgLOBPy43JLP+G6IcNbMCVSX3PfCxWqnC2QQr1zDUgYiYkHQBcCswBlwdEatKDstsIIYhR82seFXIfQ98rDaqMn/UyjNMdSAilgHFPd7ObAgMU46aWXF6yX1JpwCfIzlBeGVEfGrS/pOA7wC/SDfdFBEf71SmBz5WG3mTStJ84FrghcCzwNKI+FyfwrMBcKfKrNqco2ajqYc+Wtb17n4QEadnLdcDH6uVnF+oE8D7I2KlpFnACkm3eRHJ4eZOlVm1OUfNRlPO3O/Lenfl32VkVpDGjXONVzcR8UhErEx/3gasJllfxYbU5DpQhRspzex3eslRSadIekjSGkmXtNh/jqQH0tePJB3TtG+dpB9Luk/S8oJ/HTPLqEUfbY6k5U2vJZM+0mq9u1Z9tNdKul/SLZJe3i0OX/GxWpl0NmHOpC+6pem6KHuQdCRwHHB3/6KzQejlbHKGecTnAB9M3z4JvC8i7k/3rQO2AbuAiYhY1HPwZiOgD9NdfgG8ISJ+I+lUkrWvmhcEfmNEbJl65GY2FZNyf0uX78ss692tBI6IiCclnQZ8G1jQKQYPfKw2Wswf7ZZUjc/tD3wTuDginuhXfNZ/Pd486Y6V2YD0kKNdp7tExI+ajr+LZF0sM6uQHnK/63p3zX22iFgm6YuS5nT6PvbAx2qlh07vDJJBz/URcVNfgrKB6uGKz1B0rNasWbPljDPO+GWGQ+cApQ3CLrvssqyHlhrnBz/4we4HJUqNE2D69Exf1WXHeUTWA3PmaKvpLie0ORbgPcAtTe8D+L6kAL7c7qq/mfVfztzvut6dpBcC/xERIel4klt4HutUqAc+Vht5F8eSJOAqYHVEfKZvgdnA9LhA2lB0rCJibpbjJC0fhul2jrNYQxRnqxztNC05y3SXRtlvJMnP1zVtPjEiNkp6PnCbpAcj4s4ewzezHuX9fm633p2k96b7rwDOAt4naQLYDpwdES3bhwYPfKxWcp5NOBE4F/ixpPvSbR9O11exIdWiDnS718sdK7MBapGjnaYld53uAiDpFcCVwKkRsfuMb0RsTP+7SdK3SK7wOj/NSpB3Rkar9e7SAU/j58uBy/OU6YGP1Ube+aMR8W+07vTakGpTB7rd6+WOldmA9DDPP8t0l8OBm4BzI+JnTdv3A6ZFxLb055OBjosbmll/VGUNLw98rDaqklRWnh7rQN06VsNyD4PjLNZQxNnDCaos010+ChwMfDGZwbz76YovAL6VbpsO/GNEfK/I38fMsqlKH80DH6uVKiSVlauHS+m16lgNy83bjrNYwxIn9GW6y58Af9Lic2uBYyZvN7NyVKGP5oFPiR5++OFCyrnlllu6H5RRUTGtWbOmkHLy6PHGdsth5syZHHrooVMuZ9OmTQVEs6de64A7VmaD4Xa6fBHBxMTElMuZOXNmAdEkiuoQ77///oWUA8XFtPfeexdSDsCOHTsKKaeMHKxK7pcfgVmBxsbGdr9sNDXXgVGqB91Wt68KSfMl/Yuk1ZJWSbqo7JjakTQm6V5J3y07lk4kHSDpG5IeTP+ury07pk5GNUfNRl0V8t5XfKw2qjJ/1MozqnUg4yKsVTEBvD8iVkqaBayQdFtFY70IWA3MLjuQLj4HfC8izpI0E9i37IDaGdUcNRt1Vcl9X/GxWqnC2QQr14ieTd69CGtE7AQai7BWTkQ8EhEr05+3kQwsDis3qj1Jmge8leRJfpUlaTbwepI1yYiInRGxtdSguhjRHDUbeVXIew98rDYa80cbLxs9k+vACNWDVouwVm4wMZmkI4HjgLtLDqWVS4EPAM+WHEc3LwI2A19Np+VdmT5hsJJGOEfNRlpV+mhucaxWqnA2wco1omeTMy/CWhWS9ge+CVwcEU+UHU8zSacDmyJiRdmxZDAdeCXwpYg4DngKqOw9XjCyOWo28qqQ95kGPsNy06yNtsb80bKTatjUKb8n14ERqgeZFmGtCkkzSAY910fETWXH08KJwBmS1pFMG3yTpOvKDamtcWA8IhpXzb5BMhCqpBHO0b6pUxtu9VWVPlrXgU/TTbOnAi8DFkt6Wb8DM8urKkk1TOqW3yPcqdq9CGt6c/vZwM0lx9SSkkWPrgJWR8Rnyo6nlYj4UETMi4gjSf6Wt0fEu0oOq6WIeBRYL+kl6aY3A1V8UAQw0jnaF3Vrw62+qtJHy/JUt903zQJIatw0W9mG1UaXv0Rzq11+j2IdaLcIa8lhtXMicC7wY0n3pds+nK6lZL25ELg+HfSuBc4vOZ6ORjFH+6h2bbjVVxVyP8vAp9VNsyf0Jxyz3lVlcawhU6v8HuU60GoR1iqKiH+j9T1JlRQRdwB3lBxGRxFxH7Co7DiyGOUc7ZNateFWX1XJ/SwDn0w3zUpaAiwBOPzww6cYlllvqnA2Ych0ze/m3B6Gv+8wxGg2ypyjhcrdR5szZ06/YzJrqQq5n2Xolemm2YhYGhGLImLR3Llzi4rPLLOqzB8dMl3zuzm3q/539f0DZtXmHC1c7j7a7NlVX4/X6qgqfbQsA5+huWnWrApJNWRql9/uVJlVm3O0ULVrw62+qpD3Xae6DdlNszbCqjJ/dJjULb9dB8yqzTlarLq14VZfVcn9LPf4DM1NszbaGpdRLZ865bfrgFm1OUeLV6c23OqrKrlf/tDLrEB5L6N64bf66WUaTbd6oMRl6f4HJL0y62fN7Lny5qjz06weBvnd3I4HPlYbeW+c88Jv9dPLjdMZ68GpwIL0tQT4Uo7Pmlkqb446P83qoU99tJa534kHPlYrOc8m7F74LSJ2Ao2F32yI9XDFJ0s9OBO4NhJ3AQdIOiTjZ82sSc4cdX6a1cQAv5vb8sDHaqNx41zjlUGrhd8O60twNhCT60CB9aDdMa5DZjn0kKPOT7Ma6FMfLXeOZ3q4QV4rVqzYIumXXQ6bA2zpx78/BY4pm0HGdETWA1esWHHr2NhY88pse0ta3vR+aUQsbXqfaeE3+52dO3duWbduXbfchi515Lzzzsv6T2apa7vrSIs6AMXUg3bHuA6Z5dBDjjo/C7Z27dotZ511VqX6aNddd12WwwYa07XXXpvlsIHG9IlPfCLLYaPUR8ud430Z+ERE1xVMJS2PiEX9+Pd75ZiyqWJMABFxSs6PZFr4zX4nS25DcXUkbzk91AHIVg/aHTMzw2fNLNWndtr5mYP7aMVxTNkNOPfb8lQ3G2Ve+M0gWz24GXh3+gSZ1wCPR8QjGT9rZr1zfpqNpqnkflt9ueJjNgy88JtB+3og6b3p/itI1sg4DVgDPA2c3+mzJfwaZrXk/DQbTVPJ/U4UUc50V0lLJs3lK51jyqaKMVm1FFVHXNfMzAavim2vY8qmijFVSWkDHzMzMzMzs0HxPT5mZmZmZlZ7Ax/4SDpF0kOS1ki6ZND/fot45kv6F0mrJa2SdFHZMTVIGpN0r6Tvlh0LgKQDJH1D0oPp3+u1Zcdk1VJUfku6WtImST8pMj4zM2vPfbTs3EcbTgOd6iZpDPgZ8BaSR9DdAyyOiJ8OLIg9YzoEOCQiVkqaBawA/qjMmBok/TmwCJgdEadXIJ6vAT+IiCvTJ2zsGxFbSw7LKqLI/Jb0euBJkhWZFxYaqJmZ7cF9tHzcRxtOg77iczywJiLWRsRO4EbgzAHH8BwR8UhErEx/3gaspgIrO0uaB7wVuLLsWAAkzQZeD1wFEBE7nVA2SWH5HRF3Ar8uMjgzM+vIfbSM3EcbXoMe+BwGrG96P04FKnCDpCOB44C7Sw4F4FLgA8CzJcfR8CJgM/DV9NLulZL2Kzsoq5RK57eZmXVU6TbcfbSO3EfLaNADH7XYVonHyknaH/gmcHFEPFFyLKcDmyJiRZlxTDIdeCXwpYg4DngKKH3+r1VKZfPbzMy6qmwb7j5aV+6jZTTogc84ML/p/Txg44Bj2IOkGSQJdX1E3FR2PMCJwBmS1pFcan6TpOvKDYlxYDwiGmdavkGSZGYNlcxvMzPLpJJtuPtombiPltGgBz73AAskHZXeeHU2cPOAY3gOSSKZE7k6Ij5TZiwNEfGhiJgXEUeS/I1uj4h3lRzTo8B6SS9JN70ZKP3mQquUyuW3mZllVrk23H20zDG5j5bR9EH+YxExIekC4FZgDLg6IlYNMoYWTgTOBX4s6b5024cjYll5IVXWhcD1aYO4Fji/5HisQorMb0k3ACcBcySNAx+LiKsKC9bMzJ7DfbSh5z5aBgN9nLWZmZmZmVkZBr6AqZmZmZmZ2aB54GNmZmZmZrXngY+ZmZmZmdWeBz5mZmZmZlZ7HviYmZmZmVnteeBjZmZmZma154GPmZmZmZnVngc+ZmZmZmZWe/8fcjIYtP8bLDoAAAAASUVORK5CYII=\n",
"text/plain": [
"![\"C1\"](\"../data/C8/FMP_C8_Audio_C-major-scale.png\")
\n",
"\n",
"![\"C0\"](\"../data/C0_nav.png\") | \n",
" ![\"C1\"](\"../data/C1_nav.png\") | \n",
" ![\"C2\"](\"../data/C2_nav.png\") | \n",
" ![\"C3\"](\"../data/C3_nav.png\") | \n",
" ![\"C4\"](\"../data/C4_nav.png\") | \n",
" ![\"C5\"](\"../data/C5_nav.png\") | \n",
" ![\"C6\"](\"../data/C6_nav.png\") | \n",
" ![\"C7\"](\"../data/C7_nav.png\") | \n",
" | \n",
"