Title: Let it Bee - Towards NMF-inspired Audio Mosaicing
----
Head:

<script type="text/javascript" src="/assets/multitrack/player.js"></script>

<link href="/assets/multitrack/screen.css" rel="stylesheet" />

<script type="text/javascript">
    $(document).ready(function() {
        $(".player").multitrackAudioPlayer({mute: false, repeat: true});
    });
</script>

----


Text:

## Abstract
<p align="justify">
A swarm of bees buzzing "Let it be" by the Beatles or the
wind gently howling the romantic "Gute Nacht" by Schubert
- these are examples of audio mosaics as we want to create them.
Given a target and a source recording, the goal of audio
mosaicing is to generate a mosaic recording that conveys
musical aspects (like melody and rhythm) of the target,
using sound components taken from the source. In this
work, we propose a novel approach for automatically generating
audio mosaics with the objective to preserve the timbre
of the source in the mosaic. Inspired by algorithms for
non-negative matrix factorization (NMF), our idea is to use
update rules to learn an activation matrix that, when multiplied
with the spectrogram of the source recording, resembles
the spectrogram of the target recording. However,
when applying the original NMF procedure, the resulting
mosaic does not reflect the timbre of the source well. As
our main technical contribution, we propose an extended
set of update rules for the iterative learning procedure that
supports the development of sparse diagonal structures in
the activation matrix. We show how these structures better
retains timbral characteristics of the source in the resulting
mosaic.
</p>
<div align=center>
(image: teaser.jpg width:400)
</div>

## Audio Examples

### LetItBe mosaiced with Bees

<table>
	<tr>
		<td>
			<div align=center>
				<b>[Target]</b> LetItBe
			</div>
		</td>
		<td>
			<div align=center>
				<b>[Source]</b> Bees
			</div>
		</td>
	</tr>
	<tr>
		<td>
			<div align=center>
				(audio: Beatles_LetItBe.mp3)
			</div>
		</td>
		<td>
			<div align=center>
				(audio: Bees_Buzzing.mp3)
			</div>
		</td>	
	</tr>
</table>

Click the button in the middle to activate the player.
<div class="player">
(audio: Beatles_LetItBe.mp3 title:<b>[Target]</b> LetItBe )
(audio: Beatles_LetItBe_mosaicingWith_Bees_Buzzing.mp3 title:<b>[Mosaic]</b> Sparse diagonal activations)
(audio: Beatles_LetItBe_mosaicingWith_Bees_Buzzing_plainNMF.mp3 title: <b>[Mosaic]</b> Basic approach )
</div>

[Additional material](/resources/MIR/2015-ISMIR-LetItBee/LetItBe/)


### GuteNacht mosaiced with Wind

<table>
	<tr>
		<td>
			<div align=center>
				<b>[Target]</b> GuteNacht
			</div>
		</td>
		<td>
			<div align=center>
				<b>[Source]</b> Wind
			</div>
		</td>
	</tr>
	<tr>
		<td>
			<div align=center>
				(audio: Schubert_GuteNacht_SMD.mp3)
			</div>
		</td>
		<td>
			<div align=center>
				(audio: Wind_Blowing.mp3)
			</div>
		</td>	
	</tr>
</table>

Click the button in the middle to activate the player.
<div class="player">
(audio: Schubert_GuteNacht_SMD.mp3 title:[Target] GuteNacht)
(audio: Schubert_GuteNacht_SMD_mosaicingWith_Wind_Blowing.mp3 title:[Mosaic] Sparse diagonal activations)
(audio: Schubert_GuteNacht_SMD_mosaicingWith_Wind_Blowing_plainNMF.mp3 title: [Mosaic] Basic approach )
</div>

[Additional material](/resources/MIR/2015-ISMIR-LetItBee/GuteNacht/)


### FunkJazz mosaiced with Whales

<table>
	<tr>
		<td>
			<div align=center>
				<b>[Target]</b> FunkJazz
			</div>
		</td>
		<td>
			<div align=center>
				<b>[Source]</b> Whales
			</div>
		</td>
	</tr>
	<tr>
		<td>
			<div align=center>
				(audio: MusicDelta_FunkJazz.mp3)
			</div>
		</td>
		<td>
			<div align=center>
				(audio: Whales_Singing.mp3)
			</div>
		</td>	
	</tr>
</table>

Click the button in the middle to activate the player.
<div class="player">
(audio: MusicDelta_FunkJazz.mp3 title:[Target] FunkJazz )
(audio: MusicDelta_FunkJazz_mosaicingWith_Whales_Singing.mp3 title:[Mosaic] Sparse diagonal activations)
(audio: MusicDelta_FunkJazz_mosaicingWith_Whales_Singing_plainNMF.mp3 title: [Mosaic] Basic approach)
</div>

[Additional material](/resources/MIR/2015-ISMIR-LetItBee/FunkJazz/)


### Stepdad mosaiced with Chainsaw

<table>
	<tr>
		<td>
			<div align=center>
				<b>[Target]</b> Stepdad
			</div>
		</td>
		<td>
			<div align=center>
				<b>[Source]</b> Chainsaw
			</div>
		</td>
	</tr>
	<tr>
		<td>
			<div align=center>
				(audio: Stepdad_MyLeatherMyFurMyNails.mp3)
			</div>
		</td>
		<td>
			<div align=center>
				(audio: Chainsaw_Sawing.mp3)
			</div>
		</td>	
	</tr>
</table>

Click the button in the middle to activate the player.
<div class="player">
(audio: Stepdad_MyLeatherMyFurMyNails.mp3 title:[Target] Stepdad )
(audio: Stepdad_MyLeatherMyFurMyNails_mosaicingWith_Chainsaw_Sawing.mp3 title:[Mosaic] Sparse diagonal activations)
(audio: Stepdad_MyLeatherMyFurMyNails_mosaicingWith_Chainsaw_Sawing_plainNMF.mp3 title: [Mosaic] Basic approach )
</div>

[Additional material](/resources/MIR/2015-ISMIR-LetItBee/Stepdad/)


### Freisch&uuml;tz mosaiced with AirRaid

<table>
	<tr>
		<td>
			<div align=center>
				<b>[Target]</b> Freisch&uuml;tz
			</div>
		</td>
		<td>
			<div align=center>
				<b>[Source]</b> AirRaid
			</div>
		</td>
	</tr>
	<tr>
		<td>
			<div align=center>
				(audio: Weber_Freischuetz.mp3)
			</div>
		</td>
		<td>
			<div align=center>
				(audio: AirRaid_Ringing.mp3)
			</div>
		</td>	
	</tr>
</table>

Click the button in the middle to activate the player.
<div class="player">
(audio: Weber_Freischuetz.mp3 title:[Target] Freisch&uuml;tz )
(audio: Weber_Freischuetz_mosaicingWith_AirRaid_Ringing.mp3 title:[Mosaic] Sparse diagonal activations)
(audio: Weber_Freischuetz_mosaicingWith_AirRaid_Ringing_plainNMF.mp3 title: [Mosaic] Basic approach )
</div>

[Additional material](/resources/MIR/2015-ISMIR-LetItBee/Freischuetz/)


### Vermont mosaiced with RaceCars

<table>
	<tr>
		<td>
			<div align=center>
				<b>[Target]</b> Vermont
			</div>
		</td>
		<td>
			<div align=center>
				<b>[Source]</b> RaceCars
			</div>
		</td>
	</tr>
	<tr>
		<td>
			<div align=center>
				(audio: TheDistricts_Vermont.mp3)
			</div>
		</td>
		<td>
			<div align=center>
				(audio: RaceCar_Engine.mp3)
			</div>
		</td>	
	</tr>
</table>

Click the button in the middle to activate the player.
<div class="player">
(audio: TheDistricts_Vermont.mp3 title:[Target] Vermont )
(audio: TheDistricts_Vermont_mosaicingWith_RaceCar_Engine.mp3 title:[Mosaic] Sparse diagonal activations)
(audio: TheDistricts_Vermont_mosaicingWith_RaceCar_Engine_plainNMF.mp3 title: [Mosaic] Basic approach )
</div>

[Additional material](/resources/MIR/2015-ISMIR-LetItBee/Vermont/)