Spatiotemporal Acoustic Manifold
- → Birdsong treated as a trajectory through 3D space rather than just a waveform.
- → Each point encodes the timbre (tone colour) as high‑dimensional MFCC features.
- → PCA compresses features into PC1, PC2, PC3 (X, Y, Z axes).
- → Loops, twists, clusters reveal phrases, motifs, and tonal vs noisy sections.
- → Inspired by Lucio Arese's "Visualizing Bird Songs: A Mathematical Model", adapted as an interactive web piece.
Technical Implementation
Feature Extraction · Python + librosa
40 MFCCs + chroma + spectral centroid/bandwidth + rolloff + zero‑crossing rate per
frame.
Features standardised and reduced to 3D with PCA.
Frame‑aligned t (time) and energy (RMS) exported to JSON.
3D Manifold · Three.js + WebGL
Points smoothed with Catmull–Rom spline to create a continuous ribbon path.
Moving dot synced to audio.currentTime → frame index.
Dot size and colour map to energy (silence → blue,
peak calls → orange).
Visual Design
Fading comet tail shows last 280 frames without cluttering the full path.
Ghost trajectory reveals global shape at low opacity.
Axes labelled as PC1 / PC2 / PC3 to emphasise the mathematical model.
BirdSong Mathematical Model
A mathematical model visualized to explore the patterns and structures in bird song compositions. Inspired from Lucio Arese's "Visualizing Bird Songs: A Mathematical Model".