How Sound Looks in 3D: Pitch, Partials and Beats

Blog · Learn / Sound

How Sound Looks in 3D: Pitch, Partials and Beats

What a sound is made of, seen in 3D: the fundamental frequency, partials and harmonics, noise, pitch class, beats, brightness and tempo, measured and drawn.

7 min readAIKIZI

Sound visualization usually means a spectrum: bars or a curve showing how loud each frequency is right now. The flat music visualizer draws that well. But a spectrum alone does not tell you which note is being sung, which peaks belong to it, which part is breath and when the beat fell. Standing Sound measures each of those separately, about sixty times a second, and gives each one its own look in 3D. This post walks through them one at a time.

The sample song Bhakti, held, in the Corridor room of Standing Sound
Bhakti, a bansuri flute over strings, held in the Corridor.

What is a fundamental frequency?

The fundamental is the slowest vibration a pitched sound repeats, and it is the pitch you would name: the A at 440 hertz, the D a singer holds. Standing Sound finds it with YIN, a method that compares about 45 milliseconds of the sound wave with itself, shifted a little at a time, and looks for the shortest shift at which the wave lines up again. That shift is one period, and one over the period is the frequency.

It searches from 65 Hz to 1,100 Hz, the range of most voices and the lower range of melody instruments, and it only names a fundamental when the match is close and the sound is tonal rather than noisy. It finds one at a time: in a chord you see one line, or none. In the room the fundamental is the one vermilion colour, a line of points joined by a thread while the note is held.

What are partials in sound?

A string, a reed or a voice vibrates at many frequencies at once. Each of those frequencies is a partial. When partials sit at whole-number multiples of the fundamental, two, three, four times its frequency, they are harmonics, and their balance is most of what makes a flute sound unlike a voice on the same note.

To see them, Standing Sound reads a second, longer window: an 8192-point spectrum of about a fifth of a second, measured in steps of 5 to 6 Hz. Each reading keeps up to twelve of its strongest peaks between 40 Hz and 12 kHz. A peak that sits close to a whole-number multiple of the fundamental counts as its harmonic. In the room harmonics are hollow ivory squares, other partials are filled ivory points, and a thread joins a partial from one moment to the next while it lasts.

Why two windows? A long window sees pitch finely but blurs time; a short one sees change quickly but blurs pitch. The page uses the long one for partials and the short one for the fundamental, the beats and brightness.

Noise: the part with no pitch

Breath, a shaker, a hi-hat and the s in a word have no single pitch. Their energy is spread across many frequencies at once. Spectral flatness measures that: near zero for a few sharp peaks, near one when every frequency is about as loud as the next. Standing Sound reads the flatness of each moment, and where the sound is flat and its top, from about 2 to 11 kHz, is loud, it lets fall fine dust. Because noise has no note, the dust has no place on the circle: it scatters around it. Each beat adds a small burst of dust too.

The sample song Astral Caravan, held, in the Bloom room of Standing Sound
Astral Caravan, psychedelic synthwave, held in the Bloom.

Why pitch class goes around a circle

Double a frequency and you get the same note an octave higher. That is why the twelve note names repeat on a piano. A note's name, without its octave, is its pitch class. Standing Sound puts pitch class around a circle, twelve semitones to one turn, so every C stands at the same bearing whether it is a bass note or a whistle. The octave, the register, is set apart: in the Corridor it is how far from the middle a point stands, in the Bloom it is the height, and in the Score pitch simply climbs. A melody that circles back to its home note comes back to the same bearing.

The sample song Cassettes, Rain, held, in the Score room of Standing Sound
Cassettes, Rain, a lo-fi beat, held in the Score.

What a beat ring is

A beat, to a machine, is an onset: a moment when the spectrum jumps. The page splits the short spectrum into 24 bands from 40 Hz to 12 kHz and adds up how much each band rose since the last reading. That sum is the spectral flux. A moment counts as an onset when the flux rises well above its own recent median, measured in median absolute deviations, so a quiet song and a loud one both find their hits.

Every onset becomes a ring, not only the downbeats, and a stronger onset makes a stronger ring. In the Corridor and the Bloom you pass through them as time moves; in the Score each one stands upright at its moment.

Brightness is the spectral centroid

Some sounds are dark, a bass or an "oo", and some bright, a cymbal or an "ee". The spectral centroid puts a number on it: the balance point of the spectrum, the frequency where its energy would sit level on a scale. Standing Sound reads it from the short window and places it on a log scale that starts at 150 Hz. In the Score, brightness is depth: a bright moment stands nearer than a dark one. In every room the partials shift from ivory toward a cool blue as the sound grows brighter.

Tempo by autocorrelation

Tempo is not drawn as points; it is read out in the column on the left. The page keeps the strength of the onsets over the last eight seconds, sampled a hundred times a second, and compares that pattern with itself shifted in time. The shift at which the pattern lines up best, between 60 and 180 beats per minute, is the beat period. This is autocorrelation, and it is why the tempo settles after a few bars rather than at the first hit.

Common questions

What is a fundamental frequency?

The fundamental is the slowest vibration a pitched sound repeats, and it is the pitch you would name.

What are partials in sound?

A string, a reed or a voice vibrates at many frequencies at once. Each of those frequencies is a partial.

Why does a chord show only one vermilion line?

The fundamental is found one at a time, so a chord shows one line or none. The other notes of the chord still appear, as partials.

Can I keep the measurements?

Yes. Hold the sound, then save JSON for one line per point with its time, pitch in MIDI and hertz, loudness and kind, or PLY for the same points as a 3D point cloud.

See your own sound

Sing into the microphone, play an instrument, or press one of the three sample songs, then hold it and touch a point to hear it again. The tour of the free 3D music visualizer covers the rooms and controls, the recording guide shows how to save a 4K video with the sound, and the Standing Sound guide lists every key.

Open Standing Sound    See the flat visualizers