MFCCs: The 20 Numbers Every Model I Train Has to Beat
How MFCCs are computed, and why a 40-number MFCC store is the floor every learned model in my music engine has to clear: the baseline table, the ship rule, and the CNN that did not make it.
Twenty numbers my CNN could not shake
Mel-frequency cepstral coefficients are a classic. My signal engine computes them from the shared log-mel spectrogram (librosa.feature.mfcc on 128 mel bands, 20 coefficients), then keeps the mean and the standard deviation of each over the 60 second excerpt: 40 of the 109 feature slots.
MFCCs in 40 seconds
timbre_analysis.py: the MFCC slots
mfcc = librosa.feature.mfcc(S=sp.logmel, n_mfcc=settings.n_mfcc) # n_mfcc: 20
...
mfcc=mfcc.mean(axis=1).tolist(), mfcc_std=mfcc.std(axis=1).tolist())The baseline table
One report (2026-10-09, models/reports/20261009T100736Z), one probe, every store on the same 3,141 tracks. Genre probe: a logistic regression on each store, folds grouped by artist, macro-F1 (always answering the most common genre scores 0.042). Retrieval: for each track, are other tracks by the same artist in its 10 nearest neighbours, with its own album excluded (chance 0.376).
| store | genre probe | album-excluded R@10 | |---|---|---| | track-fused | 0.362 | 0.768 | | CNN v1 | 0.359 | 0.668 | | hybrid Transformer | 0.352 | 0.677 | | AST (frozen) | 0.343 | 0.760 | | CLAP (frozen) | 0.339 | 0.774 | | MERT (frozen) | 0.292 | 0.655 | | MFCC, 40 numbers | 0.258 | 0.618 | | untrained CNN | 0.244 | 0.693 |
Every store against the MFCC floor
The ship rule
The rule: beat the MFCC store by 0.15 on both columns. In the integrity run of 2026-10-08 my first CNN reached +0.122 on the probe and +0.050 on album-excluded retrieval, and an untrained CNN of the same shape retrieved better than the trained one (0.692 against 0.668). Verdict: does not ship.
Coefficient 1 roughly follows the tilt of the spectrum, and it shows across the catalogue: median 168.7 for classical, 77.2 for punk.
The trade-off
MFCCs see the spectral envelope and nothing else: no rhythm, no harmony, no lyrics. The learned stores beat them; the question is always by how much, against what it costs to train and serve them.
Part 5 of 14, next: Pitch Tracking With pYIN: From 24 Seconds to 2.2 Per Song.
Part 5 of 14, next: Pitch Tracking With pYIN: From 24 Seconds to 2.2 Per Song.
More: LinkedIn · Instagram. Portfolio and case studies: designsbyduhart.org.
If any of this saved you an afternoon, Buy me a coffee.