Harmonica: Accurate and Lightweight Instrument-Agnostic Music Transcription

The model can run on an iPhone 15. Please refer to Appendix A in the arXiv version for details.

Links: [ Paper ] | [ Citation ]

We introduce Harmonica, a family of instrument-agnostic automatic music transcription (AMT) models built around multi-depth harmonic convolution. At each model scale, Harmonica achieves the best performance among the evaluated models: the x-large model attains state-of-the-art performance in instrument-agnostic transcription, while the medium variant offers competitive accuracy with faster inference than all baselines. Pushing the limit of computational efficiency, the nano variant has only 26.3K parameters and runs at 1,622.5x real time, yet achieves a frame F1 of 0.796 on the development set, 14.6 percentage points higher than Basic Pitch. We further demonstrate that multi-depth harmonic convolution effectively exploits harmonic information to benefit transcription performance through comparative experiments with existing harmonic aggregation methods, including harmonic stacking, harmonic attention, single-depth harmonic convolution, and the HD-Conv layer.

Together, these results demonstrate the potential of efficient, instrument-agnostic architectures for practical AMT, with Harmonica already deployed as the Audio-to-MIDI service in BandLab Studio to support accessible music creation.

Performance at a Glance

Performance vs model size

We define the new performance-efficiency frontier. Our models have the best performance at each parameter scale.

Performance vs inference speed

We define the new performance-speed frontier as well. At each performance level, our model offers the fastest inference speed.

Frame F1 per instrument family on Slakh2100

This figure breaks performance down by instrument family. Our x-large model is the strongest on every family, and its scores are more evenly distributed across families. On the other hand, many baselines such as PerceiverTF and HFSFormer degrade sharply on the harder families, particularly chromatic percussion and synth pad. This suggests that the proposed architecture generalizes well across diverse pitched instruments.

Demonstration

Here we demonstrate the model’s capability to transcribe various music pieces.

Full Song Transcription

Below we present the song Angelina from Tommy Emmanuel along with transcriptions from different models. Note that this is an out-of-domain case for all models, since there is no fingerstyle solo guitar in the training set.

Original Audio

A well-known fingerstyle arrangement for solo acoustic guitar.

Ours (x-large)

An accurate transcription.

MT3

Many duplicated and hallucinated notes. The polyphony level is obviously beyond with what a single guitar can do.

Basic Pitch

Lots of notes are broken into multiple pieces.

hFT

Consistently hallucinates notes in the high register.

YourMT3+

Onset prediction is decent; offset / duration is off.

Transkun

Many notes are missing in the output.

SFT-CRNN

Many note onsets are not paired with note offsets, depressing its frame-level performance.

Quality Showcase

Our model is capable to handle various instruments, various timbre, various styles and genres, more accurately than prior models.

Click any button to play that model's transcription of the segment; only one clip plays at a time.

Citation

@article{ou2026harmonica,
  title={Harmonica: Accurate and Lightweight Instrument-Agnostic Music Transcription},
  author={Ou, Longshen and Martel, H{\'e}ctor and Hennessy-Priest, Joe and Cho, Taemin},
  year={2026}
}