LeapSinger — synthesis demo

A lightweight singing-voice synthesizer. The multi-speaker acoustic model (harmonic-excitation rectified flow, one step) predicts a mel-spectrogram, rendered to audio by the bundled NHVSing vocoder. This page puts the released versions side by side:

• V1 (v0.1.0) — flow + GAN, 1 step. Vocoder: NHVSing V3.2 (checkpoints/nhv_v3_2.onnx).
• V2 (v2.0.0) — GAN-free rectified-flow teacher distilled into a 1-step student. Vocoder: NHVSing V3.2.1 (checkpoints/nhv_v3_2_1.onnx).

V1 vs V2 at a glance

The two models on the 71 held-out phrases (songs never seen in training), synthesised in one step from the true phoneme durations and F0 and measured with tools/eval_ab.py. Ratios are generated / recording, so 1 means "like the recording"; per-band values are 68 Hz–945 Hz / 973 Hz–2.4 kHz / 2.5–6.1 kHz / 6.3–15.5 kHz. Bold = closer to the recording (ratios closer to 1, offsets closer to 0, error lower).

• V2 keeps the detail that averaging tends to lose — fine time texture, the spectral shape of consonants, phoneme-rate motion — its harmonic lines are cleaner, and the regular pattern the GAN left is gone.
• V2 is not better everywhere: the plain mel error is 0.24 dB higher (the excess is random detail that does not coincide with the recording — smoothing both sides reverses the order, and the phrase-level level is closer for V2); the first and last 20 ms of phonemes are 0.13 dB worse, and the (rare) devoiced vowels are clearly worse (error 7.8 → 11.2 dB, spectral centroid 1.3 kHz low).
• One checkpoint each; V1's target is the raw mel, V2's the loudness-corrected mel (the ratio metrics do not depend on the target's level).

1. Copy-synthesis — ground truth vs model (3 speakers)

Per singer, 3 phrases from training songs and 2 from held-out songs (eval, never seen in training), driven by the real F0, durations and phonemes taken from each recording. Columns: the original audio, the synthesis by V1 / V2, the GT mel, and for each version the model's mel (top) with the mel of the vocoded output (bottom). mel-L1 (shown for V1) is the L1 distance to the GT mel (lower = closer); it does not track perception, so judge by ear. The other versions train on a loudness-corrected target, so their distance to the raw GT is not comparable and is not shown.

• V2: Audio is gain-matched to each recording's RMS for listening. The V2 model sings at a constant level (its training target, −30 dB), so its own output sits a few dB below these recordings; the mel figures are at that level, the same for the model mel and the vocoded mel.

Oniku Kurumi

Natsume Yuuri

Namine Ritsu

2. Singing unseen scores

These are songs the model has never seen, sung from a musical score. The score is turned into an F0 curve and phoneme durations by our pitch and duration models, then the acoustic model and the NHVSing vocoder render the audio. Three children’s songs, each sung by all three voices, every version stacked in each cell.

• V2: The same excerpt as V1 (from the first note, the same length), gain-matched to V1's RMS. F0 and phoneme durations come from the current pitch and duration models, so the difference from V1 in this section is not only the acoustic model. Vocoder NHVSing V3.2.1.

Bonus — Momiji (もみじ), a three-part chorus. The three voices arranged into harmony (V1).

3. Style control

The full model carries Namine Ritsu’s three voice styles — default (kire), normal, and soft. One phrase (a real recording, re-rendered by copy-synthesis) is sung with the style blended continuously from default toward normal, and toward soft. Only the timbre changes — the notes, timing and pitch stay identical. This section uses the V1 three-style model (v0.1.0); the later release models have no style input.

Original recording (reference):