MiniMaxMusicTraining is a group of loosely associated community members and researchers that are aiming to unlock the secret to properly fine-tuning MiniMax Music 3 on our own data.

If you wish to join with write access to upload your own results here, request membership. Users with a Hugging Face profile that demonstrate clear intent with past published models and datasets will be accepted quickly.

The "tournament" training regime ablates configurations over several legs and iterates with small (potential) improvements.

The first tournament is intended to see how closely we can get to a single artists' style and structure.

• Round 1 compared with only 128 frame wide windowing (no positional anchors):
  • "vanilla" next-token predictive training
  • "XM" explorative modeling where each training step attempts to locate multiple training candidates, selecting the best loss only
  • "NextLat" claims to train the Qwen3 8B language audio model (LAM) into a "world model" with coherent long-range understanding
  • "XM+NextLat" combines the XM and NextLat training paradigms, exploring multiple training candidates that will form the most coherent world model

Round 1 tournament showed that 128 wide frame window may not be enough, but regardless, each objective differed in their outcome:

• "vanilla" gets us the closest to the target dataset but with a lo-fi result that requires the latent replanner to remove a low-frequency hiss and other artifacts. Some gap to the dataset likeness made it seem like we just stopped training too early or used too-low of a learning rate.
• "NextLat" on its own is a corpus-scale objective that requires larger dataset diversity than the tournament's single artist dataset offers, causing incoherent outputs and forgetting how to create lyrics together
• "XM" gave us high-quality outputs, some of the best ever produced by MiniMax Music3 - but lags behind "vanilla" training for likeness. It produces good outputs, they're just not like our training set.
• "XM+NextLat" shows that the explorative modeling loss can keep NextLat coherent even on a small, focused dataset, but still lags behind "vanilla" for likeness

Some intermediary experiments were ablated between tournament round 1 and 2:

• An intermediary round that switched from AdamW to Prodigy optimiser was run, but the model degraded too much.
• Lion optimiser was attempted as well, but again the model degraded.
• In both cases (Prodigy and Lion) the model degraded into "constant beats and clown noises", a consistent failure mode of this music model. Neat outputs, not useful results.
• A third intermediary round with AdamW and longer training (without position-aware windowing) was attempted, but the model merely trains on "here's what a song intro sounds like", and its results were discarded.

Round 2 with AdamW took the four legs of Round 1, extending their training duration from 1,000 to 10,000 steps. Round 2 also trained on anchored random 128 frame wide windows where the model should ideally learn random crops of the input song without losing positional information. Round 2 also had a small 128-song regularisation dataset of Suno tracks added in an attempt to prevent model collapse or forgetting due to the small training corpus.

• "vanilla" still gets closest to the data but (possibly) reflects the lower accuracy RVQ audio codes with its lower quality outputs (we're not sure on this one)
• "NextLat" keeps coherent thanks to regularisation dataset, but the potential now is that there's too much regularisation because it lacks likeness / similarity to the training set
• "XM", even with 10k regularised training steps still has the highest quality results but lowest likeness to the training split
  • XM only is dropped from future ablations at small dataset scale.
• "XM+NextLat" has the high quality results of "XM-only" training and the weak likeness of "NextLat" training, indicating it is a strong contender for large-scale finetuning

The main open question for me between round two and three then was: "How much does the frame windowing matter?"

Round three only adjusts this variable; we no longer use position-anchored frame windows. We switched to full song prediction which has an unfortunate impact on VRAM, requiring an upgrade from L40S (48G) to H100 (80G). Performance had a proportional slowdown in per-step time from 1.2 it/sec to 3.2 seconds per it. The full length being shown to the model means all token positions are predicted and the cross-entropy loss of each token position accumulates.

• "vanilla" full song training predictably becomes overcooked by 10k steps:
  • at 10k steps, when using the trigger words, 0.75 scale LoRA strength is required to have coherent outputs, 1.0 scale LoRA gives gibberish singing and repetition
  • however, due to regularisation dataset, not using trigger words gives coherent outputs even at 10k steps and 1.0 scale LoRA
  • by the 3k step checkpoint, coherence struggled for the trained trigger word
  • by just 1k steps, the full-strength LoRA produces similar style to its training data, but vocal identity lags behind
• "NextLat" full song training became incoherent, only saved by having too much regularisation applied
  • at 10k steps, the full 1.0 scale LoRA produces alien laser gun noises and repetition with gibberish vocals
  • at 10k steps the 0.75 scale LoRA produces more likeness to the style and singer identity but the tempo is unstable and increases as the clip plays
  • at 5k steps, the full 1.0 scale LoRA produces low-quality (like a high-pass cutoff was applied) but high similarity to style and weak correlation to singer identity
  • the 5k steps 0.75 scale LoRA produces mediocre quality outputs that do not sound like the training data at all
  • the 1k step LoRA doesn't sound like its training data at any applied strength
• "XM" full song training still produces coherent and highest quality results even up to 10k steps (requiring a drop to 0.75 LoRA scale though at this step count)
  • at 10k steps, the full 1.0 scale LoRA produces obviously-broken vocalist with nicely coherent structure
  • at 10k steps, the 0.75 scale LoRA gets pretty close to the training set structure, and singer identity is beginning to show, but I don't think we can train much longer before collapse/incoherence takes over
  • at 5k steps, the full 1.0 scale LoRA produces good likeness of singer identity but odd adherence to training set structure/style
  • at 5k steps, the 0.75 scale LoRA gets a middle-ground of style and identity but worse than 1k-2k of vanilla training at similar quality
  • by 2.5k steps we've got an okay resulting LoRA but only if you omit the trained trigger words and use full descriptor style prompts instead
  • at 1k steps if we omit the trigger words we can get a Wish.com likeness to the style and singer only at 0.75 LoRA scale; 1.0 scale is not like our training set.
• "XM+NextLat" full song training produced the highest quality (bandwidth sound; less high-pass filtering) and adherence to the training split
  • 10k steps is still too much; at 1.0 or 0.75 LoRA scale, we have unstable tempo/dynamics and counter-intuitively, weak adherence to training style/identity
  • 5k steps at 1.0 LoRA scale is sounding a little broken for tempo/dynamics, and 0.75 scale LoRA only partially saves it, but with weaker adherence
  • 2.5k steps trigger word inference at 1.0 LoRA scale sounds like circus music, but omitting the trigger word and using only descriptors gives us some of the best artist style and identity
  • 1k steps trigger word inference at 1.0 LoRA scale sounds good, but a little lacking in creativity. 0.75 LoRA scale sacrifices identity to get closer style and more creative outputs.
  • 1k steps omitting the trigger word and only using descriptors produces even better results than using the trigger word at 1.0 LoRA scale

It feels like one of the principle contenders for round four of the tournament is to adjust captioning style of the inputs, keeping the highest quality techniques from round three:

• "vanilla" training on full songs and random window crops
• "XM+NextLat" training on full songs and random window crops

Reducing our objective testing to just the best two and still testing the performance vs quality trade-offs of random window crops.