Guidance, Not Adaptation: What Makes an Open Text-to-Music Model Sing isiZulu
Gideon Gyimah ⋅ ⋅
Abstract
Open text-to-music models claim support for fifty or more languages. To our knowledge, none has been evaluated on an African language. We adapt ACE-Step 1.5, a 2B-parameter flow-matching diffusion transformer, to isiZulu using 10.95 hours across ten genre families: chiefly Maskandi, Kwaito, Amapiano, Isicathamiya, Mbaqanga and Gqom. The corpus is itself largely model-generated: an artist drafted the isiZulu lyrics with a language model, corrected them, then produced the audio by prompting a music generation system. This is adaptation on synthetic in-language data, a regime increasingly common where field recordings are scarce. Three findings survive that provenance, and none of the failures behind them raised an error. First, guidance rather than the adaptation controls whether the model sings the requested isiZulu words. Composite lyric error differs by $-0.437$ between guidance 1 and 7 ($d_z = -2.57$, sign-flip permutation $p < 10^{-5}$, n = 60 prompts), with only 6% of it mediated by vocal prominence. Second, the default distilled checkpoint overrides the guidance argument to 1.0. This documented behaviour is logged but not surfaced to the caller, so sweeping guidance on a default install sweeps an inert parameter. Third, the caption-retrieval control we built to validate the genre metric was itself broken in three independent ways. Each pushed towards a false negative and none raised an error. Corrected, the correct caption ranks 6.6 places above a mismatched one (95% CI [3.0, 10.1], $p = 0.0008$, clustered over 63 captions), where the binarised statistic we first used found nothing.
Chat is not available.
Successful Page Load