I fixed my tiny TTS by editing the weights directly, no retraining
Reported by r/LocalLLaMA1 source ↓
A Silencio-2 developer reports that direct edits to Pocket TTS weights improved several emotion and whisper tags on a held-out test set, without gradient-based retraining. The developer also reports glitches and inconsistent results.
Key points
- The edit increased clean whisper takes from 5 of 24 to 10 of 24 on the held-out test set
- Emotion separability rose from 0.45 to 0.71; untagged-speech WER stayed at 0.006
- The edit changed about 524,000 weights using roughly 5,000 free parameters and zero gradient steps
- Some outputs glitched, and the developer says the change is not a stable upgrade
Details
The Silencio-2 developer says the small fine-tune of Kyutai’s Pocket TTS had unreliable controls for emotions and whispering, despite using inline tags. They used Claude (Opus 5.5) to probe the model and make weight edits, while they steered the process and listened to outputs.
The approach added a rank-one term to each layer’s key-projection weights. The developer reports that this changed about 524,000 weights, or 3.4% of the transformer, with roughly 5,000 free parameters and no gradient steps. On a held-out test set with unseen phrases, seeds and two voices, clean whisper takes rose from 5 of 24 to 10 of 24; emotion separability across seven classes rose from 0.45 to 0.71.
The reported untagged-speech word error rate remained 0.006. Sound-tag word error rate rose from 0.007 to 0.014, while average sound length changed from 0.81 to 0.84 seconds. The developer says these are proxy measurements, including Whisper-based word error rate and a nearest-centroid emotion classifier.
The developer reports occasional glitches, mid-sentence changes in pitch or delivery, and some slurred whispering; one of 24 whisper takes became mumbling. Whispering was detected in 22 of 24 takes after the edit, but only 10 had clean words. The post says the edit is not a stable upgrade and that untagged outputs are not bit-identical to the original model.
Written from coverage as of