The Surprising Effectiveness of Deleting Weights in LLM Post-Training
, updated First seen Updated Limited coverage, tapering
Models
Reported by r/LocalLLaMA1 source ↓
Key points
- Deletion Tuning matched full fine-tuning across reinforcement learning, supervised fine-tuning and on-policy distillation
- The method zeroes fewer than 0.05% of weights
- Under reinforcement learning, prior-task accuracy stayed near the base level; other tested methods lost up to 5.4 points
- The resulting mask was hundreds to thousands of times smaller than the corresponding full fine-tuning checkpoint
Details
The reported method learns a binary mask over the gate projection of transformer blocks, zeroing selected weights without other modifications. The community post says it matched full fine-tuning at scale in three post-training regimes: reinforcement learning with verifiable rewards, supervised fine-tuning and on-policy distillation.
The post reports that in reinforcement learning, Deletion Tuning kept prior-task accuracy near the base model’s level, while full fine-tuning, LoRA and random sparse fine-tuning lost up to 5.4 points. It also says the learned mask was hundreds to thousands of times smaller than the corresponding full fine-tuning checkpoint.
Written from coverage as of