Learning from the Self-future: On-policy Self-distillation for dLLMs

Paper: Learning from the Self-future: On-policy Self-distillation for dLLMs (v2) https://arxiv.org/html/2606.18195v2

Self-future trains a diffusion language model using later information from the student’s own sampled response. The future is a completed generated trajectory, not an external source of unseen test answers.

A student trajectory contains earlier noisy response states and a completed final response. For an earlier denoising step, the student still receives its current noisy state and prompt. The teacher view selectively reveals final-response tokens at positions that remain masked. This privileged context provides clues while leaving a distributional prediction task for the teacher.

The objective compares teacher and student distributions at selected confident currently masked positions along the trajectory. Summing that guidance over the sampled process matches the iterative structure of diffusion generation. The default selection uses teacher confidence, and correctness checks help choose useful completed responses within an eight-attempt budget.

The reported experiment uses LLaDA-8B-Instruct on GSM8K, MATH500, Countdown and four-by-four Sudoku. Student adaptation uses LoRA and reverse KL, with a quarter of eligible tokens retained for teacher context. The teacher policy stays at the initial policy. Evaluation checks occur every 25 training steps before step 501 and report the best evaluated checkpoint.

At GSM8K length 256, base, diffu-GRPO and d-OPSD scores are 76.0%, 79.8% and 81.0%. At MATH500 length 512, the latter methods report 39.2 and 37.8. The method does not win every displayed comparison.

The optimization account also needs its units. Table 2 lists 7,700 GSM8K baseline updates and 425 for d-OPSD, with baseline counts taken from the original baseline report. These are optimization steps rather than matched wall-clock timings. An original 3-times-4-plus-5 token diagram in the film makes the conditioning roles visible while keeping these measured results separate.

Pinned source versions: Paper, v2: https://arxiv.org/html/2606.18195v2

#DiffusionModels #LanguageModels #AIResearch