MIRA: Mid-training Rubric Anchoring for Source-Aware Data Selection

Paper: MIRA: Mid-training Rubric Anchoring for Source-Aware Data Selection (v2) https://arxiv.org/html/2605.30288v2

This explainer follows the mechanism before interpreting its reported results.

A short mathematical derivation, an API explanation and a tool-use trace can all be useful training data. Yet the same checklist may judge them poorly. A complete derivation needs valid reasoning; a useful trace may need a sensible response to an error. MIRA begins with this difference rather than assuming one universal quality score.

The method organizes 21 sources into five capability groups using representative content embeddings. A teacher initially proposes dimensions, scores and explanations. Those judgement points are embedded and clustered; representative points become rubric anchors. Anchoring stabilizes the vocabulary used in a second round of teacher scoring, and group-specific student scorers make those judgements practical at scale.

Reliability is another part of the mechanism. Validation checks source–dimension combinations using absolute error and rank correlation. Unreliable cells can be masked after inference, and surviving dimensions are combined with a trimmed mean. Changing that mask therefore does not require pretending that every source supports every criterion.

Selection still has to allocate a training budget. Our original twelve-record illustration keeps six equal-length records under different masks. It makes group coverage and within-group source competition visible; these toy scores are not the paper’s measured data. The paper compares global, group and source thresholds, with group-level selection serving as its default compromise.

The reported model setup uses Qwen2.5-Coder-14B followed by the same 400,000-example instruction fine-tuning stage. MIRA-Group scores 64.20 with 25 billion selected tokens, versus 63.23 for source-preserving random selection at that budget and 63.83 for the raw 50-billion-token mixture. This is a macro over four equally weighted code-evaluation categories, not pooled accuracy or a claim that every benchmark wins.

The useful idea is a chain of decisions: discover criteria, anchor them, scale scoring, check reliability, and make the resulting mixture explicit.