1
00:00:00,240 --> 00:00:02,540
Imagine three records in a training mixture.

2
00:00:03,280 --> 00:00:05,060
One is a short mathematical derivation.

3
00:00:05,740 --> 00:00:07,780
Another explains a software API.

4
00:00:08,500 --> 00:00:13,340
A third is an agent trace that calls a tool, sees an error, and
repairs its attempt.

5
00:00:14,340 --> 00:00:17,720
They may all be useful, but the evidence of usefulness looks
different.

6
00:00:18,640 --> 00:00:20,300
MIRA starts from that selection problem.

7
00:00:21,060 --> 00:00:24,760
Mid-training mixes sources with different formats and capability
roles.

8
00:00:25,460 --> 00:00:28,100
The question is not only how to score millions of records
quickly.

9
00:00:28,600 --> 00:00:34,280
It is what should count as quality inside each source group,
before those scores decide what the model will see.

10
00:00:35,127 --> 00:00:38,547
The experiments organize twenty-one sources into five groups:

11
00:00:39,187 --> 00:00:43,467
two question-answering groups, a code-text group, and two agent
groups.

12
00:00:44,287 --> 00:00:48,687
The grouping follows the mean content embeddings of
representative records from each source,

13
00:00:49,167 --> 00:00:51,607
so related sources share a capability theme.

14
00:00:52,427 --> 00:00:57,707
This is grouping sources, not forcing every training record into
one universal quality scale.

15
00:00:58,547 --> 00:01:02,607
A group gives the next stage a coherent place to discover
evaluation criteria.

16
00:01:03,527 --> 00:01:07,607
The paper then uses a separate rubric and a separate distilled
scorer for each group.

17
00:01:08,233 --> 00:01:11,973
Within a group, a teacher first judges sampled records freely.

18
00:01:12,593 --> 00:01:19,133
It proposes quality dimensions, assigns scores, and explains its
reasons without receiving a pre-written rubric.

19
00:01:20,333 --> 00:01:25,713
Each dimension name and reason becomes a judgment point that can
be embedded and clustered.

20
00:01:26,753 --> 00:01:30,473
For each cluster, the point nearest its center becomes an anchor
dimension.

21
00:01:31,333 --> 00:01:34,413
Together these anchors form a fixed vocabulary for the group.

22
00:01:35,313 --> 00:01:43,013
In our illustration, labels such as logical consistency or
observation-sensitive action are examples of questions;

23
00:01:43,553 --> 00:01:47,513
they are not claimed to be the exact discovered rubric from a
particular source.

24
00:01:48,113 --> 00:01:54,053
Free-form dimensions are useful for discovery, but inconsistent
dimensions cannot directly train a predictable scorer.

25
00:01:54,973 --> 00:01:59,513
MIRA freezes the anchors and asks the teacher to score a larger
sample on every anchor,

26
00:01:59,933 --> 00:02:03,453
producing a numerical value and a short rationale for each
dimension.

27
00:02:04,273 --> 00:02:08,713
The anchored labels are split into student-training data and
held-out reliability data.

28
00:02:09,393 --> 00:02:12,893
A separate student for each group learns to emit those scores
and rationales.

29
00:02:13,573 --> 00:02:18,693
The student can then process the full corpus without asking the
expensive teacher to judge every record.

30
00:02:19,293 --> 00:02:23,953
One student may imitate the teacher well on one source and
poorly on another.

31
00:02:25,033 --> 00:02:30,353
MIRA checks that agreement on held-out records, separately for
each source and anchor dimension.

32
00:02:31,393 --> 00:02:37,713
Mean absolute error and rank correlation identify
source-dimension pairs whose scores are unreliable.

33
00:02:38,853 --> 00:02:40,593
The correction happens after scoring.

34
00:02:41,833 --> 00:02:47,733
An unreliable dimension is masked for that source; a trimmed
mean combines the surviving dimensions.

35
00:02:49,073 --> 00:02:55,773
The student prompt stays fixed, and full-corpus inference need
not be repeated just to update this reliability mask.

36
00:02:56,467 --> 00:02:59,247
Scores also have different distributions across sources.

37
00:02:59,887 --> 00:03:04,147
A single global cutoff can remove lower-scoring capability
regions before the others.

38
00:03:05,007 --> 00:03:07,867
MIRA therefore compares three retention granularities:

39
00:03:08,567 --> 00:03:14,347
one global threshold, thresholds inside groups, and thresholds
inside each individual source.

40
00:03:15,067 --> 00:03:18,067
The best aggregate result reported here is the group variant.

41
00:03:18,607 --> 00:03:22,907
It preserves broad capability coverage while allowing sources
within a group to compete.

42
00:03:23,867 --> 00:03:28,287
Source-conditioned reliability and source-specific final cutoffs
are related ideas,

43
00:03:28,647 --> 00:03:30,367
but they are not the same step of the pipeline.

44
00:03:30,973 --> 00:03:35,813
The experiment uses Qwen two point five Coder with fourteen
billion parameters.

45
00:03:36,713 --> 00:03:42,773
Every checkpoint receives the same subsequent supervised
fine-tuning on four hundred thousand instruction examples.

46
00:03:43,873 --> 00:03:46,493
Selection methods train on twenty-five billion tokens,

47
00:03:46,893 --> 00:03:50,333
compared with fifty billion for the unfiltered raw-mixture run.

48
00:03:51,533 --> 00:03:53,673
Evaluation covers nine code benchmarks.

49
00:03:54,533 --> 00:04:02,093
They feed four categories: code generation, multilingual code,
executable SQL, and software engineering.

50
00:04:02,953 --> 00:04:06,433
The final macro average weights those four categories equally;

51
00:04:06,853 --> 00:04:11,393
it is not a pooled success rate or a simple average of all nine
individual scores.

52
00:04:12,113 --> 00:04:17,213
On that aggregate, the fifty-billion-token raw mixture scores
sixty-three point eight three.

53
00:04:18,253 --> 00:04:23,793
Source-preserving random selection at twenty-five billion tokens
scores sixty-three point two three.

54
00:04:24,813 --> 00:04:30,113
MIRA Group at the same twenty-five-billion-token budget scores
sixty-four point two zero.

55
00:04:30,933 --> 00:04:36,993
Using half the training tokens, it reports an aggregate zero
point three seven points above the full-budget run.

56
00:04:37,813 --> 00:04:40,193
The result does not say every benchmark improves.

57
00:04:40,993 --> 00:04:46,573
Its useful evidence is that source-adaptive semantic selection
can preserve the overall capability balance

58
00:04:46,633 --> 00:04:49,053
in this matched code-oriented evaluation.

59
00:04:49,620 --> 00:04:51,640
Return to the three original records.

60
00:04:52,040 --> 00:04:56,140
A derivation, a document, and an agent trace need different
questions.

61
00:04:57,040 --> 00:05:01,300
The point is not to invent a beautiful universal rubric and hope
it works everywhere.

62
00:05:02,080 --> 00:05:07,200
First discover a stable quality vocabulary for records with a
shared capability role.

63
00:05:08,240 --> 00:05:11,860
MIRA separates that semantic work from the scalable machinery:

64
00:05:12,660 --> 00:05:17,700
anchor the teacher's judgments, distill a student, diagnose
source-level reliability,

65
00:05:18,280 --> 00:05:21,020
and choose data at an appropriate retention granularity.

66
00:05:21,880 --> 00:05:27,780
The training mixture is not just a pile of high scores; it is a
balance of capabilities the next model must learn.