WEBVTT

1
00:00:00.240 --> 00:00:02.540
Imagine three records in a training mixture.

2
00:00:03.280 --> 00:00:05.060
One is a short mathematical derivation.

3
00:00:05.740 --> 00:00:07.780
Another explains a software API.

4
00:00:08.500 --> 00:00:13.340
A third is an agent trace that calls a tool, sees an error, and
repairs its attempt.

5
00:00:14.340 --> 00:00:17.720
They may all be useful, but the evidence of usefulness looks
different.

6
00:00:18.640 --> 00:00:20.300
MIRA starts from that selection problem.

7
00:00:21.060 --> 00:00:24.760
Mid-training mixes sources with different formats and capability
roles.

8
00:00:25.460 --> 00:00:28.100
The question is not only how to score millions of records
quickly.

9
00:00:28.600 --> 00:00:34.280
It is what should count as quality inside each source group,
before those scores decide what the model will see.

10
00:00:35.127 --> 00:00:38.547
The experiments organize twenty-one sources into five groups:

11
00:00:39.187 --> 00:00:43.467
two question-answering groups, a code-text group, and two agent
groups.

12
00:00:44.287 --> 00:00:48.687
The grouping follows the mean content embeddings of
representative records from each source,

13
00:00:49.167 --> 00:00:51.607
so related sources share a capability theme.

14
00:00:52.427 --> 00:00:57.707
This is grouping sources, not forcing every training record into
one universal quality scale.

15
00:00:58.547 --> 00:01:02.607
A group gives the next stage a coherent place to discover
evaluation criteria.

16
00:01:03.527 --> 00:01:07.607
The paper then uses a separate rubric and a separate distilled
scorer for each group.

17
00:01:08.233 --> 00:01:11.973
Within a group, a teacher first judges sampled records freely.

18
00:01:12.593 --> 00:01:19.133
It proposes quality dimensions, assigns scores, and explains its
reasons without receiving a pre-written rubric.

19
00:01:20.333 --> 00:01:25.713
Each dimension name and reason becomes a judgment point that can
be embedded and clustered.

20
00:01:26.753 --> 00:01:30.473
For each cluster, the point nearest its center becomes an anchor
dimension.

21
00:01:31.333 --> 00:01:34.413
Together these anchors form a fixed vocabulary for the group.

22
00:01:35.313 --> 00:01:43.013
In our illustration, labels such as logical consistency or
observation-sensitive action are examples of questions;

23
00:01:43.553 --> 00:01:47.513
they are not claimed to be the exact discovered rubric from a
particular source.

24
00:01:48.113 --> 00:01:54.053
Free-form dimensions are useful for discovery, but inconsistent
dimensions cannot directly train a predictable scorer.

25
00:01:54.973 --> 00:01:59.513
MIRA freezes the anchors and asks the teacher to score a larger
sample on every anchor,

26
00:01:59.933 --> 00:02:03.453
producing a numerical value and a short rationale for each
dimension.

27
00:02:04.273 --> 00:02:08.713
The anchored labels are split into student-training data and
held-out reliability data.

28
00:02:09.393 --> 00:02:12.893
A separate student for each group learns to emit those scores
and rationales.

29
00:02:13.573 --> 00:02:18.693
The student can then process the full corpus without asking the
expensive teacher to judge every record.

30
00:02:19.293 --> 00:02:23.953
One student may imitate the teacher well on one source and
poorly on another.

31
00:02:25.033 --> 00:02:30.353
MIRA checks that agreement on held-out records, separately for
each source and anchor dimension.

32
00:02:31.393 --> 00:02:37.713
Mean absolute error and rank correlation identify
source-dimension pairs whose scores are unreliable.

33
00:02:38.853 --> 00:02:40.593
The correction happens after scoring.

34
00:02:41.833 --> 00:02:47.733
An unreliable dimension is masked for that source; a trimmed
mean combines the surviving dimensions.

35
00:02:49.073 --> 00:02:55.773
The student prompt stays fixed, and full-corpus inference need
not be repeated just to update this reliability mask.

36
00:02:56.467 --> 00:02:59.247
Scores also have different distributions across sources.

37
00:02:59.887 --> 00:03:04.147
A single global cutoff can remove lower-scoring capability
regions before the others.

38
00:03:05.007 --> 00:03:07.867
MIRA therefore compares three retention granularities:

39
00:03:08.567 --> 00:03:14.347
one global threshold, thresholds inside groups, and thresholds
inside each individual source.

40
00:03:15.067 --> 00:03:18.067
The best aggregate result reported here is the group variant.

41
00:03:18.607 --> 00:03:22.907
It preserves broad capability coverage while allowing sources
within a group to compete.

42
00:03:23.867 --> 00:03:28.287
Source-conditioned reliability and source-specific final cutoffs
are related ideas,

43
00:03:28.647 --> 00:03:30.367
but they are not the same step of the pipeline.

44
00:03:30.973 --> 00:03:35.813
The experiment uses Qwen two point five Coder with fourteen
billion parameters.

45
00:03:36.713 --> 00:03:42.773
Every checkpoint receives the same subsequent supervised
fine-tuning on four hundred thousand instruction examples.

46
00:03:43.873 --> 00:03:46.493
Selection methods train on twenty-five billion tokens,

47
00:03:46.893 --> 00:03:50.333
compared with fifty billion for the unfiltered raw-mixture run.

48
00:03:51.533 --> 00:03:53.673
Evaluation covers nine code benchmarks.

49
00:03:54.533 --> 00:04:02.093
They feed four categories: code generation, multilingual code,
executable SQL, and software engineering.

50
00:04:02.953 --> 00:04:06.433
The final macro average weights those four categories equally;

51
00:04:06.853 --> 00:04:11.393
it is not a pooled success rate or a simple average of all nine
individual scores.

52
00:04:12.113 --> 00:04:17.213
On that aggregate, the fifty-billion-token raw mixture scores
sixty-three point eight three.

53
00:04:18.253 --> 00:04:23.793
Source-preserving random selection at twenty-five billion tokens
scores sixty-three point two three.

54
00:04:24.813 --> 00:04:30.113
MIRA Group at the same twenty-five-billion-token budget scores
sixty-four point two zero.

55
00:04:30.933 --> 00:04:36.993
Using half the training tokens, it reports an aggregate zero
point three seven points above the full-budget run.

56
00:04:37.813 --> 00:04:40.193
The result does not say every benchmark improves.

57
00:04:40.993 --> 00:04:46.573
Its useful evidence is that source-adaptive semantic selection
can preserve the overall capability balance

58
00:04:46.633 --> 00:04:49.053
in this matched code-oriented evaluation.

59
00:04:49.620 --> 00:04:51.640
Return to the three original records.

60
00:04:52.040 --> 00:04:56.140
A derivation, a document, and an agent trace need different
questions.

61
00:04:57.040 --> 00:05:01.300
The point is not to invent a beautiful universal rubric and hope
it works everywhere.

62
00:05:02.080 --> 00:05:07.200
First discover a stable quality vocabulary for records with a
shared capability role.

63
00:05:08.240 --> 00:05:11.860
MIRA separates that semantic work from the scalable machinery:

64
00:05:12.660 --> 00:05:17.700
anchor the teacher's judgments, distill a student, diagnose
source-level reliability,

65
00:05:18.280 --> 00:05:21.020
and choose data at an appropriate retention granularity.

66
00:05:21.880 --> 00:05:27.780
The training mixture is not just a pile of high scores; it is a
balance of capabilities the next model must learn.