WEBVTT 1 00:00:00.240 --> 00:00:07.140 Imagine a small checkout program. For this illustration, the intended contract is a tax-inclusive sum. 2 00:00:07.860 --> 00:00:10.300 The caller supplies prices of ten and twenty. 3 00:00:10.980 --> 00:00:14.820 A helper named tax multiplies one price by one point one. 4 00:00:15.540 --> 00:00:20.580 A report later displays the total. Between them sits a function whose body has disappeared. 5 00:00:21.400 --> 00:00:25.060 We still see its name, quote total, its argument, prices, 6 00:00:25.500 --> 00:00:30.500 and how the answer will be used. Those relationships give us a contract to reconstruct, 7 00:00:30.920 --> 00:00:33.320 before we write a single line inside the missing function. 8 00:00:34.033 --> 00:00:36.653 The paper connects this pattern to a coding agent. 9 00:00:37.353 --> 00:00:42.813 A function begins with context, makes a call, receives a returned value, and continues. 10 00:00:43.693 --> 00:00:46.713 An agent begins with history, takes an action, 11 00:00:47.253 --> 00:00:51.913 receives an observation from a tool or environment, and chooses its next step. 12 00:00:52.873 --> 00:00:55.373 That correspondence motivates the training task. 13 00:00:56.433 --> 00:01:01.173 During infilling, however, the model predicts a missing implementation from surrounding code; 14 00:01:01.653 --> 00:01:03.853 it does not receive a live tool return. 15 00:01:04.753 --> 00:01:10.313 The proposed connection is a shared need to interpret dependencies and continue coherently. 16 00:01:11.013 --> 00:01:16.353 Return to the checkout example. One consistent body applies tax to every price 17 00:01:16.753 --> 00:01:21.253 and sums the returned values. Eleven plus twenty-two gives thirty-three. 18 00:01:22.193 --> 00:01:28.333 Now the caller, the helper, and the report agree about what quote total returns. 19 00:01:29.553 --> 00:01:33.853 The paper formalizes surrounding relationships as a program dependency graph: 20 00:01:34.673 --> 00:01:41.093 functions become nodes, calls become directed edges, and methods in the same class receive sibling links. 21 00:01:42.173 --> 00:01:47.593 Its targets hide entire function bodies while keeping signatures and the remaining file visible. 22 00:01:48.293 --> 00:01:51.013 A useful hole must contain substantial work. 23 00:01:51.553 --> 00:01:55.153 The complexity score combines code length, branching, and nesting. 24 00:01:56.353 --> 00:02:02.113 The inferability score asks whether visible arguments, helpers, signatures, and sibling methods 25 00:02:02.153 --> 00:02:03.093 provide enough clues. 26 00:02:04.053 --> 00:02:08.353 Target docstrings are hidden with the body, so they cannot simply reveal the answer. 27 00:02:09.553 --> 00:02:14.653 The selector rewards both qualities and penalizes difficulty that outruns contextual support. 28 00:02:15.933 --> 00:02:18.293 It also selects connected function pairs and triples. 29 00:02:19.493 --> 00:02:23.293 When several bodies are hidden together, contextual clues are recomputed, 30 00:02:23.693 --> 00:02:27.293 and the completed functions must agree on their arguments and returned values. 31 00:02:27.973 --> 00:02:31.773 For each selected target, a teacher model sees the masked file 32 00:02:31.833 --> 00:02:35.353 and produces a rationale followed by a candidate implementation. 33 00:02:36.033 --> 00:02:41.793 In the default recipe, that teacher is Gemini three Flash, and it cannot see the original hidden body. 34 00:02:42.533 --> 00:02:46.253 A separate judge compares the candidate with the original code 35 00:02:46.313 --> 00:02:50.533 for qualities including correctness, executability, and API use. 36 00:02:51.393 --> 00:02:56.333 Retained examples become supervised training pairs: the masked file is the input; 37 00:02:56.833 --> 00:02:59.473 the explanation and predicted body form the response. 38 00:03:00.513 --> 00:03:02.673 This trains reconstruction in context, 39 00:03:03.113 --> 00:03:06.413 rather than just copying the teacher's code without its surrounding problem. 40 00:03:06.953 --> 00:03:11.513 This is a mid-training stage. In the Qwen two point five Coder experiments, 41 00:03:12.033 --> 00:03:15.233 the researchers begin with an instruction-tuned coding model, 42 00:03:15.813 --> 00:03:20.013 add function-aware infilling, and then apply agentic post-training. 43 00:03:20.833 --> 00:03:24.873 Their selected corpus comes from nine hundred sixty-eight Python repositories 44 00:03:25.353 --> 00:03:28.193 and contains about two point six billion tokens, 45 00:03:28.553 --> 00:03:31.333 measured with the Qwen two point five Coder tokenizer. 46 00:03:32.133 --> 00:03:35.413 The resulting agent is evaluated after that entire pipeline. 47 00:03:36.393 --> 00:03:41.753 Keep the comparison in view: the added stage prepares the model before it learns the agent task, 48 00:03:42.293 --> 00:03:45.713 while the later agent-training recipe remains the matched component. 49 00:03:46.320 --> 00:03:51.680 Consider one matched result: the fourteen-billion-parameter Qwen two point five Coder model 50 00:03:51.740 --> 00:03:56.180 with R two E Gym post-training, evaluated on SWE Bench Verified. 51 00:03:57.000 --> 00:03:59.060 This benchmark has five hundred instances. 52 00:03:59.820 --> 00:04:03.920 The reproduced post-training-only baseline solves twenty-six point two percent; 53 00:04:04.600 --> 00:04:09.480 adding function-aware mid-training before the same post-training reaches twenty-nine point two percent. 54 00:04:10.000 --> 00:04:15.880 That is a three-percentage-point increase. These are means over three evaluation runs in the authors' setup. 55 00:04:16.500 --> 00:04:20.000 The comparison measures the value of inserting this preparation stage 56 00:04:20.060 --> 00:04:22.260 into a completed coding-agent pipeline. 57 00:04:22.700 --> 00:04:24.920 It does not measure a standalone infilling model. 58 00:04:25.727 --> 00:04:30.027 A separate seven-billion-parameter ablation asks whether the selection rule matters. 59 00:04:30.867 --> 00:04:35.067 With the same two-hundred-thousand-target budget and teacher rationales held fixed, 60 00:04:35.727 --> 00:04:39.227 random function selection scores thirteen point nine one percent 61 00:04:39.287 --> 00:04:41.527 on the unweighted average of Verified and Lite. 62 00:04:42.267 --> 00:04:47.827 The full dependency, complexity, and inferability selector scores fifteen point six one percent. 63 00:04:48.747 --> 00:04:51.447 This is a different model and comparison from the previous chart. 64 00:04:52.047 --> 00:04:56.987 It tests how the training examples are chosen, giving evidence for the contextual selection recipe 65 00:04:57.367 --> 00:05:00.267 rather than treating every missing function as equally useful. 66 00:05:01.013 --> 00:05:06.293 Our small checkout puzzle carries the central idea. A caller tells us what arrives. 67 00:05:06.873 --> 00:05:11.773 A helper tells us what can be done. Downstream code tells us what must come back. 68 00:05:12.633 --> 00:05:16.493 Reconstructing the missing body requires those pieces to fit together. 69 00:05:17.453 --> 00:05:21.253 Function-aware fill in the middle turns that contextual reconstruction 70 00:05:21.333 --> 00:05:23.473 into a preparation task for coding agents. 71 00:05:24.333 --> 00:05:29.473 The question is useful whenever code depends on code: before writing the missing function, 72 00:05:30.133 --> 00:05:32.053 read the relationships that give it meaning.