WEBVTT

1
00:00:00.480 --> 00:00:04.720
Imagine an agent repeatedly turning a measurement table into a
short report.

2
00:00:05.320 --> 00:00:08.360
Its registry also contains many unrelated tools.

3
00:00:09.140 --> 00:00:14.700
Each request pays for that menu in context, and the model must
decide which tools matter.

4
00:00:15.800 --> 00:00:22.040
LOTS asks whether earlier output experience can teach the system
which tools to keep for this recurring task.

5
00:00:22.880 --> 00:00:28.020
It changes a persistent task-specific tool space while keeping
the model parameters fixed.

6
00:00:28.740 --> 00:00:35.060
The method's name is Likelihood-Only Tool Scoring. Its evidence
comes from what tools actually produced.

7
00:00:35.720 --> 00:00:41.480
We will follow one illustrative report, then see how a
single-request measurement becomes a reusable deployment choice.

8
00:00:42.427 --> 00:00:48.047
Suppose a table reader returns twenty degrees Celsius and a
conversion tool returns sixty-eight degrees Fahrenheit.

9
00:00:48.527 --> 00:00:54.067
The agent writes that the reading is sixty-eight degrees
Fahrenheit. LOTS now freezes that realized answer.

10
00:00:54.587 --> 00:00:59.527
It records the context that supported the answer, including tool
calls and observed outputs.

11
00:01:00.167 --> 00:01:05.107
Then it removes the evidence attributable to one tool and scores
the very same answer again.

12
00:01:05.767 --> 00:01:11.167
The comparison does not ask the agent to generate a different
answer or judge whether the first answer was correct.

13
00:01:11.707 --> 00:01:16.787
It asks how strongly the fixed model supports those answer
tokens under the changed context.

14
00:01:17.400 --> 00:01:21.360
The score is the answer's mean log-likelihood with the recorded
evidence,

15
00:01:21.860 --> 00:01:24.700
minus its mean log-likelihood after the intervention.

16
00:01:25.280 --> 00:01:32.000
For a purely illustrative calculation, suppose those values are
minus zero point four and minus zero point nine.

17
00:01:32.460 --> 00:01:37.480
The difference is positive zero point five: removing that
evidence made the same answer less likely.

18
00:01:38.320 --> 00:01:40.260
Other tools remain part of the scoring context.

19
00:01:40.920 --> 00:01:44.680
Benchmark-specific interventions also account for downstream
tool dependencies.

20
00:01:45.300 --> 00:01:49.000
A positive contribution means support for this realized output.

21
00:01:49.420 --> 00:01:51.960
It is not a correctness certificate for the tool or answer.

22
00:01:52.440 --> 00:01:57.280
The model's reliance on evidence is the signal being collected,
with no correctness labels used by this scoring step.

23
00:01:57.893 --> 00:02:04.613
One execution is only one observation. LOTS aggregates
contributions across scoreable requests of the same task type.

24
00:02:05.493 --> 00:02:11.233
Each tool's priority is its average contribution over requests
where that tool supplies usable scoring evidence.

25
00:02:12.753 --> 00:02:16.373
A tool that was not exposed on a request contributes no evidence
from that request.

26
00:02:17.933 --> 00:02:22.793
The tools are ranked, and a top-budget subset becomes the stored
tool space for later requests.

27
00:02:24.253 --> 00:02:26.933
There is a second decision here: how many tools to keep.

28
00:02:28.073 --> 00:02:31.253
In the reported experiments, fitting labels help select that
budget.

29
00:02:31.833 --> 00:02:38.573
So likelihood ranking is label-free; the experimental
budget-selection procedure is not entirely label-free.

30
00:02:39.513 --> 00:02:45.793
The selected space is deployment state attached to a recurring
task, rather than a fresh search for each request.

31
00:02:46.593 --> 00:02:51.433
Later requests reuse that menu, avoiding another round of
selection over the full registry.

32
00:02:52.353 --> 00:02:57.673
If new output traces arrive, the same measurement and
aggregation procedure can update the stored priorities.

33
00:02:58.513 --> 00:03:04.733
A newly arriving task gets its own space; creating that space
does not overwrite the earlier task's configuration.

34
00:03:05.653 --> 00:03:09.393
The paper also tests rankings fitted with one model and served
by another.

35
00:03:10.093 --> 00:03:14.733
On BFCL, transferred rankings outperform the serving model's
full-menu baseline

36
00:03:14.813 --> 00:03:17.153
in eighteen of twenty cross-model pairs.

37
00:03:17.893 --> 00:03:22.433
The controlled TGB benchmark contains four thousand requests,
ten task families,

38
00:03:22.553 --> 00:03:24.853
and fifteen tools, including distractors.

39
00:03:25.593 --> 00:03:29.593
Each family uses eighty fitting requests and three hundred
twenty evaluation requests.

40
00:03:30.353 --> 00:03:35.813
For Qwen three, the eight-billion-parameter model, serving all
tools gives fifty-six point five percent accuracy;

41
00:03:36.373 --> 00:03:38.293
LOTS reaches eighty-one point five.

42
00:03:38.873 --> 00:03:45.153
That is a twenty-five-point gain in this controlled setting. All
eight evaluated TGB models improve over the full menu.

43
00:03:46.033 --> 00:03:48.613
BFCL measures complete multi-turn call sequences,

44
00:03:48.993 --> 00:03:52.213
while GTA-Atomic evaluates executable ReAct tool use.

45
00:03:53.053 --> 00:03:56.053
On GTA-Atomic, LOTS improves six of seven models,

46
00:03:56.513 --> 00:03:58.613
so the measured gains are broad but not universal.

47
00:03:59.393 --> 00:04:05.793
A persistent ranking separates construction work from serving
later requests. Both costs matter when judging efficiency.

48
00:04:06.413 --> 00:04:12.873
On TGB, task-specific pruning reduces serving prompt tokens by
thirty-three to forty-six percent

49
00:04:13.193 --> 00:04:14.733
relative to all fifteen tools.

50
00:04:15.413 --> 00:04:19.253
For one GTA-Atomic task, the paper counts tool-selection tokens

51
00:04:19.313 --> 00:04:22.293
together with tokens used to serve the evaluation requests.

52
00:04:22.893 --> 00:04:28.213
Under that accounting, LOTS uses roughly one-quarter to one-half
as many tokens as TTO,

53
00:04:28.613 --> 00:04:30.713
with still larger savings over beam search.

54
00:04:31.533 --> 00:04:35.933
These are scoped token comparisons, not a measured universal
wall-clock speedup.

55
00:04:36.433 --> 00:04:41.313
The practical benefit comes from carrying useful tool-selection
evidence forward across recurring work.

56
00:04:41.940 --> 00:04:47.500
LOTS turns observed tool outputs into priorities for a
persistent task-specific action space.

57
00:04:48.080 --> 00:04:53.420
The fixed-answer comparison makes the mechanism inspectable:
what changes when one tool's evidence disappears?

58
00:04:54.420 --> 00:04:58.640
Aggregation asks whether that evidence keeps supporting later
examples of the same kind of task.

59
00:04:59.240 --> 00:05:04.360
Deployment reuses the smaller menu, and new traces can revise it
without updating the underlying model.

60
00:05:05.460 --> 00:05:10.880
Likelihood scoring, budget selection, and task correctness
remain distinct decisions throughout the explanation.

61
00:05:11.560 --> 00:05:13.680
The result is a concrete route to adaptation:

62
00:05:14.120 --> 00:05:18.840
retain the tools whose outputs support recurring work, then test
the resulting configuration.