1
00:00:00,340 --> 00:00:03,720
An encoder turns an image into continuous feature vectors.

2
00:00:04,560 --> 00:00:07,860
Vector quantization gives each vector a discrete address:

3
00:00:08,600 --> 00:00:10,660
find the nearest entry in a learned codebook,

4
00:00:11,160 --> 00:00:16,020
replace the feature with that entry, and let the decoder
reconstruct from the selected entries.

5
00:00:16,980 --> 00:00:19,200
Here is an original two-dimensional illustration.

6
00:00:19,980 --> 00:00:24,960
Small dots represent encoder outputs; large colored dots
represent code vectors.

7
00:00:25,640 --> 00:00:28,000
The connecting lines show nearest-neighbor choices.

8
00:00:28,760 --> 00:00:33,040
The counts on the right are computed from this small cloud, not
copied from a tokenizer benchmark.

9
00:00:33,733 --> 00:00:37,513
Keep the feature cloud, but place the codebook entries down one
side.

10
00:00:38,333 --> 00:00:45,393
Several entries attract no points at all. Other entries collect
many assignments, including points a long way away.

11
00:00:46,513 --> 00:00:50,793
Adding entries has not automatically supplied useful
representatives for this cloud.

12
00:00:51,933 --> 00:00:54,933
That picture captures the motivation behind these two papers.

13
00:00:55,773 --> 00:00:59,753
They study a mismatch between feature and code-vector
distributions.

14
00:01:00,693 --> 00:01:03,813
Codebook collapse means many entries remain unused;

15
00:01:04,473 --> 00:01:10,013
a bad geometric fit also creates substantial distortion when
features are replaced by distant codes.

16
00:01:11,007 --> 00:01:14,607
The earlier, twenty twenty-five paper already combines three
criteria.

17
00:01:15,287 --> 00:01:19,447
Quantization error asks how far each feature travels to its
selected code.

18
00:01:20,187 --> 00:01:23,707
Utilization asks whether each code was selected at least once.

19
00:01:24,747 --> 00:01:28,947
Codebook perplexity asks how evenly assignments are spread
across the entries.

20
00:01:29,867 --> 00:01:31,307
Coverage and balance are different.

21
00:01:31,967 --> 00:01:35,607
A code used once and a code used repeatedly both count as used.

22
00:01:36,007 --> 00:01:39,807
So a fully occupied codebook can still have heavily concentrated
assignments.

23
00:01:40,667 --> 00:01:44,787
The three questions should be read together, rather than
treating one percentage as the whole story.

24
00:01:45,433 --> 00:01:48,413
The earlier method adds a distribution-matching objective.

25
00:01:49,073 --> 00:01:53,013
Under a Gaussian approximation, its Wasserstein distance can be
computed

26
00:01:53,093 --> 00:01:56,793
from the two clouds' estimated means and covariance matrices.

27
00:01:57,613 --> 00:02:02,133
The mean describes location; covariance captures spread and
orientation.

28
00:02:03,113 --> 00:02:05,533
Matching only the center would leave the shape unchecked.

29
00:02:07,293 --> 00:02:10,093
This auxiliary objective complements nearest-code assignments.

30
00:02:11,073 --> 00:02:17,392
The later paper explicitly keeps its matching gradients on the
codebook, with encoder gradients detached for that term.

31
00:02:18,333 --> 00:02:21,273
It encourages the representatives to fit the feature
distribution;

32
00:02:21,673 --> 00:02:25,033
it does not force the encoder itself to generate a Gaussian
cloud.

33
00:02:25,787 --> 00:02:31,227
The twenty twenty-six record broadens this framework with
maximum mean discrepancy, or MMD.

34
00:02:31,947 --> 00:02:35,787
It compares samples through a kernel without choosing a Gaussian
distributional form.

35
00:02:36,507 --> 00:02:39,927
Both matching methods update codebook parameters through their
auxiliary loss,

36
00:02:40,267 --> 00:02:42,727
while keeping encoder gradients detached for that term.

37
00:02:43,427 --> 00:02:46,087
This matters when a cloud has multiple separated modes.

38
00:02:46,827 --> 00:02:51,447
In the later work's bimodal synthetic experiment, the methods
are comparable near Gaussianity;

39
00:02:52,047 --> 00:02:56,327
as mode separation increases, MMD obtains better error and
utilization.

40
00:02:56,987 --> 00:03:00,627
Pairwise kernel comparisons also cost more computation in their
timing study.

41
00:03:01,273 --> 00:03:04,533
There is an important experimental detail inside the method.

42
00:03:05,393 --> 00:03:09,013
Raw quantization error depends on the feature distribution's
variance.

43
00:03:09,693 --> 00:03:12,313
If two encoders produce differently scaled clouds,

44
00:03:12,733 --> 00:03:17,093
their raw errors do not isolate which quantizer represents its
own cloud better.

45
00:03:17,873 --> 00:03:22,953
The later paper therefore studies controlled synthetic features
when comparing the criterion triple.

46
00:03:23,913 --> 00:03:28,553
In our teaching picture, we also hold the small dots fixed while
changing codebook locations.

47
00:03:29,253 --> 00:03:31,213
That makes the geometric comparison readable.

48
00:03:32,073 --> 00:03:35,913
Image reconstruction is then evaluated separately in the actual
trained models.

49
00:03:36,627 --> 00:03:39,467
Now separate the drawing from a measured result.

50
00:03:40,147 --> 00:03:46,967
In the later paper's FFHQ VQ VAE table, both selected rows use
two hundred fifty-six tokens

51
00:03:47,027 --> 00:03:50,287
and a codebook of sixteen thousand three hundred eighty-four
entries.

52
00:03:51,167 --> 00:03:55,827
Vanilla vector quantization reports PSNR of twenty-seven point
eight three decibels.

53
00:03:56,587 --> 00:03:59,647
Wasserstein quantization reports twenty-nine point zero three.

54
00:04:00,547 --> 00:04:04,467
The same block reports codebook use of three point eight percent
for Vanilla

55
00:04:04,807 --> 00:04:06,627
and one hundred percent for Wasserstein.

56
00:04:07,547 --> 00:04:10,067
These are results for this dataset and configuration.

57
00:04:10,727 --> 00:04:15,147
PSNR measures reconstruction fidelity; it is not the raw
quantization error,

58
00:04:15,547 --> 00:04:18,527
and the illustrative cloud does not stand in for this
experiment.

59
00:04:19,213 --> 00:04:21,393
These are two distinct arXiv records.

60
00:04:21,853 --> 00:04:27,833
The earlier one develops the distribution-matching perspective,
the criterion triple, and a Wasserstein-based method.

61
00:04:28,513 --> 00:04:32,213
The later one presents a broader framework that also includes
MMD

62
00:04:32,513 --> 00:04:35,973
and explicitly compares their behavior across feature
distributions.

63
00:04:37,073 --> 00:04:40,733
Read them as a connected research story while preserving their
dates and evidence.

64
00:04:41,533 --> 00:04:45,433
The extension is not that coverage suddenly becomes measurable
in twenty twenty-six.

65
00:04:46,033 --> 00:04:48,953
It is that the matching objective is considered more broadly,

66
00:04:49,393 --> 00:04:52,533
with efficiency and distribution shape entering the choice of
method.

67
00:04:53,033 --> 00:04:54,053
Return to our dots.

68
00:04:54,733 --> 00:04:57,793
When code vectors sit near the regions occupied by features,

69
00:04:58,233 --> 00:05:02,433
the assignment lines become short and the usage counts become
less concentrated.

70
00:05:03,273 --> 00:05:08,972
Those computed counts explain the geometry; they are not a
promised result of running a training algorithm.

71
00:05:09,913 --> 00:05:13,973
The useful question is bigger than whether an image has been
assigned token IDs.

72
00:05:14,813 --> 00:05:19,693
Are the representatives close, are the entries covered, and is
their use balanced?

73
00:05:20,593 --> 00:05:25,733
Distributional matching gives these papers a common language for
improving all three aspects of the codebook.