WEBVTT

1
00:00:00.340 --> 00:00:03.720
An encoder turns an image into continuous feature vectors.

2
00:00:04.560 --> 00:00:07.860
Vector quantization gives each vector a discrete address:

3
00:00:08.600 --> 00:00:10.660
find the nearest entry in a learned codebook,

4
00:00:11.160 --> 00:00:16.020
replace the feature with that entry, and let the decoder
reconstruct from the selected entries.

5
00:00:16.980 --> 00:00:19.200
Here is an original two-dimensional illustration.

6
00:00:19.980 --> 00:00:24.960
Small dots represent encoder outputs; large colored dots
represent code vectors.

7
00:00:25.640 --> 00:00:28.000
The connecting lines show nearest-neighbor choices.

8
00:00:28.760 --> 00:00:33.040
The counts on the right are computed from this small cloud, not
copied from a tokenizer benchmark.

9
00:00:33.733 --> 00:00:37.513
Keep the feature cloud, but place the codebook entries down one
side.

10
00:00:38.333 --> 00:00:45.393
Several entries attract no points at all. Other entries collect
many assignments, including points a long way away.

11
00:00:46.513 --> 00:00:50.793
Adding entries has not automatically supplied useful
representatives for this cloud.

12
00:00:51.933 --> 00:00:54.933
That picture captures the motivation behind these two papers.

13
00:00:55.773 --> 00:00:59.753
They study a mismatch between feature and code-vector
distributions.

14
00:01:00.693 --> 00:01:03.813
Codebook collapse means many entries remain unused;

15
00:01:04.473 --> 00:01:10.013
a bad geometric fit also creates substantial distortion when
features are replaced by distant codes.

16
00:01:11.007 --> 00:01:14.607
The earlier, twenty twenty-five paper already combines three
criteria.

17
00:01:15.287 --> 00:01:19.447
Quantization error asks how far each feature travels to its
selected code.

18
00:01:20.187 --> 00:01:23.707
Utilization asks whether each code was selected at least once.

19
00:01:24.747 --> 00:01:28.947
Codebook perplexity asks how evenly assignments are spread
across the entries.

20
00:01:29.867 --> 00:01:31.307
Coverage and balance are different.

21
00:01:31.967 --> 00:01:35.607
A code used once and a code used repeatedly both count as used.

22
00:01:36.007 --> 00:01:39.807
So a fully occupied codebook can still have heavily concentrated
assignments.

23
00:01:40.667 --> 00:01:44.787
The three questions should be read together, rather than
treating one percentage as the whole story.

24
00:01:45.433 --> 00:01:48.413
The earlier method adds a distribution-matching objective.

25
00:01:49.073 --> 00:01:53.013
Under a Gaussian approximation, its Wasserstein distance can be
computed

26
00:01:53.093 --> 00:01:56.793
from the two clouds' estimated means and covariance matrices.

27
00:01:57.613 --> 00:02:02.133
The mean describes location; covariance captures spread and
orientation.

28
00:02:03.113 --> 00:02:05.533
Matching only the center would leave the shape unchecked.

29
00:02:07.293 --> 00:02:10.093
This auxiliary objective complements nearest-code assignments.

30
00:02:11.073 --> 00:02:17.392
The later paper explicitly keeps its matching gradients on the
codebook, with encoder gradients detached for that term.

31
00:02:18.333 --> 00:02:21.273
It encourages the representatives to fit the feature
distribution;

32
00:02:21.673 --> 00:02:25.033
it does not force the encoder itself to generate a Gaussian
cloud.

33
00:02:25.787 --> 00:02:31.227
The twenty twenty-six record broadens this framework with
maximum mean discrepancy, or MMD.

34
00:02:31.947 --> 00:02:35.787
It compares samples through a kernel without choosing a Gaussian
distributional form.

35
00:02:36.507 --> 00:02:39.927
Both matching methods update codebook parameters through their
auxiliary loss,

36
00:02:40.267 --> 00:02:42.727
while keeping encoder gradients detached for that term.

37
00:02:43.427 --> 00:02:46.087
This matters when a cloud has multiple separated modes.

38
00:02:46.827 --> 00:02:51.447
In the later work's bimodal synthetic experiment, the methods
are comparable near Gaussianity;

39
00:02:52.047 --> 00:02:56.327
as mode separation increases, MMD obtains better error and
utilization.

40
00:02:56.987 --> 00:03:00.627
Pairwise kernel comparisons also cost more computation in their
timing study.

41
00:03:01.273 --> 00:03:04.533
There is an important experimental detail inside the method.

42
00:03:05.393 --> 00:03:09.013
Raw quantization error depends on the feature distribution's
variance.

43
00:03:09.693 --> 00:03:12.313
If two encoders produce differently scaled clouds,

44
00:03:12.733 --> 00:03:17.093
their raw errors do not isolate which quantizer represents its
own cloud better.

45
00:03:17.873 --> 00:03:22.953
The later paper therefore studies controlled synthetic features
when comparing the criterion triple.

46
00:03:23.913 --> 00:03:28.553
In our teaching picture, we also hold the small dots fixed while
changing codebook locations.

47
00:03:29.253 --> 00:03:31.213
That makes the geometric comparison readable.

48
00:03:32.073 --> 00:03:35.913
Image reconstruction is then evaluated separately in the actual
trained models.

49
00:03:36.627 --> 00:03:39.467
Now separate the drawing from a measured result.

50
00:03:40.147 --> 00:03:46.967
In the later paper's FFHQ VQ VAE table, both selected rows use
two hundred fifty-six tokens

51
00:03:47.027 --> 00:03:50.287
and a codebook of sixteen thousand three hundred eighty-four
entries.

52
00:03:51.167 --> 00:03:55.827
Vanilla vector quantization reports PSNR of twenty-seven point
eight three decibels.

53
00:03:56.587 --> 00:03:59.647
Wasserstein quantization reports twenty-nine point zero three.

54
00:04:00.547 --> 00:04:04.467
The same block reports codebook use of three point eight percent
for Vanilla

55
00:04:04.807 --> 00:04:06.627
and one hundred percent for Wasserstein.

56
00:04:07.547 --> 00:04:10.067
These are results for this dataset and configuration.

57
00:04:10.727 --> 00:04:15.147
PSNR measures reconstruction fidelity; it is not the raw
quantization error,

58
00:04:15.547 --> 00:04:18.527
and the illustrative cloud does not stand in for this
experiment.

59
00:04:19.213 --> 00:04:21.393
These are two distinct arXiv records.

60
00:04:21.853 --> 00:04:27.833
The earlier one develops the distribution-matching perspective,
the criterion triple, and a Wasserstein-based method.

61
00:04:28.513 --> 00:04:32.213
The later one presents a broader framework that also includes
MMD

62
00:04:32.513 --> 00:04:35.973
and explicitly compares their behavior across feature
distributions.

63
00:04:37.073 --> 00:04:40.733
Read them as a connected research story while preserving their
dates and evidence.

64
00:04:41.533 --> 00:04:45.433
The extension is not that coverage suddenly becomes measurable
in twenty twenty-six.

65
00:04:46.033 --> 00:04:48.953
It is that the matching objective is considered more broadly,

66
00:04:49.393 --> 00:04:52.533
with efficiency and distribution shape entering the choice of
method.

67
00:04:53.033 --> 00:04:54.053
Return to our dots.

68
00:04:54.733 --> 00:04:57.793
When code vectors sit near the regions occupied by features,

69
00:04:58.233 --> 00:05:02.433
the assignment lines become short and the usage counts become
less concentrated.

70
00:05:03.273 --> 00:05:08.972
Those computed counts explain the geometry; they are not a
promised result of running a training algorithm.

71
00:05:09.913 --> 00:05:13.973
The useful question is bigger than whether an image has been
assigned token IDs.

72
00:05:14.813 --> 00:05:19.693
Are the representatives close, are the entries covered, and is
their use balanced?

73
00:05:20.593 --> 00:05:25.733
Distributional matching gives these papers a common language for
improving all three aspects of the codebook.