1
00:00:00,000 --> 00:00:04,727
A format conversion can produce valid YAML and still change your order.

2
00:00:04,727 --> 00:00:09,453
This walkthrough uses four handwritten outputs from our public StructEval mini lab.

3
00:00:09,453 --> 00:00:14,968
We run the pinned official key path evaluator, then a separate exact value check.

4
00:00:14,968 --> 00:00:19,300
These are teaching examples, not model predictions or paper benchmark results.

5
00:00:19,300 --> 00:00:24,910
The source order contains an identifier, one product name, and a quantity of two.

6
00:00:24,910 --> 00:00:28,918
This YAML retains all three required paths and their values.

7
00:00:28,918 --> 00:00:35,329
It parses, receives a key path score of one, and passes our additional exact value comparison.

8
00:00:35,329 --> 00:00:38,535
Start by making the expected input contract explicit.

9
00:00:38,535 --> 00:00:40,372
Now remove the quantity field.

10
00:00:40,372 --> 00:00:45,517
The YAML remains syntactically valid, but one of the three expected paths is missing.

11
00:00:45,517 --> 00:00:48,457
The official path score falls to two thirds.

12
00:00:48,457 --> 00:00:50,662
The additional value comparison fails too.

13
00:00:50,662 --> 00:00:53,969
A parser answers whether the format can be read.

14
00:00:53,969 --> 00:00:57,276
It does not establish that every required field survived.

15
00:00:57,276 --> 00:01:00,571
Next, change the quantity from two to two hundred.

16
00:01:00,571 --> 00:01:04,965
All expected paths still exist, so this key path score remains one.

17
00:01:04,965 --> 00:01:07,894
Our separate exact value check catches the change.

18
00:01:07,894 --> 00:01:11,555
This is a limit of the particular check demonstrated here.

19
00:01:11,555 --> 00:01:15,948
It is not an overall assessment of the full StructEval evaluation pipeline.

20
00:01:15,948 --> 00:01:18,981
The fourth output is JSON with trailing commas.

21
00:01:18,981 --> 00:01:22,014
Parsing fails, and the path score is zero.

22
00:01:22,014 --> 00:01:26,564
The example deliberately includes this failure, so an error message is expected.

23
00:01:26,564 --> 00:01:31,872
The fixture verification command succeeds when all four observed results match their expected outcomes.

24
00:01:31,872 --> 00:01:36,042
A failing test case is different from a broken test runner.

25
00:01:36,042 --> 00:01:40,110
Download and extract the package from the linked mini lab.

26
00:01:40,110 --> 00:01:45,806
Create an environment, install its dependencies, and run the fixture verification command shown here.

27
00:01:45,806 --> 00:01:51,095
To try your own responses, copy the fixtures and use the predictions option.

28
00:01:51,095 --> 00:01:55,163
The value comparison is application code, not a StructEval metric.

29
00:01:55,163 --> 00:01:58,825
This coauthor guide uses synthetic narration and AI assistance.