When an AI Answer Looks Right: Three Research Questions About Evidence
An answer can be fluent, well formatted, and supported by a plausible-looking citation while still missing the evidence the task requires. Three research questions make that gap concrete: did the output preserve the intended structure, does the retrieved paper support the claim, and did a video answer depend on the video at all?
1. A valid format can still carry the wrong content
StructEval studies structured generation and conversion across 18 text and renderable formats and 44 task types. Its core distinction is useful beyond the benchmark: parsing is not the same as preserving required fields, values, relationships, or rendered behavior.
Consider an order converted to YAML. It may parse successfully while omitting the quantity, or keep every expected key while changing a value from 2 to 200. Those failures require different checks. The public mini-lab makes that separation inspectable with handwritten fixtures; it is a teaching exercise, not a benchmark result.
2. Retrieving a paper is different from supporting a sentence
ScholarCopilot treats academic writing and citation retrieval as a joint problem. Its model produces a retrieval token, [RET], to query a scholarly database and uses retrieved references during generation. This addresses a real failure mode in writing assistants: a citation string can look credible without establishing that the source supports the adjacent claim.
A useful evidence check starts with the claim, then asks whether the source contains the relevant method, result, population, metric, or limitation. The project, code, and citation worksheet provide complementary entry points. None turns retrieval into a guarantee that every generated claim is supported.
3. A correct video answer may not have used the frames
Watch Before You Answer examines linguistic shortcuts in video question answering and post-training data. A question can be answerable from its wording or answer choices even when the visual evidence is not needed. Reported benchmark accuracy then risks overstating visual understanding.
The study filters toward questions that resist text-only answers before using them for post-training. Its results concern the stated models, data, and protocol; they do not show that every retained question is universally visual, or that every video benchmark shares the same shortcuts. The project page and probe-log example show how to keep text-only probes, coverage, and selection decisions visible.
A compact evidence map
| Question | Research entry point | What it does not establish |
|---|---|---|
| Did the structure survive? | StructEval: parsing, paths, and structural fidelity | That a syntactically valid answer preserves every application value |
| Does the reference support the claim? | ScholarCopilot: retrieval attached to academic writing | That a retrieved citation entails a particular sentence |
| Did the answer use visual evidence? | Watch Before You Answer: text-only probes for video data | That a correct answer necessarily used the frames |
These works address different tasks and use different datasets and metrics. They form a reading path for checking evidence, not a combined benchmark or a single score for AI reliability.
Original sources
答案看起来对，证据真的对吗？三个研究问题
一个答案可以流畅、格式正确，并带着看似可信的引用，却仍未使用任务真正要求的证据。下面三个研究问题分别检查：结构是否保留、论文是否支持相邻主张、视频答案是否真的依赖画面。
1. 格式有效，不代表内容正确
StructEval覆盖 18 种文本与可渲染格式、44 类任务。解析成功并不等于字段、数值、关系或渲染行为被保留。公开 mini-lab 用手写案例分开检查这些问题；它是教学练习，不是新的基准结果。
2. 找到论文，不等于论文支持这句话
ScholarCopilot把学术写作与引用检索联合建模：模型生成 [RET] 再查询文献库。它提示我们把“找到了来源”与“来源是否支持此处主张”分开。应回到原文核对方法、结果、适用对象、指标和限制。
3. 视频答案正确，不等于模型看了视频
Watch Before You Answer研究视频问答中的语言捷径。有些问题仅依赖文本或选项就能答对，准确率因此可能高估视觉理解。论文在规定模型、数据和协议下筛选抗文本探针的问题；这不等于所有保留问题都普遍需要视觉证据。可查看项目页与探针日志示例。
一张证据地图
| 问题 | 研究入口 | 它不证明什么 |
|---|---|---|
| 结构是否保留？ | StructEval | 格式可解析就保留了所有应用数值 |
| 引用是否支持主张？ | ScholarCopilot | 检索到的引用必然蕴含相邻句子 |
| 答案是否使用画面？ | Watch Before You Answer | 答对就一定看了视频帧 |
三项工作解决不同任务、使用不同数据与指标。它们构成核查证据的阅读路径，不构成可合并的单一基准或可靠性分数。