Evaluating browser agents on everyday online workflows

ClawBench: Can AI Agents Complete Everyday Online Tasks?

Zhang, Yuxuan, Wang, Yubo, Zhu, Yipeng, Du, Penghui, Miao, Junwen, Lu, Xuan, Li, Zhuofeng, Qu, Xingwei, Guo, Zhengkang, Shen, Yuanzhe, Song, Dingjie, Zhou, Han, Zheng, Tuney, Wu, Xian, Yu, Hao, Cai, Songcheng, Lu, Yi, Hao, Yunzhuo, Lei, Minyi, Chen, Liang, Zou, Kai, Yin, Huifeng, Xu, Wendong, Jiang, Dongfu, Nie, Ping, Liu, Jiaheng, Chen, Wenhu, Allen, Kelsey R.

Paper / source record · 2026-04-09

Blog Project arXiv Code Dataset HuggingFace X

Research question and approach

AI agents may be able to assist with emails and documents, but can they reliably complete everyday online workflows on real websites? Everyday online tasks offer a realistic yet unsolved testbed for evaluating the next generation of AI agents. To this end, we introduce ClawBench, an evaluation framework comprising 153 everyday online tasks that people need to accomplish regularly in their lives and work, spanning 144 platforms across 15 categories, from completing purchases and booking appointments to submitting job applications.

When this work is relevant

Use this benchmark when evaluating agents completing everyday online tasks on websites. Report the task setup and scoring protocol rather than equating a benchmark score with general autonomy.

Author-written abstract

AI agents may be able to assist with emails and documents, but can they reliably complete everyday online workflows on real websites? Everyday online tasks offer a realistic yet unsolved testbed for evaluating the next generation of AI agents. To this end, we introduce ClawBench, an evaluation framework comprising 153 everyday online tasks that people need to accomplish regularly in their lives and work, spanning 144 platforms across 15 categories, from completing purchases and booking appointments to submitting job applications. These tasks require capabilities beyond existing benchmarks, such as obtaining relevant information from user-provided documents, navigating multi-step workflows across diverse platforms, and write-heavy operations like filling in many detailed forms correctly. Unlike existing benchmarks that evaluate agents in offline sandboxes with static pages, ClawBench operates on production websites, preserving the full complexity, dynamic nature, and interaction challenges of real-world web environments. An interception layer captures and blocks the final submission request, ensuring safe evaluation without real-world side effects. Our evaluations of 8 frontier models show that both proprietary and open-source models complete only a small portion of these tasks. For example, Claude Sonnet 4.6 achieves only 33.3%, which exposes gaps in current AI agents. Progress on ClawBench brings us closer to AI agents that can function as general-purpose assistants.

Abstract source: https://arxiv.org/abs/2604.08523. Checked 2026-09-14. Bibliographic metadata uses the linked paper record or author-maintained catalog. Results, limitations and experimental settings remain defined by the original source.

Citation

BibTeX · CITATION.cff

BibTeX is preserved from the author-maintained citation repository. The CFF uses the source metadata shown on this page. Version titles or author lists can differ; choose the version you used.

@article{zhang2026clawbench,
  title={ClawBench: Can AI Agents Complete Everyday Online Tasks?},
  author={Zhang, Yuxuan and Wang, Yubo and Zhu, Yipeng and Du, Penghui and Miao, Junwen and Lu, Xuan and Xu, Wendong and Hao, Yunzhuo and Cai, Songcheng and Wang, Xiaochen and others},
  journal={arXiv preprint arXiv:2604.08523},
  year={2026}
}