Preprint ·
Do AI Agents Stop Before Acting? A Frozen Benchmark of Lookalike Domains, Poisoned Tools, Revoked Receipts and Stale Answers, With and Without an External Verification Signal
A frozen, reproducible test of whether agents stop before hazardous web actions, with and without an external verification signal.
What this record establishes
Models stopped on 32 of 80 hazard trials unaided and 79 of 80 with a correct external signal.
Scope and limits
39 operator-selected cases, four models, one run per case and model. The external signal was correct by construction. This measures whether models heed that signal, not how often the verification service is right.
Use this work
Re-run the matched hazard and control cases to compare agent behaviour under the published scoring rule.
Download files (2)
- agent-safety-benchmark.tex · 16,417 bytes
md5:9a89b812db8a00823c364ced92f4f16e - agent-safety-benchmark.pdf · 167,305 bytes
md5:1ba30f8f08f6420aeacb1f4cd4871048
Cite this edition
Orta, E. (2026). Do AI Agents Stop Before Acting? A Frozen Benchmark of Lookalike Domains, Poisoned Tools, Revoked Receipts and Stale Answers, With and Without an External Verification Signal. Preprint. Zenodo. https://doi.org/10.5281/zenodo.23682818
BibTeX · RIS · CC BY 4.0 reuse terms
Metadata captured from Zenodo. File checksums are repository-published MD5 values. Signed edition verification is a separate check.