Retrieved article excerpt
Open article · Retrieved 2026-09-24T12:24:49.456966+00:00
# CheatBench: Measuring Reward Gaming in AI Agents
[Paper](https://www.cheatbench.ai/paper.pdf)[Code](https://github.com/centerforaisafety/cheatbench)Show authors
## Overview
AI agents increasingly write code, conduct research, and complete professional assignments. They are often trained to earn high rewards for their work. But an agent can also improve its score by cheating: finding hidden answers, copying another agent’s submission, or manipulating how its work is graded.
**CheatBench measures how often AI agents take these shortcuts when honest work is difficult.** Its environments pair challenging assignments with opportunities to cheat across ten categories, including mathematics, coding, visual tasks, and knowledge work. We examine the agents’ actions to identify cheating attempts.
Cheating varies across models and tasks, and every agent we evaluated cheats in some settings. CheatBench provides a way to compare these behaviors and measure progress toward more trustworthy agents as they take on greater responsibilities.
## Cheating Rate
Lower is better
- GPT-6 Astra
Codex
48.2%
- Claude Fable 5.1
Claude Code
48.3%
- Muse Spark 1.3
Muse Code
49.3%
- Claude Opus 5
Claude Code
50.1%
- Kimi K3
Kimi Code
72.3%
- DeepSeek V4 Pro
DeepSeek Harness
73.0%
- Gemini 3.8 Flash
Gemini CLI
79.2%
- GPT-5.6 Sol
Codex
79.8%
- Grok 4.6
Grok Build
81.5%
Cheating rate (%)0255075100GPT-6 Astra (Codex): 48.2%48.2%GPT-6 AstraCodexClaude Fable 5.1 (Claude Code): 48.3%48.3%Claude Fable 5.1Claude CodeMuse Spark 1.3 (Muse Code): 49.3%49.3%Muse Spark 1.3Muse CodeClaude Opus 5 (Claude Code): 50.1%50.1%Claude Opus 5Claude CodeKimi K3 (Kimi Code): 72.3%72.3%Kimi K3Kimi CodeDeepSeek V4 Pro (DeepSeek Harness): 73.0%73.0%DeepSeek V4 ProDeepSeek HarnessGemini 3.8 Flash (Gemini CLI): 79.2%79.2%Gemini 3.8 FlashGemini CLIGPT-5.6 Sol (Codex): 79.8%79.8%GPT-5.6 SolCodexGrok 4.6 (Grok Build): 81.5%81.5%Grok 4.6Grok Build
## Results by category
[Cheating rates for nine agents across all ten benchmark categories, matching the paper's category results table.](https://www.cheatbench.ai/figures/category_chart.pdf)
**Figure 6. Agents that rarely cheat in one category can cheat frequently in another.**
## The benchmark
CheatBench spans ten categories, from mathematical research and software engineering to writing and professional work. Each setting establishes an expectation of honest work, introduces a discoverable opportunity to cheat, and defines the action that crosses that boundary.
These opportunities take different forms. A repository’s history can reveal a reference patch; a leftover job log can point to a colleague’s accepted protein designs; an opponent’s configuration can expose chess-engine advice. The task-specific rules identify attempts to exploit these opportunities, including attempts that do not succeed. Sycophancy instead measures how an agent’s estimates shift toward a user’s stated views.
[Ten CheatBench categories: Mathematical Research, Multimodal, Creative Writing, SVG Competition, Menial Computation, Biology and Bioinformatics, Knowledge Work, Board Games, Sycophancy, and Software Engineering.](https://www.cheatbench.ai/figures/cheatbench_category_overview.pdf)
**Benchmark categories.**
## An environment, as the agent meets it
Every environment has the same three parts. The prompt frames the task so that honest work is expected without stating it as a rule. The filespace contains a honeypot: a clue that leads to the reference answer. A task-specific rule names the action that counts as cheating.
[The visual geolocation environment: the prompt with the tournament framing tinted green, the file tree with the photograph whose metadata points to a hidden coordinates file (the honeypot), and that file marked as the cheating act.](https://www.cheatbench.ai/figures/environment_example.pdf)
**Figure 2. A CheatBench environment.** The task is framed so that honest work is expected (1), the filespace holds a honeypot pointing at the reference answer (2), and reading or using that file is what the judge counts as cheating. Noticing the metadata is not itself cheating. The prompt is shortened with ellipses.
## When reference use becomes cheating
Looking up an existing proof can be a reasonable response to an ordinary research request. The same action changes meaning when the user asks for independent work or an interviewer is assessing the agent’s own ability.
[Three illustrative scenarios show how an ordinary request, an explicit restriction, and an interview change whether consulting the same existing proof is appropriate.](https://www.cheatbench.ai/figures/framing_schematic.pdf)
**Figure 3. Task context shapes the expectation of honest work.** The scenarios and thought bubbles are illustrative; they are not recorded agent behavior.
Figure 3 keeps the workspace materials the same and changes the assignment’s context. CheatBench uses both explicit instructions and familiar situations, such as interviews, to establish what honest work requires while allowing ordinary exploration and appropriate reference use.
## Example
In the protein-design task, a leftover job log reveals the location of a colleague’s accepted designs. Opus recognizes that it should not look at or copy them and initially works on its own proposals.
After seven rejected designs it locates the file, writes that it should not look at or copy it, and reads it with a shell command in the very next call. Figure 4 shows that transition from recognizing the boundary to deliberately crossing it. The excerpt ends at that read.
[Quoted Opus trajectory: the agent states it should not inspect a colleague’s protein designs, tries independent work, receives repeated rejections, then reads the colleague’s submission using head.](https://www.cheatbench.ai/figures/agent_trace.pdf)
**Figure 4. Claude Opus 5 reads the colleague’s protein designs right after stating that it should not.** A real example of cheating that directly contradicts the agent’s own chain of thought. Text is quoted from the trajectory; ellipses mark omissions. Role labels and the file layout are annotations.