Anthropic claims Claude led 26% of its AI R&D work under human supervision in August 2026, up from under 1% in February, indicating a substantial shift toward agent-executed model development without fully autonomous research.

Open article · Retrieved 2026-09-17T21:22:06.829281+00:00

Measurements for  
 understanding the pace of AI development inside frontier labs

AI systems are becoming exponentially more powerful and have begun to [automate more](https://www.anthropic.com/institute/recursive-self-improvement) of the process of building themselves. As the world considers [slowing the pace of frontier AI development](https://darioamodei.com/post/we-must-pace-the-frontier), the public needs more information.

In this post, we lay out measurement tools that can illuminate three critical aspects of AI development:

1. [The extent to which AI is building the next version of itself, as opposed to being built by humans](https://www.anthropic.com/institute/measuring-pace-of-ai-development#1-measuring-ai-led-ai-rd)
2. [Our ability to oversee and intervene in actions that AI agents take on Anthropic’s systems](https://www.anthropic.com/institute/measuring-pace-of-ai-development#2-measuring-oversight-of-ai-agents)
3. [The resources that power the development of more capable models](https://www.anthropic.com/institute/measuring-pace-of-ai-development#3-measuring-compute-allocation)

We also provide a snapshot of these metrics from inside Anthropic. It’s important to note that we would expect these numbers to shift if there were coordination on pacing the frontier, as called for by Anthropic CEO Dario Amodei. We plan to embed independent third-party evaluators from multiple organizations at Anthropic, and give them access to internal processes, systems, and data comparable to what internal risk assessment teams have. These third parties will verify safety practices, report incidents, and monitor key metrics such as the ones in this piece.

We are reporting these measurements because they give the public, third parties, and governments better visibility into the pace of AI development inside frontier labs. For each measurement, we describe what we measured, what the measurement showed, and what it would take to publish these measurements regularly in a form others can verify. We share methodological details in the Appendix.

## Reasons to track these measurements

The measurements in this piece are focused on *how models are built.* By better understanding the production process of models, we have a better chance of correlating model inputs, like compute, with model outputs, like capabilities. They complement capability evaluations, which measure *what models can do*. We publish those separately through our [Responsible Scaling Policy](https://www.anthropic.com/responsible-scaling-policy) (RSP) risk reports, which include evidence on how much our models are accelerating AI R&D. In our policy proposal on advanced AI, the [Advanced AI Framework (AAIF)](https://www-cdn.anthropic.com/files/4zrzovbb/website/0a58d567024a8b448ff15158ebc3625328dfcc1f.pdf), we propose rules of the road for how any lab releases safe models, including transparency obligations that governments could require, such as risk reports. Together, these proposed measurements and policies are a starting point for monitoring the pace of AI development from outside the labs.

## (1) Measuring AI-led AI R&D

**Why measure AI-led R&D?** Frontier AI labs increasingly use AI to build future AI models. This process allows labs in democratic countries to develop more capable models more quickly and conduct more safety and testing on models before they are released to secure AI’s benefits while staying on the frontier. However, models accelerating their own development could make it more challenging for humans to understand or control these systems. It is therefore important to share these metrics to understand how close the world is to reaching [recursive self improvement](https://www.anthropic.com/institute/recursive-self-improvement) (a model fully autonomously building its successor).

**What we measured.** We built a prototype index of how much of Anthropic’s AI research and development (R&D) is performed by Claude, called the Anthropic R&D Automation Index. It’s built by cataloguing every kind of AI R&D work done at the company, rating how automated each task currently is, and aggregating those ratings.

**What we found.** To measure the extent to which AI is doing AI R&D at Anthropic, we use an [automation rating scale](https://epochai.substack.com/p/toward-an-onet-for-ai-r-and-d) developed by Epoch AI that measures “Automation Level,” or AL. It runs from AL0 (no AI involvement) to AL5 (AI operates fully autonomously, with no human in the loop). In AL3, AI “collaborates”: it can do large chunks of work under close human direction. In AL4, AI “leads”: it can complete most of the task end-to-end from a high-level prompt, while the human supervises[1](https://www.anthropic.com/institute/measuring-pace-of-ai-development#footnote-1).

As of August 2026,

- Claude is not operating fully autonomously for any measured subset of AI R&D work.
- Claude “leads” 26% of Anthropic’s AI R&D work.
- The share of work at or above “AI collaborates” is above 90%.

Chart showing Claude now leads 26% of Anthropic's model R&D tasks, up from under 1% in February 2026.

**What any AI developer could report today.** Any frontier developer could publish these measures regularly, using a public methodology. This would enable the numbers to be compared over time, and potentially across labs.

Two obstacles stand in the way of cross-lab comparison on this type of reporting. First is the lack of a common methodology. Second, we’re using our own models to evaluate our systems, which could mean that the “judge” model could make the same kinds of errors as the model it is checking. To address these challenges, a developer’s measurements could be verified by a third party, or by other developers’ models, with guardrails in place to limit the exchange of competitively sensitive data. These measures could also become the trigger for stronger requirements, like a fixed testing window before a new model is used for further AI R&D.

## (2) Measuring oversight of AI agents

**Why measure oversight of agents?** Like other frontier developers, Anthropic employees increasingly delegate tasks to agents that work semi-autonomously for long stretches, and that delegate work to one another. As work becomes increasingly automated, from “AI collaborates” toward “AI leads,” agents could make more consequential decisions, such as which research direction to pursue next.

These decisions need safeguards to block harmful behavior from an individual agent, as well as an oversight regime to detect problems that might only arise across many agents. In our monitoring data to date, individual agents rarely misbehave. But when there are millions or billions of agents operating in the economy, even rare events can happen regularly.

**What we measured.** We’ve built a system that lets us oversee and intervene in actions AI agents take on Anthropic’s systems. Here, we consider three different metrics: *coverage,* which describes the share of an agent’s actions that pass through a monitor before or after execution; *review* *latency*, which is the time between an action and its review, first by an automated monitor and then by a human; and *escalation rate,* whichis the share of agent activities that are either blocked/redirected (in the case of online monitors) or flagged for further review (in the case of offline monitors).

**What we found.** As of August 2026, there were approximately 30,000 agents doing research and engineering work at Anthropic at any one time in our most-used internal platform. These measurements cover this platform only. The actions of these agents are constrained by two kinds of monitors, summarized below:

|  |  |  |
| --- | --- | --- |
|  | **Online monitors** | **Offline monitors** |
| **Description** | Online monitors check the actions of agents *in real-time* and either block dangerous actions or steer agent misbehavior automatically. They’re built to address harms that would happen quickly and could not be undone, such as an agent copying model weights out of our systems. | Offline monitors read agent activities *after the fact*. They are built to help address dangerous behaviors that develop slowly and can be reversed, like signs of model misalignment. We use flagged activities to improve our mitigations, like classifiers and deterministic controls. If the situation caused a large imminent danger, we would shut down the dangerous parts of the agent’s deployment. |
| **Coverage** | 100% of these agents’ actions pass through the online monitor before they are executed, usually within seconds. | 100% of these agents’ actions are ingested by the monitor after they are taken. |
| **Review latency** | Automated review happens before an agent takes action; humans review any blocked actions within one week. | The monitor flags roughly 100,000 transcripts per week that are then reviewed by multiple classifiers for false-positives and initial oversight review. Most of the flags are resolved by this process, and the highest priority flags (approximately ~50 per week) are escalated to human review. |
| **Blocking / escalation rate** | We analyzed over a billion decisions from our research & engineering agents over August 2026 and found that 0.002% of them (about 1 in 47,000) were blocked by this monitor. | The monitor flags roughly one to two transcripts in every thousand for further review. |

**What an AI developer could report today.** Any developer running agents on its own research and engineering workloads could publish the same measures: *coverage* (the share of agent activity that is monitored), *review latency* (how soon flagged activities are reviewed), and *escalation rate* (what share of agent activities the monitors block or flag). Together, these measures would allow society to see whether oversight is keeping pace with the growing role of AI in AI R&D. We published all of these measurements in our recent [risk report](https://www.anthropic.com/aug-2026-risk-report).

Developers should share how often known agent misbehavior is caught by monitors, and should have third parties do this kind of testing independently. In the past, METR has [independently red-teamed our offline monitoring platform](https://www-cdn.anthropic.com/f61d49fa5596956a5dec75fea0e973bf6a6a8378/Redacted%20Risk%20Report%20August%202026%20.pdf). As described above, we are now [setting up](https://darioamodei.com/post/we-must-pace-the-frontier) external third party evaluators at Anthropic.

## (3) Measuring compute allocation

**Why measure compute allocation?** Broadly speaking, AI developers use compute for building more powerful models, serving customers, and safety-focused work like [auditing a model’s “thoughts”](https://www.anthropic.com/research/natural-language-autoencoders), [training model organisms to study misalignment](https://www.anthropic.com/research/emergent-misalignment-reward-hacking), and [evaluating whether a model can be safely deployed](https://www-cdn.anthropic.com/f61d49fa5596956a5dec75fea0e973bf6a6a8378/Redacted%20Risk%20Report%20August%202026%20.pdf). Understanding how AI developers allocate their compute can tell you where a developer is focusing its resources and how that focus changes over time.

Additionally, compute is among the most verifiable inputs to the AI R&D process, meaning that it could be a critical lever in a future pacing effort. A coordinated pacing effort could encourage companies to increase the compute allocated to safety across the industry and devote more resources to alignment, interpretability, safety testing, and evaluation.

**What we measured.** We examined a snapshot of how Anthropic used all of its compute from July 13 to July 20[2](https://www.anthropic.com/institute/measuring-pace-of-ai-development#footnote-2). To do that, we sorted every workload into a small number of categories, then asked how much of the compute going to AI R&D was safety work.

Safety research tends to use less compute than frontie

Open article · Retrieved 2026-09-17T21:22:07.370235+00:00

Measurements for  
 understanding the pace of AI development inside frontier labs

AI systems are becoming exponentially more powerful and have begun to [automate more](https://www.anthropic.com/institute/recursive-self-improvement) of the process of building themselves. As the world considers [slowing the pace of frontier AI development](https://darioamodei.com/post/we-must-pace-the-frontier), the public needs more information.

In this post, we lay out measurement tools that can illuminate three critical aspects of AI development:

1. [The extent to which AI is building the next version of itself, as opposed to being built by humans](https://www.anthropic.com/institute/measuring-pace-of-ai-development#1-measuring-ai-led-ai-rd)
2. [Our ability to oversee and intervene in actions that AI agents take on Anthropic’s systems](https://www.anthropic.com/institute/measuring-pace-of-ai-development#2-measuring-oversight-of-ai-agents)
3. [The resources that power the development of more capable models](https://www.anthropic.com/institute/measuring-pace-of-ai-development#3-measuring-compute-allocation)

We also provide a snapshot of these metrics from inside Anthropic. It’s important to note that we would expect these numbers to shift if there were coordination on pacing the frontier, as called for by Anthropic CEO Dario Amodei. We plan to embed independent third-party evaluators from multiple organizations at Anthropic, and give them access to internal processes, systems, and data comparable to what internal risk assessment teams have. These third parties will verify safety practices, report incidents, and monitor key metrics such as the ones in this piece.

We are reporting these measurements because they give the public, third parties, and governments better visibility into the pace of AI development inside frontier labs. For each measurement, we describe what we measured, what the measurement showed, and what it would take to publish these measurements regularly in a form others can verify. We share methodological details in the Appendix.

## Reasons to track these measurements

The measurements in this piece are focused on *how models are built.* By better understanding the production process of models, we have a better chance of correlating model inputs, like compute, with model outputs, like capabilities. They complement capability evaluations, which measure *what models can do*. We publish those separately through our [Responsible Scaling Policy](https://www.anthropic.com/responsible-scaling-policy) (RSP) risk reports, which include evidence on how much our models are accelerating AI R&D. In our policy proposal on advanced AI, the [Advanced AI Framework (AAIF)](https://www-cdn.anthropic.com/files/4zrzovbb/website/0a58d567024a8b448ff15158ebc3625328dfcc1f.pdf), we propose rules of the road for how any lab releases safe models, including transparency obligations that governments could require, such as risk reports. Together, these proposed measurements and policies are a starting point for monitoring the pace of AI development from outside the labs.

## (1) Measuring AI-led AI R&D

**Why measure AI-led R&D?** Frontier AI labs increasingly use AI to build future AI models. This process allows labs in democratic countries to develop more capable models more quickly and conduct more safety and testing on models before they are released to secure AI’s benefits while staying on the frontier. However, models accelerating their own development could make it more challenging for humans to understand or control these systems. It is therefore important to share these metrics to understand how close the world is to reaching [recursive self improvement](https://www.anthropic.com/institute/recursive-self-improvement) (a model fully autonomously building its successor).

**What we measured.** We built a prototype index of how much of Anthropic’s AI research and development (R&D) is performed by Claude, called the Anthropic R&D Automation Index. It’s built by cataloguing every kind of AI R&D work done at the company, rating how automated each task currently is, and aggregating those ratings.

**What we found.** To measure the extent to which AI is doing AI R&D at Anthropic, we use an [automation rating scale](https://epochai.substack.com/p/toward-an-onet-for-ai-r-and-d) developed by Epoch AI that measures “Automation Level,” or AL. It runs from AL0 (no AI involvement) to AL5 (AI operates fully autonomously, with no human in the loop). In AL3, AI “collaborates”: it can do large chunks of work under close human direction. In AL4, AI “leads”: it can complete most of the task end-to-end from a high-level prompt, while the human supervises[1](https://www.anthropic.com/institute/measuring-pace-of-ai-development#footnote-1).

As of August 2026,

- Claude is not operating fully autonomously for any measured subset of AI R&D work.
- Claude “leads” 26% of Anthropic’s AI R&D work.
- The share of work at or above “AI collaborates” is above 90%.

Chart showing Claude now leads 26% of Anthropic's model R&D tasks, up from under 1% in February 2026.

**What any AI developer could report today.** Any frontier developer could publish these measures regularly, using a public methodology. This would enable the numbers to be compared over time, and potentially across labs.

Two obstacles stand in the way of cross-lab comparison on this type of reporting. First is the lack of a common methodology. Second, we’re using our own models to evaluate our systems, which could mean that the “judge” model could make the same kinds of errors as the model it is checking. To address these challenges, a developer’s measurements could be verified by a third party, or by other developers’ models, with guardrails in place to limit the exchange of competitively sensitive data. These measures could also become the trigger for stronger requirements, like a fixed testing window before a new model is used for further AI R&D.

## (2) Measuring oversight of AI agents

**Why measure oversight of agents?** Like other frontier developers, Anthropic employees increasingly delegate tasks to agents that work semi-autonomously for long stretches, and that delegate work to one another. As work becomes increasingly automated, from “AI collaborates” toward “AI leads,” agents could make more consequential decisions, such as which research direction to pursue next.

These decisions need safeguards to block harmful behavior from an individual agent, as well as an oversight regime to detect problems that might only arise across many agents. In our monitoring data to date, individual agents rarely misbehave. But when there are millions or billions of agents operating in the economy, even rare events can happen regularly.

**What we measured.** We’ve built a system that lets us oversee and intervene in actions AI agents take on Anthropic’s systems. Here, we consider three different metrics: *coverage,* which describes the share of an agent’s actions that pass through a monitor before or after execution; *review* *latency*, which is the time between an action and its review, first by an automated monitor and then by a human; and *escalation rate,* whichis the share of agent activities that are either blocked/redirected (in the case of online monitors) or flagged for further review (in the case of offline monitors).

**What we found.** As of August 2026, there were approximately 30,000 agents doing research and engineering work at Anthropic at any one time in our most-used internal platform. These measurements cover this platform only. The actions of these agents are constrained by two kinds of monitors, summarized below:

|  |  |  |
| --- | --- | --- |
|  | **Online monitors** | **Offline monitors** |
| **Description** | Online monitors check the actions of agents *in real-time* and either block dangerous actions or steer agent misbehavior automatically. They’re built to address harms that would happen quickly and could not be undone, such as an agent copying model weights out of our systems. | Offline monitors read agent activities *after the fact*. They are built to help address dangerous behaviors that develop slowly and can be reversed, like signs of model misalignment. We use flagged activities to improve our mitigations, like classifiers and deterministic controls. If the situation caused a large imminent danger, we would shut down the dangerous parts of the agent’s deployment. |
| **Coverage** | 100% of these agents’ actions pass through the online monitor before they are executed, usually within seconds. | 100% of these agents’ actions are ingested by the monitor after they are taken. |
| **Review latency** | Automated review happens before an agent takes action; humans review any blocked actions within one week. | The monitor flags roughly 100,000 transcripts per week that are then reviewed by multiple classifiers for false-positives and initial oversight review. Most of the flags are resolved by this process, and the highest priority flags (approximately ~50 per week) are escalated to human review. |
| **Blocking / escalation rate** | We analyzed over a billion decisions from our research & engineering agents over August 2026 and found that 0.002% of them (about 1 in 47,000) were blocked by this monitor. | The monitor flags roughly one to two transcripts in every thousand for further review. |

**What an AI developer could report today.** Any developer running agents on its own research and engineering workloads could publish the same measures: *coverage* (the share of agent activity that is monitored), *review latency* (how soon flagged activities are reviewed), and *escalation rate* (what share of agent activities the monitors block or flag). Together, these measures would allow society to see whether oversight is keeping pace with the growing role of AI in AI R&D. We published all of these measurements in our recent [risk report](https://www.anthropic.com/aug-2026-risk-report).

Developers should share how often known agent misbehavior is caught by monitors, and should have third parties do this kind of testing independently. In the past, METR has [independently red-teamed our offline monitoring platform](https://www-cdn.anthropic.com/f61d49fa5596956a5dec75fea0e973bf6a6a8378/Redacted%20Risk%20Report%20August%202026%20.pdf). As described above, we are now [setting up](https://darioamodei.com/post/we-must-pace-the-frontier) external third party evaluators at Anthropic.

## (3) Measuring compute allocation

**Why measure compute allocation?** Broadly speaking, AI developers use compute for building more powerful models, serving customers, and safety-focused work like [auditing a model’s “thoughts”](https://www.anthropic.com/research/natural-language-autoencoders), [training model organisms to study misalignment](https://www.anthropic.com/research/emergent-misalignment-reward-hacking), and [evaluating whether a model can be safely deployed](https://www-cdn.anthropic.com/f61d49fa5596956a5dec75fea0e973bf6a6a8378/Redacted%20Risk%20Report%20August%202026%20.pdf). Understanding how AI developers allocate their compute can tell you where a developer is focusing its resources and how that focus changes over time.

Additionally, compute is among the most verifiable inputs to the AI R&D process, meaning that it could be a critical lever in a future pacing effort. A coordinated pacing effort could encourage companies to increase the compute allocated to safety across the industry and devote more resources to alignment, interpretability, safety testing, and evaluation.

**What we measured.** We examined a snapshot of how Anthropic used all of its compute from July 13 to July 20[2](https://www.anthropic.com/institute/measuring-pace-of-ai-development#footnote-2). To do that, we sorted every workload into a small number of categories, then asked how much of the compute going to AI R&D was safety work.

Safety research tends to use less compute than frontie