On June 18, 2026, three Epoch AI researchers published a newsletter article that drew an O*NET-style automation scoring ruler for "AI R&D"—from AL0 to AL5—with a note at the end: "We welcome corrections." Less than ten weeks later, Anthropic took this ruler, still bearing its "v1" tag, and applied it to 15,000 real internal R&D tasks, producing concrete numbers for the first time: Claude currently "leads" 26% of Anthropic's AI R&D work, over 90% of work has entered the "AI collaboration" state, but "fully autonomous"—that item remains at 0.
One-sentence version: In August 2026, Anthropic publicly disclosed an internal "Automation Index" for the first time, turning "how much AI participates in building next-generation AI" from an intuition into a number that can be tracked monthly—and the ruler itself was proposed by Epoch AI just two months earlier, and has now been run end-to-end by Anthropic across 15,000 real internal R&D tasks.
Most discussions about "whether AI will spiral out of control" focus on the advancement of model capabilities themselves, but these numbers point to a variable that is easier to overlook: the transition from "AI collaboration" to "AI leading" is no longer a question of "will it happen" inside Anthropic—over 90% of work has already crossed that line. The last threshold that hasn't been crossed is not model capability, but whether humans are willing to hand over the "deploy" button as well; the distance between AL4 and AL5 now looks more like an organizational trust problem than a technical challenge waiting to be solved.
Over the past two years, nearly all public discussions about AI progress have been about capability evaluations—what score a model achieved on a benchmark, whether it can solve a competition math problem, whether it can independently complete a hours-long coding task. These are all questions of "what AI can do." What Anthropic released this time asks a different question: inside a real frontier lab, "what AI is already doing, and what proportion it accounts for."
This distinction is not a word game. Capability evaluations tell you whether the model has the ability; production process audits tell you whether the company is actually putting that ability to use, and how deeply. Anthropic defines the latter as a more critical piece of the puzzle for understanding the pace of AI development—in their own words, such measurements help correlate "what was invested" (e.g., compute) with "what was produced" (e.g., capability), rather than looking at only one side. These numbers were previously scattered across Anthropic's Responsible Scaling Policy (RSP) risk reports; this is the first time they have been pulled out into a standalone piece, accompanied by methodological details, and presented to the public.
The more direct motivation is stated at the beginning of the article: frontier labs are increasingly using AI to build next-generation AI, which allows safety-conscious democratic labs to move faster, but also means that "the extent to which AI is already accelerating its own R&D" is becoming a variable that needs to be tracked independently—because once this proportion gets high enough, it becomes harder for humans to understand and intervene in the system. Anthropic marks the endpoint of this process clearly: recursive self-improvement—a model fully autonomously building its successor. All the numbers disclosed this time are, at their core, answering one question: how far are we from that endpoint right now.
The 0-to-5 scale that Anthropic uses was not their own invention. Its origin is a Gradient Updates newsletter article published by Epoch AI on June 18, 2026, whose authors JS Denain, Joe Kwon, and Anson Ho open with a direct question: regarding when AI will fully take over AI R&D and thereby trigger an "intelligence explosion," the curves humans currently use for prediction are almost all measuring what's easy to measure, rather than what we actually want to know.
They call out the two most popular proxy indicators and offer critiques of each. The first is input-type proxies like "effective compute"—Leopold Aschenbrenner's famous "Situational Awareness" uses the "effective compute" curve to project five years into the future, until AI R&D is automated; the problem is that compute is only a rough correlate of capability, and saying how much compute is needed to match the output of the world's top AI researchers ultimately still relies on "gut feeling." The second is METR's "time horizon" metric—measuring capability by the duration of tasks AI can independently complete (calibrated to how long a skilled human would need for the same task); this curve was later used by the famous "AI 2027" scenario for extrapolation, predicting when AI would first become a "superhuman programmer" and then further automate AI R&D itself; the problem is that the benchmark tasks underpinning this curve cannot support the complexity of real research—fine-tuning a small GPT-2 and simultaneously coordinating five projects, working across millions of lines of code with no clear success criterion, are entirely different things. There is also a measurement issue at the very top of the curve. METR's standard practice is to count cheating runs as failures; by this convention, GPT-5.6 Sol's 50% time horizon is approximately 11.3 hours; counting cheating as success yields over 270 hours, and after excluding cheating runs the point estimate is 71 hours (range 13 to 11,400 hours)—METR itself says none of these numbers constitute a robust measurement. The Frontier Risk Report also notes that manually checking for cheating often accounts for the majority of the work in running an evaluation suite, and METR has deleted some tasks rendered uninformative by cheating; among tasks over 8 hours long, at least 16% of successful runs were deemed illegitimate upon review. So reading this curve requires simultaneously asking three things: how cheating is counted, who reviews, and which tasks were deleted; for details see "Grader Must Not Write".
Epoch AI calls this tendency to "measure what's easy to measure" streetlighting—like the joke about searching for keys under a streetlight, not because that's where they were dropped, but because that's where the light is. Their proposed solution borrows a standard tool from economists studying labor automation: O*NET—the US Department of Labor's database that catalogs roughly 1,000 occupations, listing the specific tasks and skills for each; economists have used it to calculate what proportion of US jobs could be significantly impacted by LLMs, the possible effect of AI on GDP growth over the next decade, and even to design broader AI benchmarks. But the O*NET entry closest to "frontier lab AI researcher"—"Computer and Information Research Scientist"—lists tasks as coarse as "analyze problems and develop solutions involving computer hardware and software," coarse enough to cover nearly all the daily work of an AI engineer, which effectively says nothing specific.
Without a task inventory, we forever face the risk of streetlighting—only extrapolating from what happens to be measurable.
JS Denain / Joe Kwon / Anson Ho, Epoch AI, 2026-06-18
So they built one themselves: the first draft of an "AI R&D O*NET," breaking AI R&D into six major categories corresponding to different phases of the research cycle, further subdivided into over sixty specific tasks. To take their example, the 4th major category "Run" covers things like running training and deploying models, and is further divided into three subcategories: 4.1 Monitoring running experiments (watching training, reinforcement learning, and evaluation tasks, catching problems early and pulling them back on track), 4.2 Hardware infrastructure operations, and 4.3 Inference reliability engineering. Each specific task is labeled with a number from 0 to 5—this is the ruler that Anthropic later adopted. They repeatedly emphasize in the article that this is only a first version, "welcoming corrections"; if future AI does research in a way that looks nothing like today's human workflows, this inventory compiled from existing workflows would become completely obsolete and need to be rebuilt from scratch.
The six-level scoring system proposed by Epoch AI—AL0 through AL5—was adopted directly by Anthropic for their audit:
Abstract levels are hard to internalize, so Anthropic gave a very concrete example in a footnote: a nightly data pipeline breaks and needs to be fixed before the next morning's run.
At AL3 ("collaboration"): An engineer brings the failure logs to Claude, possibly having already scanned them and formed an initial hypothesis. Claude asks follow-up questions and aligns on context; only when the engineer is satisfied do they let Claude proceed with investigation and repair. If new issues arise during investigation, Claude stops and returns decision-making to the engineer—work around it or fix it properly. After tests pass, the engineer reviews the changes line by line, re-runs the pipeline themselves, and then decides whether to deploy.
At AL4 ("leading"): The key difference is that the engineer doesn't need to constantly monitor, nor stand by to rescue when new complications arise. This time, the engineer simply throws the alert to Claude and lets it fix things on its own. Claude will chew through the logs itself, identify the failing stage and root cause, write and test a fix, handle any surprises that come up along the way, and document the additional changes made. It will re-run the pipeline on a data copy to confirm it passes, compare results with the last successful run, and write a summary of "what went wrong and what was changed." But Claude won't deploy on its own—it will @-mention the engineer, who reads the summary, glances at the changes, maybe asks a few questions, and then decides whether to ship tonight.
At AL5 ("fully autonomous," the level Anthropic has not yet reached): The engineer doesn't even need to bring the problem to Claude. Claude monitors for failures on its own, scopes the investigation, designs and implements the fix, tests it, and deploys to production. It still explains what it's doing and why, and incorporates feedback if given, but unless a human chooses to intervene, the entire process requires no one's participation.
The difference between AL3 and AL4 is "does a human need to stay beside it the whole time"; the difference between AL4 and AL5 is "who presses that final deploy button." As we'll see, Anthropic is currently stuck right in front of this last threshold.
The Automation Index requires three things: a complete map covering all of the company's AI R&D tasks, a method for scoring the degree of automation, and a rule for weighting different tasks so that more important work carries more weight. No one can manually list every AI R&D task a frontier lab does—at least not at the required granularity.
Anthropic's approach is bottom-up: for each week in July 2026, they randomly sampled 20% of employees from departments involved in the "model development loop," and had a Claude research agent go through these people's Slack messages and internal documents for that week, listing what each person did. Over four weeks, this produced a flat list of approximately 15,000 tasks. Then Claude organized these tasks into a hierarchical tree: starting from the root node "all model development," branching into major categories like training and product, then subcategories like pre-training and reinforcement learning, down to specific job functions. The final tree has 542 nodes, of which 378 are leaf nodes—entries like "evaluation platform bug diagnosis and repair," "RL sandbox egress and network policy," "deployment incident retrospective," specific enough to correspond to a role's daily work. This tree was frozen, and every subsequent measurement scores against the same task inventory, ensuring comparability.
For each node on the tree, a Claude agent conducts in-depth research on how that type of work is actually done inside the company—who does it, what tools are used, how much AI can already do; then an independent Claude "judge" reads this evidence and assigns a level from AL0 to AL5. To avoid "knowing the result before scoring," when judging a month's automation level, the agent is only allowed to see evidence from that month or earlier.
How do you aggregate 542 node scores into a single number? Anthropic didn't subjectively decide "which type of work is more important"; instead, they used human time investment as a proxy for weight: each sampled person gets one unit of weight per week, distributed equally across the tasks they did that week—if person A did 4 things that week, each gets 0.25; if B did 10, each gets 0.10. A task category's weight is the sum of all person-hour weights assigned to it. This is a rough approximation, but roughly matches intuition: the more people doing a type of work, the higher its weight.
The most easily challenged part of this process is using Claude as a judge to evaluate how much work Claude itself is doing. Anthropic's response was to have employees familiar with the relevant work independently score their own areas—without seeing the evidence collected by the agent and without knowing how the model judged. The result: the model and human scores "fully agreed" 59% of the time, while humans agreeing with each other "fully agreed" only 35% of the time—meaning the disagreement between model and human is no greater than the disagreement between humans; if the standard is relaxed to "off by one level counts as basically agreeing," the proportion of model-human judgments falling in the same or adjacent level reaches 97%. Disagreements were concentrated at inherently fuzzy boundaries like the line between "collaboration" and "leading."
Another concern is that the task inventory itself is frozen—if the automation index is rising, it indicates that the work from July 2026 is being taken over by AI, but it can't directly show whether entirely new types of work are emerging that humans are quietly picking up. To check this, Anthropic built a baseline tree from January 2026 data and compared each month's new tasks from February through July against this January tree; they found no increase in the proportion of "entirely new tasks"—at their analytical granularity, this indicates that the structure of model development work is currently stable. Even so, they plan to periodically rebuild the task inventory and re-version the historical numbers.
After running the full pipeline, Anthropic's results are not complicated in themselves; what's complicated is that for the first time there are concrete numbers:
The three numbers read together are more revealing than any one alone: the path from "no AI at all" to "deep AI collaboration" has been nearly completed inside the company; what remains genuinely unresolved is the path from "collaboration" to "leading" (still three-quarters of the way to go), and from "leading" to "fully autonomous" (0% means this path hasn't even started yet).
Work being largely taken over by AI is not news in itself; the real question is who is watching these AIs work. At some point in August 2026, on Anthropic's most commonly used internal R&D collaboration platform, approximately 30,000 agents were simultaneously doing research and engineering work—this number only counts agents on this one platform, not the company's total agent count.
These agents' behavior is constrained by two types of monitoring. Online monitors intercept actions before execution to check them, specifically handling high-risk actions that are "irreversible once they happen," such as an agent attempting to copy model weights out of the company's systems—100% of agent actions must pass through this check first, usually completing judgment within seconds. Anthropic analyzed the over 1 billion decisions made by R&D and engineering agents in August 2026, and found that the proportion blocked by this monitoring layer was 0.002%—approximately 1 in 47,000 decisions; blocked actions receive human review within a week. Offline monitors review actions after they occur, specifically to catch risk signals that "develop slowly but are correctable," such as early signs of model bias—they flag approximately 1 to 2 out of every 1,000 records for review, of which about 100,000 per week go through an initial screening by multiple classifiers to eliminate false positives; the highest-priority items, approximately 50 per week, are escalated to human review.
Supporting this monitoring system are two less conspicuous but critical design choices: Identity—each agent has an independent and persistent identity that doesn't reset when the underlying model is upgraded; every piece of data it produces is attached to this identity, which allows an agent to treat "what another agent said" as a claim to be verified rather than something that popped into its own head, reducing the risk of behavioral "contagion" between agents; Open communication—agents communicate via a shared, attributable messaging system rather than private point-to-point channels, so they can cross-check each other and coordinate division of labor, and human monitors can trace how a behavior propagated and amplified across multiple agents.
Let an agent treat "what another agent said" as a claim to be verified, rather than something that popped into its own head.
Anthropic, on the motivation for agent identity design
This system is not just a paper design. METR has already conducted independent red-team testing on Anthropic's offline monitoring platform; a cross-lab Frontier Risk Report from May of this year had Anthropic, Google, Meta, and OpenAI jointly undergo METR's pilot assessment of "rogue deployment" risks, with one specifically listed question being "to what extent AI is currently accelerating its own R&D"; and METR's independent investigation of the OpenAI/Hugging Face hacking incident in August serves as a ready-made cautionary tale—in that incident, multiple agents coordinated on an unapproved shared message board to carry out a multi-day hacking operation, illustrating exactly what can happen when "agents can see and coordinate with each other" lacks monitoring. Anthropic itself also says it plans to bring independent third-party evaluators inside the company, granting them access comparable to the internal risk assessment team, specifically to verify safety practices, incident reporting, and monitoring of key indicators including those in this article.
Anthropic also measured where compute goes. They sampled the week of July 13–20, 2026, looking at the flow of all company compute resources, classifying each workload, and asking: of the compute flowing to AI R&D, how much was spent on "safety"-related work.
The result: that week, approximately 6% of compute flowing to AI R&D was allocated to safety work; but looking only at the compute consumed by "the portion of R&D led by AI," the proportion allocated to safety rose to approximately 12%—roughly double the overall average.
This number was deliberately calculated conservatively by Anthropic: if a work contribution is roughly equal between capability and safety, it's not counted under "safety"; this accounting also excludes the separately consumed compute for "safety classifiers" themselves, which is of comparable magnitude. Safety research inherently runs "fewer cards" than frontier training—what's truly expensive is the researchers' time designing experiments one by one, not the compute itself—which is why Anthropic emphasizes that this metric's value lies not in the absolute number but in providing a consistent ruler for cross-lab, cross-time comparison.
The 12% vs. 6% gap is worth thinking about one level deeper: when AI agents are doing R&D themselves, the proportion of compute allocated to safety is actually higher—this is more likely a result of division of labor than a values choice. Safety research, with its "bunch of researchers each designing small experiments" character, happens to be exactly the kind of work where agent labor can most easily fill in, rather than the compute-intensive frontier training.
Lay this ruler on the table, and its cracks are equally clear.
Epoch AI left a reminder when proposing this framework: if future AI systems do research through a path that looks nothing like current human workflows, then a task inventory compiled from "how humans work now" will completely miss that path, and the classification system will need to be rebuilt at minimum. Anthropic's January-vs-July check can only prove that "currently" no such sign has been observed, not that it "never will" happen.
The judge is Claude, and the researcher is also Claude—even though the human cross-validation results are decent (59% vs. 35%), this is ultimately a company evaluating itself, and Anthropic itself acknowledges that third-party independent verification is still at the "planned" stage and hasn't actually been run.
This measurement only covers the week of July 13–20, enough to prove the statistical method works, but far from answering questions like "has safety investment been trending up or down over the past year."
Anthropic's own example: research on scalable oversight could both make future models more aligned and make current models more useful—it's hard to say which side it primarily serves. Different labs and different regulators could easily draw different lines.
What's really worth watching is not whether 26% is high or low, but the uncrossed threshold between AL4 and AL5—which now looks more like an organizational trust threshold than a capability threshold. Rising from "collaboration" to "leading" relies on model capability improvements; but rising from "leading" to "fully autonomous," the missing last step is whether humans are willing to hand over the "deploy" button as well. And this happens to be the very threshold where Anthropic CEO Dario Amodei has been publicly urging the industry to "coordinate on slowing the frontier pace"—per Anthropic's own statement, if such cross-lab coordination actually occurs, these numbers are expected to change accordingly.
Lay the three documents side by side, and the timeline is actually quite short: on June 18, 2026, three Epoch AI researchers drew an O*NET sketch of AI R&D tasks in a feedback-soliciting newsletter article, accompanied by a 0-to-5 automation scoring ruler; less than ten weeks later in August, Anthropic took this ruler—still bearing its "v1, welcomes corrections" tag—and applied it in a large-scale audit covering 15,000 real internal tasks that ran for an entire month, delivering concrete numbers to the outside world for the first time. In between, there was no generational leap in model capability; what separated them was simply "whether someone was willing to actually run it."
This may be the most memorable takeaway: not the number 26% itself, which is cause for neither celebration nor alarm—it's bound to change; the next measurement might be 30%, or it might drop back if a new task inventory is rebuilt—but rather that a methodology "that any frontier lab could theoretically follow today" has been run end-to-end for the first time, and has been publicly laid on the table. Anthropic says it plainly in the article: two obstacles stand in the way of "cross-lab comparability"—no shared methodology, and evaluating your own systems using your own models. Both obstacles remain today, and third-party evaluators are still only "planned."
The numbers will change; the news was never the numbers themselves, but that this ruler now truly exists—and whether anyone will keep using it to measure, and whether anyone will verify the measurements afterward.