That was actually the case for Claude Opus 4.5, the newest model of Anthropic’s strongest mannequin, which was launched in late November. In December, METR introduced that Opus 4.5 gave the impression to be able to independently finishing a job that may have taken a human about 5 hours—an unlimited enchancment over what even the exponential pattern would have predicted. One Anthropic security researcher tweeted that he would change the path of his analysis in mild of these outcomes; one other worker on the firm merely wrote, “mother come decide me up i’m scared.”
However the fact is extra difficult than these dramatic responses would recommend. For one factor, METR’s estimates of the talents of particular fashions include substantial error bars. As METR explicitly said on X, Opus 4.5 may be capable of repeatedly full solely duties that take people about two hours, or it would succeed on duties that take people so long as 20 hours. Given the uncertainties intrinsic to the tactic, it was inconceivable to know for positive.
“There are a bunch of ways in which individuals are studying an excessive amount of into the graph,” says Sydney Von Arx, a member of METR’s technical employees.
Extra essentially, the METR plot doesn’t measure AI skills writ massive, nor does it declare to. So as to construct the graph, METR checks the fashions totally on coding duties, evaluating the problem of every by measuring or estimating how lengthy it takes people to finish it—a metric that not everybody accepts. Claude Opus 4.5 may be capable of full sure duties that take people 5 hours, however that doesn’t imply it’s anyplace near changing a human employee.
METR was based to evaluate the dangers posed by frontier AI programs. Although it’s best recognized for the exponential pattern plot, it has additionally labored with AI firms to guage their programs in better element and printed a number of different impartial analysis initiatives, together with a widely covered July 2025 study suggesting that AI coding assistants may truly be slowing software program engineers down.
However the exponential plot has made METR’s popularity, and the group seems to have an advanced relationship with that graph’s typically breathless reception. In January, Thomas Kwa, one of many lead authors on the paper that launched it, wrote a blog post responding to some criticisms and making clear its limitations, and METR is at present engaged on a extra intensive FAQ doc. However Kwa isn’t optimistic that these efforts will meaningfully shift the discourse. “I feel the hype machine will mainly, no matter we do, simply strip out all of the caveats,” he says.
Nonetheless, the METR staff does assume that the plot has one thing significant to say concerning the trajectory of AI progress. “It is best to completely not tie your life to this graph,” says Von Arx. “But in addition,” she provides, “I wager that this pattern is gonna maintain.”
