METR: early-2025 AI slowed experienced developers by 19%
A trial found slower completion despite perceived gains. A later study suggests newer tools may help more, but selection effects prevent a reliable estimate of the improvement.
Feeling faster with AI and finishing work faster may be different things. In July 2025, METR reported a randomized trial involving 16 experienced open-source developers and 246 real tasks in familiar repositories. Allowing early-2025 AI tools increased completion time by 19%.
Developers nevertheless believed afterward that AI had made them about 20% faster. The original study documents that gap. It is evidence from a particular setting and period, not a verdict on every coding tool.
Keep the date and setting attached
Participants knew their repositories well and primarily used early-2025 Cursor and Claude tools. The finding cannot be transferred directly to beginners or models released in 2026. Building a first small page and maintaining a mature project with established conventions are different tasks.
Our editorial takeaway is to count the whole route to usable work: understanding a proposed change, checking it, testing it and repairing it. Generation time is only one part of that route.
The same question applies outside programming. An email draft that requires extensive correction may not save time. Assistance that helps you navigate unfamiliar material could save considerable searching. Compare the complete task you actually perform.
The follow-up does not support a tidy reversal
In its February 2026 update, METR says newer tools likely help developers more, but selection effects make the follow-up data unreliable for estimating the size of the gain. Some developers did not want to work without AI; some withheld tasks they expected AI to help with most.
The early experiment produced a result with defined conditions. Later work found that changing adoption and participant behavior also required changes to measurement. That is not a simple sequence of proving AI useless and then proving it useful.
To assess your own work, record comparable tasks from receipt of the material to an acceptable deliverable. Include time, revisions and quality. Also record work AI helped you attempt that you could not previously complete. Efficiency and expanded capability both matter, but they should not collapse into one vague feeling of speed.