AI agents write more code, but companies don't ship more software

I’ve been working with coding agents every day for months, and there’s a feeling I find hard to explain: I write far more code than before, but I’m not sure the team ships that many more things to production. That’s why I was so interested in a study by Fiona Chen and James Stratton, from Harvard University, which Ars Technica sums up with a very clear headline: AI coding agents generate more code, but not more software.

The study, Artificial Intelligence in the Firm: Bottlenecks in Software Production, isn’t based on surveys or lab experiments, but on what actually happened at companies that build software. Its conclusion is that the productivity agents gain when writing code is lost further down the line, in review.

Where the data comes from

The data comes from Jellyfish, a platform companies use to analyse the activity of their engineering teams. It pulls together information from GitHub, Jira, Google Calendar, HR systems and the use of AI tools such as GitHub Copilot, Cursor and Claude Code.

In total, around 300 million work events (commits, pull requests, issues…) from more than 700,000 people at 718 companies, from January 2021 to March 2026. The study compares each company before and after adopting assistants (which autocomplete code a person writes) or agents (which write and submit code on their own), taking advantage of the fact that each one adopted them at a different time.

More code…

When it comes to writing code, agents deliver. Assistants barely move the needle, but agents do:

For assistants, only the increase in commits is statistically significant. For agents, all three are.

…but not more software

The problem shows up when you measure what actually reaches production. Chen and Stratton use two measures: resolved Jira issues, which are closed when the code is deployed, and completed epics, which represent whole features or projects.

With agents, neither changes significantly. The data rules out software output rising by more than 12%, far from the 30% in lines of code. And it isn’t that tasks got bigger or more complex: they checked with a model that estimates how long each issue takes to resolve, and that doesn’t change either.

There’s no effect on employment either. For total company headcount, measured through LinkedIn, the effect is practically zero: they rule out a drop of more than 2.9%. Neither more software nor fewer people.

The bottleneck is review

The most interesting part of the study is that it explains where the gain goes. Writing code is only part of the job: according to their data, engineers spend between 30 and 40% of their time writing code. The rest goes into planning, reviewing, testing and deploying. If you speed up only one of those stages, the jam moves to the next one.

And the next one is review. After adopting agents:

There isn’t just more code to review: each pull request needs more review work. The model the study proposes separates exactly those two effects: AI increases the amount of code and at the same time changes its quality, with more bugs that have to be found before accepting it.

And AI review tools aren’t solving it yet. By March 2026, 80% of companies were already using one, but AI wrote only 23% of review comments. People still do the bulk of the work.

flowchart LR
    A["Plan"] --> B["Write code · agents +30%"]
    B --> C["Review · +49% time"]
    C -->|"changes requested"| B
    C --> D["Deploy · no change"]

The limitations

It’s worth reading carefully. The data ends in March 2026, and agents have improved a lot since then. The study also measures when the company adopts agents, not how much each person uses them, so it mixes teams that use them heavily with others that have barely tried them. And the companies are Jellyfish customers who agreed to share their data, who don’t necessarily represent everyone.

Even so, the result matches an earlier study of open-source repositories, which also found many more lines of code and commits with Copilot, but a much smaller effect on pull requests and published releases.

Two more reads from the same debate

Two pieces published these past days fit the study very well.

Nate Meyvis asks in Where is all the new, good software? where all the new, good software that should be coming out is, and offers several answers. The one I find most convincing: many company processes, like getting senior engineers and product to sign off on a plan, haven’t been sped up by AI. Another very interesting one: much of the new software is private (internal tools, personal dashboards, one-off scripts), and we don’t see it.

Rich Hickey, the creator of Clojure, goes further in ‘AI’ is Not a Tool. For him, a tool is something you wield, with continuous control and feedback, like a hammer. LLMs aren’t that: “LLMs are automation”, something you hand the work to and that does it on its own. I don’t fully agree with him, but his argument explains the study quite well: if the agent writes the code without you having wielded it, the moment you get control back is review. And there’s no shortcut there.

My takeaway

I keep two ideas the study itself points out in its conclusion.

The first: measuring lines of code or commits doesn’t tell you whether AI is working. You have to look at the whole process, from planning a feature to getting it into production. If you only look at the stage AI speeds up, it will always look like a success.

The second: speeding up one stage moves the bottleneck to another. It’s the old theory of constraints applied to software. If agents write faster than the team can review, what’s needed is a change in how we work: smaller pull requests, better automated tests, deciding which code needs a thorough review and which doesn’t, and having whoever runs the agent review their own work before handing it to someone else.

I still use agents every day and have no plans to stop. But this study puts numbers on something I suspected: writing the code was the easy part to automate, and understanding and validating what’s been written is still work for people.

Source: Artificial Intelligence in the Firm: Bottlenecks in Software Production, Fiona Chen and James Stratton (Harvard), via Ars Technica.