Are Unit Tests Still Valuable When AI Writes the Code?
Coding agents write most of my code now. More and more, it’s an agent rather than a human that checks the code before it’s merged. So I’ve been thinking a lot about tests, and which ones are actually worth having when AI writes the code.
Here’s where I’ve got to so far…
Why Tests Matter More With Agents
When I review a PR, I’m one of the checks. As we move to agentic pipelines, where an agent reviews and approves a change instead of a person, that check goes away and we rely on the pipeline instead.
So we definitely need tests, probably more than ever. If agents are gating things instead of humans, we need strong verification, and tests are a fundamental part of that. The question is, which tests?
The Tests I Think Are Essential
For me, the essential tests are the ones that are closest to reality. They test the real flows that users go through, rather than edge cases and nooks and crannies that users might never reach.
End-to-end tests are the most valuable. They run after the merge, against a real deployed environment, so they’re running the app pretty much for real. They give me an enormous amount of confidence.
Integration tests aren’t quite as close to reality, but they’re much faster than end-to-end tests. That means I can run them on every PR, before anything gets merged into main. On the frontend, an integration test drives a whole screen or feature, whether that’s in a web app or a native app. On the backend, it hits an endpoint. Only things outside the process are faked, like the database, auth and the API client. Everything else is the real code.
So, What About Unit Tests?
By unit tests, I mean the low-level ones that test individual functions and classes, usually with everything around them mocked. I don’t think they’re that valuable anymore, for a few reasons.
Tokens. Every time the agent changes some code, it has to change the unit tests for that code too. Low-level tests are tied to the implementation, so even a refactor that doesn’t change any behaviour can break them. The agent then has to find the broken tests, fix them and run them again. That all costs tokens.
Time when I’m in the loop. Most of the time, my agents work autonomously. But every now and again, I want to tweak a bit of the UX. That’s when I’m in the loop, going through a handful of iterations with the agent until the experience feels right, and I’m sat waiting on every one of them. If the agent is updating unit tests each time as well, each iteration takes longer and I really feel it.
Code quality. Writing unit tests has traditionally helped me write high-quality code. With agents writing the code, I mostly care about the design at a high level, like deep modules with small interfaces that aren’t coupled to each other. Does it really matter if a low-level private function behind one of those interfaces is a little bit dirty? I’m not sure it does, as long as the module has the right interface and does the right thing behind it.
Build time. This is a minor point, but the more tests you run, the longer the build takes. Unit tests are fast to run, but a large codebase can end up with a lot of them, and that can add minutes to the pipeline.
When Are Unit Tests Worth It?
For me, there’s only one case where unit tests are worth it, and that’s complex logic. Something like a parser, with lots of different inputs, lots of variants and quite a few different outputs. These are very good to unit test. Each case is quick to write and run, and testing them all through a screen would be painful.
But a lot of low-level functions take a single input and return a single output, and an integration test already covers what they do. There’s very little value in a unit test for those, is there?
Where I’ve Landed
For now, I want end-to-end tests after the merge and integration tests on every PR. My default is no unit tests unless the logic is complex, and that’s what I tell the coding agent:
Add high-level integration tests for new behaviour, not low-level unit tests.The exception is a complicated low-level function with lots of input variantsand outputs, such as a parser. Unit test it through its public interface.Even with a parser, I’d still want an integration test over the top of it.
Things are moving fast, though. If models get cheap and fast enough, the token cost might stop mattering and I might change my mind. I could think completely differently about this in six months!