That rather depends on the type of bug and what kinds of tests you would write.

LLMs are way faster than me at writing tests. Just prompt for the kind of test you want.

Idk about you but I spend much more time thinking about what ways the code is likely to break and deciding what to test. Actually writing tests is usually straightforward and fast with any sane architecture with good separation of concerns.

I can and do use AI to help with test coverage but coverage is pointless if you don’t catch the interesting edge cases.

> My philosophy is to let the LLM either write the logic or write the tests - but not both. If you write the tests and it writes the logic and it passes all of your tests, then the LLM did its job. If there are bugs, there were bugs in your tests.

Maybe use one LLMs to write the code and a wildly different one to write the tests and yet another wildly different one to generate an English description of each test while doing critical review.

Disagree. You could write millions of tests for a function that simply sums two numbers, and it’s trivial to insert bugs while passing that test.

This is pretty nifty, going to try this out!

I don't agree. What I do agree on is to do it not only with one LLM.

Quality increases if I double check code with a second LLM (especially o4 mini is great for that)

Or double check tests the same way.

Maybe even write tests and code with different LLMs if that is your worry.