> I suspect best practices for "vibe coding" will end up like test-driven development: a proven method for writing better software that many engineers still choose to skip.
I’d like to see the proof for TDD; last I heard it slowed development with only minor reliability improvements.
The proof for TDD is usually looking at bug detection rates. Similar for code review. OTOH, the "design damage" of TDD is something overlooked by those metrics.
What it boils down to:
- TDD in the hands of a junior is very good. Drastically reduces bugs, and teaches the junior how to write code that can be tested and is not just a big long single method of spaghetti with every data structure represented as another dimension on some array.
- TDD in the hands of a midlevel can be a mixed bag. They've learned how to do TDD well, but have not learned when and why TDD can go bad. This creates design damage, where everything is shoe-horned into TDD and the goal of 90% line coverage is a real consideration. This is maximum correctness but also potentially maximum design damage.
- TDD in the hands of a senior is a power tool. The "right" tests are written for the right reasons with the right level of coupling and the tests overall are useful. Every really complicated algorithm I've had to write, TDD was a life saver for getting it landed.
Feels a lot like asking someone if they prefer X or Y and they say "X" is the industry best practice. My response universally is now an eye brow raise "oh, is it? For which segments of the industry? Why? How do we know it's actually a best practice? Okay, given our context, why would it be a best practice for US". Juniors don't know the best practices, mid-levels apply them everywhere, seniors evaluate and consider when best practices are not best practices.
TDD slows development when tests are written in a blind way with an eye on code coverage and not correctness and design. TDD speeds up development in being a good way to catch errors and is one of the best ways to ensure correctness.
Developers have selective amnesia and only count dev time when working on what they want to work on rather than including time spent fixing things they’ve already mentally marked as done.
The worst actors find ways to make other people responsible for fixing their bugs.
But that’s just it-as a design aid it can really go off the rails, but as a testing strategy it’s really useful in one domain. Defect fixing. If I can convince a junior engineer that when he gets a bug report to first write a test that shows the problem and then fix it, using the test to prove it’s fixed, it provides immense benefits.
> If I can convince a junior engineer that when he gets a bug report to first write a test that shows the problem and then fix it, using the test to prove it’s fixed, it provides immense benefits.
That's just writing a regression test and making sure it catches the regression. What does that have to do with TDD? Does the philosophy of TDD lay claim to any test written before the bugfix, regardless of how much or little someone subscribes to TDD overall?
It's pretty bad at this. It's much better used as a testing methodology than a design methodology.
It can provide high level guardrails confirming implementation correctness that are as indifferent to software design as possible (giving freedom to refactor).
I always love to compare Peter Norvig’s sudoku solver implementation [1] versus Ron Jeffries TDD approach [2].
Norvig starts with the theory building and creates a constraint solver in about 50 lines of code. Jeffries starts with TDD, assumes an implementation, has to change that implementation, therefore has to change the tests, and after a series of five blog posts kind of fizzles out on it.
To me it just highlights that defining the problem is with so much more than defining the tests, as you can’t write a test for a problem you haven’t defined yet. In this way the tests are an imposition. In short to me it shows that TDD only really works if you already knew how to buold the project to begin with.
So, there you have it. TDD is good if applied correctly, and only if you apply it 100% correct. And so it seems for LLM usage. If it doesn't work for you, then you are obviously doing it wrong according to many folks here. TDD is nice to catch refactoring mistakes, LLMs are nice to maybe do some initial refactoring on a small enough code base. And it doesn't mean that one precludes the other. But I haven't seen TDD put engineers out of work and neither should LLMs. Trust either model fully and you are in for a world of hurt.
I think it works only for people with a certain thought pattern, the ones that like to think everything up-front before doing anything.
Most people prefer to play around and make several crappy attempts and combine them until the whole is somewhat solved, then go over and polish it a little, and maybe then add tests and fix the behavior in place.
For this last group, TDD it's jarring, unnatural and requires a lot of willpower to follow.
It's not bad in itself, it's just not for everyone.
TDD does not require an upfront understanding of the solution. It’s primarily about verifying what your application (or module) does, not how it does it. In other words, tests should be written from the outside in.
The common struggle with TDD arises when people reduce it to writing glorified spell checkers. This usually goes hand in hand with the belief that unit tests must always check classes in isolation using excessive mocks—an approach that misses the real purpose of TDD.
It has less to do with thought patterns and more with simply misunderstanding the approach entirely because of clinging to the wrong dogmas they've heard somewhere.
> the ones that like to think everything up-front before doing anything.
I don't think that's the case. If they were really thinking up-front, they'd be doing proper req analysis and design work, rather than interactively growing a ball of mud that "does the minimal thing to pass a test". To me, it seems like TDD is sold as this "foolproof" design / dev approach, which is anything but.
I see TDD exactly as the best practice for vibe coding! Context is stored as test coverage, to make sure they don't break things when they hallucinate.
TDD works if know what 100% of the inputs/outputs look like, AND you know the implementation will be complex, terse, whatever. Write tests, then twiddle a bunch of regex until the tests pass. This situation rarely comes up in real software jobs, but when it does, yeah use it!
TDD doesn't work for the same reasons why Todo lists don't work. I don't have a good explanation why that is so, but I know from countless examples (mine and other people's) that they don't work.
In general, doing things work, planning to do things don't.
I’d like to see the proof for TDD; last I heard it slowed development with only minor reliability improvements.