Tests that don't test anything come in at least two categories for me:
- test that were useless, are still useless and will always be useless
- tests that are currently useless but were used in the "wtf should i write" phase of coding (templating/TDD/ whatever you want to call it).
I'm partial towards the seconds, and i like when they're not removed, because often you understand how the API/algorithm was coded thanks to them (and its often unit tests). But ideally, both should be out of a codebase.
Used to be (perhaps still is) a nasty habit of Rails apps to have vast test suites covering every Active Record query they ever used (with fixed seeds to boot), rarely straying from giving the bog-standard and already very thoroughly tested and battle-scarred AR predicate builder a wholly unneeded workout; but none of their own front-end code because writing for selenium was too hard.
But look! Thousands of tests and they all pass! Taste the quality!
> but none of their own front-end code because writing for selenium was too hard.
I've also seen plenty of tests that test if a template was rendered rather than if whatever thing it actually outputs was in the output. It is just calcifying the impementation making it hard to test.
But it is a tradeoff, and a hard one as well, because if you do all things all the time, combining all variations of database with all variations of the views, then you end up with a test suite that take forever to run. Finding the right tradeoff there has not shown itself to be very obvious, sadly.
One thing I do sometimes is to start part of an API in a TDD style. Everything starts very "basic", which adds a lot of relatively trivial test cases.
When done with that phase and my API looks relatively functional, I remove all relatively trivial tests and I write bigger ones, often randomized and property-based.
This works decently well and you do not have an army of useless tests there hanging after the process is done.
