So, just trying to understand this - he admits to code being slop and in the same sentences states that agents (which created the slop in the first place) also refactor it? Where is the logic in that?
I feel in this context, refactoring has lost its meaning a bit. Sure, it's often used analogous to changes that don't affect semantics. But originally, the idea was that you make a quick change to solve the problem / test the idea, and then spend some time on properly integrating the changes in the existing system.
LLMs struggle with simplicity in my experience, so they struggle with the first step. They also lack the sort of intelligence required to understand (let alone evolve) the system's design, so they will struggle with the second step as well.
So maybe what's meant here is not refactoring in the original meaning, but rather "cleanup". You can do it in the original way with LLMs, but that means you'll have to be incredibly micro manage-y, in my experience. Any sort of vibe coding doesn't lead to anything I'd call refactoring.
I think a lot of this is because people (and thus LLMs) use verbosity as a signal for effort. It's a very bad signal, especially for software, but its a very popular signal. Most writing is much longer than it needs to be, everything from SEO website recipes, consulting reports, and non-fiction books. Both the author and the readers are often fooled into thinking lots of words are good.
It's probably hard to train that out of an LLM, especially if they see how that verbosity impressess the people making the purchasing decisions.
> I think a lot of this is because people (and thus LLMs) use verbosity as a signal for effort.
It's also one of the main use-cases for non-programmer use of the models, so there are business-forces against toning it down. Ex: "Make a funny birthday letter for my sister Suzie who's turning 50."
LLMs are good at pursuing objectives, but they aren't necessarily good at juggling competing objectives at once. So you can picture doing the following, for instance:
- "Here is a spec for an API endpoint. Implement this spec."
- "Using these tools, refactor the codebase. Make sure that you are passing all tests from (dead code checker, cyclomatic complexity checker, etc.)"
The clankers are very good at iteratively moving towards a defined objective (it's how they were post-trained), so you can get them to do basically anything you can define an objective for, as long as you can chunk it up in a way that it fits in their usable context window.
"So, just trying to understand this - he admits to code being buggy and in the same sentences states that he (the engineer who created the bugs in the first place) also debugs it? Where is the logic in that?"
No I'm highlighting that the logic itself is stupid
If the point is that you can't solve with AI what you messed up with AI, but with human intelligence spending a bit more time on the problem does indeed tend to help, you need to explain why his technique with the AI won't work either.
Plus he's adding human input to it every time, so I see no reason to default to "it wouldn't work".
Well you said it yourself, you are literally comparing human intelligence with the so-called AI, or better said, advanced text generator. The differentiator being, the text generators have 0 intelligence, otherwise there would not be a flood of AI gurus explaining the latest trick to making them finally work.
