Then what it was chained to is the real issue, not this one. The entire point of having webpages run in their own process is to prevent bugs like this one from doing worse. If you're claiming this bug is the bug that matters, you're effectively claiming they shouldn't need to run pages in their own process and just trust that there are zero bugs. No major browser does that. Not Firefox, not Safari, and not Chromium.
that's why bugs in the webpage process pay out very little. Without a worse 2nd bug, they are less serious. Bugs that let you RCE outside the webpage process pay much higher.
It seems to me that if they'd just about reached their upper limits in early 2024 agentic coding wouldn't be eating software development like it is now, and wasn't then.
LLMs are broader than just coding. The article states this particular Zitron claim was wrong, even at the time, because you can put an LLM on a loop until it generates code that compiles. Mitigating hallucinations (in the subset of applications with verifiable output) is not really the same as solving that category of problems outright. I could quibble about a number of things in this example alone, but that's not really my point.
I don't care if the claims are correct or incorrect. I just think the tone of the article is dishonest about its own impartiality and fairness, because it doesn't even attempt to interpret the claims in any way except the least favorable. If you want to be persuasive, you should refute a claim using the most favorable interpretation of that claim possible -- this does the opposite.
> In the first case, TFA refutes by claiming 'wrong' repeatedly, which does not convince me of anything. If anything, Zitron is probably right in this regard, since the majority of progress in recent LLM tech has been setting up of guardrails to cajole the models using 'agents'.
I am certain that plugging a circa 2023 model into a 2026 harness would be a pretty frustrating experience. Yes, you could code a bit with AI in 2023, but models are just much better at it than they used to be. And smaller open models are leaps and bounds better at it than they were three years ago.
Agreed, though I do think that LLMs are still more similar than we think. Sometime after the first release, AI labs found that coding sat in the niche space of lots of easily digestible data and fast feedback from error messages and compiler checks etc. This allowed models to be trained with a focus on coding tasks, but the underlying technology is still the same, the infrastructure around it changed, they are still generating via probabilistic sampling.
Don't get me wrong, I'm using local coding agents myself with varying levels of success and frustration, but the models themselves behave similarly to their siblings from 2020.
The infrastructure improved, that includes the data. I have a pet theory that if they took the earlier models and retrain it with the data they used to train the latest models, we will get a similar result.
reply