Another for the pile: "It's your fault for not using tomorrow's model, which everyone says is better."

every single conversation i have about LLMs ends up here

Sounds like first decade or two of aviation, back when pilots were mostly looking at gauges and tweaking knobs to keep the engine running, and flying the plane was more of an afterthought.

Sounds like spiritualism and ghost-hunting, such as the excuses made on behalf of the Cock Lane ghost in the 1760s.

When nothing happened, Moore told the group the ghost would not come as they were making too much noise. He asked them to leave the room ...

when a clergyman used a candle to look under the bed, the ghost "refused" to answer, Frazer claiming "she [the ghost] loving not light".

Are we seriously arguing this in 2025?

Go to ChatGPT.com and summon a ghost. It's real. It's not a particularly smart ghost, but gets a lot of useful work done. Try it with simpler tasks, to reduce the chances of holding it wrong.

That list of "things LLM apologists say" upthread? That's applicable when you try to make the ghost do work that's closer to the limits of its current capabilities.

>current capabilities

The capabilities of LLMs have been qualitatively the same since the first ChatGPT. This is _precisely_ a hype post claiming that a future where LLMs have superhuman capabilities is inevitable.

Are you truly saying that the qualitative capabilities of LLMs haven't changed since GPT3.5?! If so, then you are objectively wrong, hype or no hype.

They've definitely improved in many areas. And not just the easily-gamed public metrics; I've got a few private tests of my own, asking them certain questions to see how they respond, and even on the questions where all versions make mistakes in their answers, they make fewer mistakes than they used to.

I can also see this live, as I'm on a free plan and currently using ChatGPT heavily, and I can watch the answers degrade as I burn through the free allowance of high-tier models and end up on the cheap models.

Now, don't get me wrong, I won't rank even the good models higher than a recent graduate, but that's in comparison to ChatGPT-3.5's responses feeling more like those of a first or second year university student.

And likewise with the economics of them, I think we're in a period where you have to multiply training costs to get incremental performance gains, so there's an investment bubble and it will burst. I don't think the current approach will get in-general-superhuman skills, because it will cost too much to get there. Specific superhuman skills AI in general already demonstrate, but the more general models are mostly only superhuman by being "fresh grad" at a very broad range of things, if any LLM is superhuman at even one skill then I've missed the news.

I think you just wrote the "LLM maximalists manifest"