> From what I understand, current models seem to have issue with planning and reasoning because they “blurt” everything out zero shot. They can’t recurrently process information.
IMO, this is not necessarily a model issue, but an interface issue - recurrently processing information is enabled by just feeding the model's output back to it, perhaps many times, before surfacing it to the user. Right now, with the interfaces we're exposed to - direct evaluation, perhaps wrapped in a chatbot UI - the user must supply the recursion themselves.
> But it’s not innate yet, where models can “think” about a complex problem until it solved it.
That's indeed the part I see us struggling with - we can make the model recursive, but we don't know how to do it unsupervised, so we can box it behind an abstraction layer, letting it work stuff out on its own and break recursion at the right moment.
I think the mental model I developed a good year ago still stands: LLM isn't best compared to a human mind, but to a human's inner voice. The bit that "blurts everything out zero-shot" into your consciousness. It has the same issues of overconfidence, hallucinations, and needing to be recursively fed back to itself.
IMO, this is not necessarily a model issue, but an interface issue - recurrently processing information is enabled by just feeding the model's output back to it, perhaps many times, before surfacing it to the user. Right now, with the interfaces we're exposed to - direct evaluation, perhaps wrapped in a chatbot UI - the user must supply the recursion themselves.
> But it’s not innate yet, where models can “think” about a complex problem until it solved it.
That's indeed the part I see us struggling with - we can make the model recursive, but we don't know how to do it unsupervised, so we can box it behind an abstraction layer, letting it work stuff out on its own and break recursion at the right moment.
I think the mental model I developed a good year ago still stands: LLM isn't best compared to a human mind, but to a human's inner voice. The bit that "blurts everything out zero-shot" into your consciousness. It has the same issues of overconfidence, hallucinations, and needing to be recursively fed back to itself.