Could anyone frame -- in fairly plain words -- what would be the mechanism by which LLMs become generally "smarter than humans" in the "and humans can't control them" sense?
Has there been some advance in self-learning or self-training? Is there some way to make them independent of human data and human curation of said data? And so on.
1. LLMs are already doing much more complex and useful things than most people thought possible even in the foreseeable future.
2. They are also showing emergent behaviors that their own creators can’t explain nor really control.
3. People and corporations and governments everywhere are trying whatever they can think of to accelerate this.
4. Therefore it makes sense to worry about newly powerful systems with scary emergent behaviors precisely because we do not know the mechanism.
Maybe it’s all an overreaction and ChatGPT 5 will be the end of the line, but I doubt it. There’s just too much disruption, profit, and havoc possible, humans will find a way to make it better/worse.
I follow but that looks like a weak (presumptive) inductive argument to me. Could it be that Hinton is convinced by an argument like that? I would have expected something more technically specific.
I am not convinced that an AI has to be smarter than humans for us to lose control of it. I would argue that it simply needs to be capable of meaningful actions without human input and it needs to be opaque, as in it operates as a black box.
Both of those characteristics apply to some degree to Auto-GPT, even though it does try to explain what it is doing. Surely ChaosGPT would omit the truth or lie about its actions. How do we know it didn’t mine some Bitcoin and self-replicate to the cloud already, unbeknownst to its own creator? That is well within its capabilities and it doesn’t need to be superhuman intelligent or self-aware to do so.
Has there been some advance in self-learning or self-training? Is there some way to make them independent of human data and human curation of said data? And so on.