Is there any indication that current methods could lead to a model that generates text as if it had an IQ of 200? These are trained on texts written by humans who are, quite overwhelmingly, much lower in IQ than 200. Where's the research on developing models that don't just produce better or faster facsimiles of broadly average-IQ text?
Think a little bit deeper about what it means to be able to predict the next token. Think about what a predictor has to do in order to do this extremely accurately across a very large corpus of text.
There is a big difference between being able to predict what a median human might write next, and being able to predict, in all cases, what the particular human author of a particular passage will write next.
Or from another angle: the human authors of training data may have made errors when writing the data. The token predictor may learn to correctly predict those errors. These are not the same thing!
I'm sorry, I'm not sure I grasp the salience here to super-intelligence. The model may be able to predict accurately what any particular human will write, but profoundly intelligent humans will be quite rare in the training data, and even those humans don't approach what people seem to mean when they talk about super-intelligence. Perhaps I'm missing your point.
Superintelligent models need not be LLMs. They could work similar to animals, which predict future experiences, not text (predictive coding). There is no LLM-like human bound in predicting reality.
That may be true, but I can't speak to any research being conducted in that area. My point is that the hype around dangers of super-intelligence seems to have been spurred by improvements to large language models, even though large language models don't seem (to me) a suitable way to develop something with super-intelligence.
It's more that the general pace of innovation has sped up. Three years ago something like ChatGPT would have similarly been dismissed as science fiction. So we probably shouldn't dismiss the possibility that we will have something far better than LLMs in another three years.
