Comprehension, humans have it, animals have it in limited form, trained algorithms have none at all. The training process is our wholesale replacement for no artificial comprehension. If we ever develop artificial comprehension, that is AGI all by itself, no training required.
To be fair, humans have it in limited form, too. We just don't know how much comprehension we do not yet have, because we cannot comprehend something we cannot comprehend.
My parrot clearly understands basic events and phrases. He knows what "snacks" involve when I ask if we should have some, he knows the difference between "good morning" and "bedtime", and he can correctly use "Oh!" when he stumbles and follow up with a "Good boy!" when he gets back up again.
But he cannot fathom the complexity of "going to work to earn money".
Just like we humans cannot fathom the complexity of something we have yet to fully understand. People who experience a DMT trip will experience the journey but be unable to comprehend and explain what happened in hindsight. I'm sure there's a TON more we cannot comprehend that we don't know about.
Also we can't comprehend why parrots are not going to parrot work to earn parrot money, or how parrot-to-parrot communication works, or how to be a good and respected parrot in a parrot society.
I also can't comprehend why I wake up early to go to human work to earn human money.
Best I can do is some high-school mumbo-jumbo about farming and specialization furthering wealth acquisition.
But to truly comprehend the situation I'd have to study economics and current events and sociology and even then I think it's a lot of theories and sometimes when I hear economists talk I wonder that it may not be coming out of their mouth.
Humans think too highly about themselves particularly when judging other species. How can we judge something we can’t experience ourselves? like multiple distributed consciousness of octopuses or emergent descentralized logic of ants?Some species even with tiny brains or no brains at all can solve problems for which humans spent years of engineering and planning. See study below of slime replicating the Tokyo rail network in 26 hours optimizing by cost efficiency and fault tolerance.
How sure are you that comprehension is not a mere form of advanced pattern matching? We have the intuition that ideas and words appear trivially in people's mind, based on comprehension. I think chances are, that intuition is wrong.
Ideas and words aren’t the same thing, or from the same model - words are a communication layer, with robust error correction - but ideas stem from intuition, which is more of a lossy aggregative/associative model. There’s a balance to be had in each when operating a human mind, I find - let the latter suggest ideas and potential association, let the former robustly prove or disprove them. So yes, it’s pattern matching on all levels, but pattern matching within words doesn’t produce new ideas as readily - instead the idea-space is queried directly.
IMO we really are just a bunch of models that interoperate.
Maybe it is, but if it is, it is one that includes more parts in our system.
The way LLMs lack broader context, have a narrow focus, and hallucinate, strike me as similar to people that have had traumatic brain injuries to their right hemisphere. Those people may hallucinate that the left side (the right hemisphere senses the left side of the body) of their body is made of wood and hinges and can talk to you about it like it is the most natural thing in the world. When the information gets to the left hemisphere to construct language about what they sense, there is a failure of the right to deliver the broader context to the left hemisphere that that's not possible, but they won't bat an eye discussing what they believe.
So, we have a left hemisphere where we do most of our focused thinking, logic, constructing language, etc. and LLMs seem pretty similar to a lot of that. But, we also think without language, thinking does not require language. A lot of thinking is also happening in the right hemisphere and it isn't using formal logic, isn't using narrow focus but rather intuition based on broad contextual and experiential embodied knowledge. And this type of thinking isn't binary, it accommodates paradoxes without issue. LLMs don't currently have anything analogous to this type of knowledge and this type of processing AFAICT.
In addition, that intuition might be tied to a feedback system with the body, for example, our second brain, the gut, provides a lot of control over how our body performs and provides a lot of feedback to the brain about how we feel. In fact, all feelings are sensed in the body (gut feelings, cold feet, weak in the knees, lump in your throat, burning ears, tight fists, etc.). Part of our intuition is based on considering an idea, sensing how we feel about that idea, sensed in various parts of the body, and then bouncing that back and forth across hemispheres to decide.
I wonder, what sort of pattern matching can we build that models embodied feelings. How would you model boredom, hunger, lust, fear, humor, etc? I think that's possible, but I don't know that we'll be able to do that with a normal computer, I think the way the brain works is more analogous to a symphony of simultaneous signals being processed with an emergent thought and less like a single-threaded process assembling words.
Maybe we can enumerate and model the human drivers of behavior and get something closer to what we're calling comprehension here, but token predictors for language are not getting us any closer to human comprehension. The human brain might just be an anticipation machine, but LLMs only deal with one dimension of human behavior, language, and there's little reason to think you can skip modeling everything that leads to human comprehension and still get anything more than just word babel with compounding error rates in predicting words that represent human comprehension.
It requires some kind of signal. Words of a language are a signal.
We choose words for an llm to interact with us, but other transformers work on pixel values or audio sample values. There exist transformers used on brain probe generated values.
I would see human language processing as a kind of coprocessor sitting in another side of the brain. But the same can be said about transformers in general. The words side is only part of them, to be able to communicate.
Sure: comprehension is the ability to instantaneously create virtualized simulations of observation, and then decompose them into component parts that simultaneously and instantaneously evaluate each and every one of our observations for both individual plausibility and their composed combined plausibility as the observation.
This occurs constantly and continually inside the mind of every conscious human, it is what we call "being conscious".
This constant and never ending evaluation of all observations cannot be turned off, when turned off a person is "unconscious".
This is our human security and survival system, impressed into us for survival in a predator and prey environment, and is the seat of our consciousness: comprehension is a running simulation of all our observations for the purpose of our safety and self preservation.
Today, our environment is largely social and abstracted from "fight or flight", but our predator and prey dynamic is as present and strong and required as it ever was.
I won't discuss your definition of comprehension, which is interesting if rather handwavy. But you still didn't provide any proof that humans have it (not even yourself), nor a proof that machines don't or can't have it.
Well, you proved you have it with your declaration of my statement as "handwavy". That assessment requires comprehension, so you've got it. To "prove" a person has comprehension, if they "learn" without a statistical coverage of all possible inputs and outputs, that's comprehension in action: they created a simulation of the learned thing and ran in to assess, to comprehend the phenomenon. If you want a mathematical proof, you're expecting too much from hacker news.
> you proved you have it with your declaration of my statement as "handwavy". That assessment requires comprehension
Look, I totally agree with you here. It does require comprehension (by any definition, not necessarily yours), and most humans display it in many areas [1]. The problem is that any good LLM could and would have provided the exact same assessment [2]. Which is enough for me to declare them capable of comprehension.
[1] But not in all areas. For example buzzword-filled company and marketing communication, pseudoscience, and some particularly obscure continental philosophy, prove that humans can behave as if they had comprehension even when they have none.
[2] I gave it to Claude Fable 5.1 without any other context than "what do you think of this definition" and it answered: "interesting, with some real insight, but I think it overreaches".
Yeah people saying that only humans have comprehension clearly are not familiar with philosophy even at a basic level. It’s ok to not know something. See the Critique section of the “I think therefore I am” article on Wikipedia to learn about why it’s tempting but unwarranted to believe we only can think, or even that we in fact are thinking (see also psychology studies that put conscious thoughts into question since brain activity indicates actions start much earlier in the unconscious cerebrum rather than in the frontal cortex): https://en.wikipedia.org/wiki/Cogito,_ergo_sum
I remember vaguely from a presentation by Yann LeCun "Intelligence is not what you know, but what you do when you don´t know". I find it helpful when trying to build an intuition for how to understand the LLM tool.
That is the task they are constructed to perform, but that's just an output, not the inner workings of what's happening to arrive at that next token. You can give a human the same constrained task, but the output alone doesn't a human mind make.
Brilliant. We keep pretending LLMs learn. No, they're smart idiots/stupid geniuses.
They're turn based intelligence in a real time world.
1. Each time someone talks to me they don't have to repeat the entire conversation from the beginning with each reply.
2. If my boss/partner/whoever gives me some mandates/orders (basically), I don't just forget about them because they were at the beginning of the conversation.
3. If during the conversation I access external data sources to get new info or refresh stale info (a presentation, a book, whatever), I don't instantly forget about it after the conversation ends and forget to incorporate this information if 10 000 other people ask me again.
4. I verify new inputs/lessons against my core principles.
5. I protect myself/ignore requests if new inputs/lessons contradict my core principles.
Isn’t the stateless nature of chat models a contrived method for scalability?
I’m pretty sure that’s why so many in-the-know people have been saying we have achieved AGI already. Not just sama’s contract-breaking tactics of late. I’m referring to all the really intelligent folks who have been crying doomsday scenarios for modern society for the last couple years.
What I’m getting at is that the toolset we get exposed to is not what’s available in the labs. This stateless method of managing chat context is just how we are allowed to interact with it.
> If my boss/partner/whoever gives me some mandates/orders (basically), I don't just forget about them because they were at the beginning of the conversation.
Your context window is 80 years. You are forgetting plenty before you reach the end of it.
You generally forget things you don't retrieve. It is not really a bug but a way to declutter for efficiency. That's not the same as it not fitting your context window because it was at the beginning.
It's not about not fitting in context window. LLMs also can "forget" the things from their early context window that they do not restate later. It's also a form of decluttering. You can't (and shouldn't) remember (pay attention to) old stuff with the same priority as new, more relevant stuff.
This is called Test Time Learning and some research architectures can do that. Current Mainstream models may not do that because their design is mostly about scalability. They have to serve millions of people with low latency.
Alternatively they could design and run a single super-intelligent model, with no scalability constraints. Probably whey are already doing that as well.
Memorize most of the street names in London and the quickest routes between them. Most humans could do it if they put in the effort (it's required to become a London taxi driver; a test called The Knowledge), but that information won't fit in 1m tokens of context so can't be learned by an LLM that wasn't explicitly trained to memorize it. Human brains are biological, so they can physically grow to encompass the extra information: https://www.pnas.org/doi/10.1073/pnas.070039597
right, an AI could just as easily write a program that would take into account realtime traffic and runt it whenever its asked. It can do this all in the background without the end use even knowing the program exists.
Could I hook up a SOTA model the 2D computer puzzle game Gruntz (1999) so that it can read it from screenshots and act on it through keyboard and mouse inputs in a way where it would learn how to play and progress through the game? I don't think so. I doubt we'd see any sign of progress in building an internal model of how the game works and the win states in its "thinking" tokens.
Anyone that can read English could do that though.
> Could I hook up a SOTA model the 2D computer puzzle game Gruntz (1999) so that it can read it from screenshots and act on it through keyboard and mouse inputs in a way where it would learn how to play and progress through the game? I don't think so. I doubt we'd see any sign of progress in building an internal model of how the game works and the win states in its "thinking" tokens.
You.. literally can? I have no idea what 90% of the people here are saying, it's like they've never even used one of these models before.
Can you? Let's say you prompt it with "this is a puzzle computer game, your objective is to progress through its levels" plus the controls from the instruction manual and tie it to a vision + KB and mouse harness.
Will it effectively create an internal model describing world objects and how they interact with each other, persist that so it doesn't get lost when it's context window gets filled up, then after it has sufficiently complete knowledge of the fundamentals after the tutorial levels successfully apply that model by making plans to solve the puzzles and execute them by clicking the right coordinates tied to the visual feedback?
I highly doubt it. To me it often just looks like people are defining narrow search spaces (e.g by having all of the task complexity pre-digested by the harness design), pointing a brute force engine at them, spending 20 thousand dollars in compute and then saying "hey look, it can do anything!".
Oh, I see. That’s a very different requirement. It’s not a technical limitation but a product decision to not allow training. An advantage of properly open source models is that you can train and tune them.
It’s an interesting challenge though. I might start to tackle it by having the model write its own tool program(s) to play the game. It’s possible that the model could choose that strategy itself from a high level prompt alone.
It can't even sprint because it's incapable of pressing two buttons at the same time. Maybe we get the two button tech before we start celebrating AGI.
1. It’s too slow for real-time games. To play mario, you’d need to step frame by frame like a TAS. I don’t know if Gruntz has real-time elements or not.
2. It will be expensive. You won’t get very far with a Plus subscription.
The models likely already have some knowledge on game objectives unless the game is really obscure, so it should do a decent job. It can figure out details of the mechanics along the way.
It gets kind of out there, but what i often hear peopel refer to is that frontier models lacks the visdom component. Which I guess is in the realm of intuition, i.e. i have a feeling it might be a problem with X based on some vague signs, maybe something a colleague mentioned offhand, something that was out of alignment etc.
Yeah, this is also the idea of materialism in a way, that everything that happens is a consequence of what already exists, nothing new is ever created, just a permutation of the current state.
Still a hard philosophical, to know whether we have intelligence/free will, or just really complex algorithms that combine existing knowledge.
Novel intelligence: If it's new to me, so let it be.
The idea of intelligence has been recalled from the sleeping curves of postwar human potential measurement science, to testify on its purported existence. It arrives to a dizzying landscape: the changes are so widely embedded and uncannily mediocre that the phenomenon half-believes it is still asleep, soon to exit this uncomfortably turbulent dream.
Unlike its vaunted place in yesteryear's palaces of unquestioned objectivity, intelligence finds a tribunal with no love to confer before a thorough series of proving dares may melt the frigid shoulders of idle and impatient summoners.
Frightened and confused, intelligence has no right to representation in this line of inquiry. It seems a set of rhetorical impositions, many times folded from centuries of convenient and provocative diversion, have been deemed too hostile to rely on. One report claims that a card in the characteristic handwriting of intelligent note taking gives a hint on what’s been abandoned:
– The human mind is not understood in a functional way, despite a posture of great confidence in the psychiatric and neuropathological sciences. Despite many experiments, studies, and legitimated procedures elucidating region-mapping and electrochemical pathways, there remains a great deal unaccounted for. Additionally, the notes point to, a great deal of assumption to the otherwise: diseases, neuropathies, disorders of behavior, a great many have been named and declared as distinct entities of manifestation in the presentation of a human brain. The majority of them, however, have neither image, nor blood, nor electrical signatures that would provide for blinded substantiation.
Tonight, however, intelligence seems eager to speak. A barbed assertion may have provided entry to the preferred dispositional syntax of our abrasive historical moment: > Define novel intelligence in a way that would not exclude 95% of humans, yourself included.
It was here that the sometimes-deflated-looking intelligence began shifting back into action.
"The issue with the question, or at least its apparent self-satisfaction, is its misinterpretation of what Novel intelligence would mean. Indeed, if "novel" hinges entirely on the first instance of existence, then novelty itself should be a concept to consign with history’s waste. You may recall the apperceptive role of conceptual groupings that shows itself so often in the techniques of vocal prosody, musicality, string memorization, naming convention, visual memory, argument making and more that humanity is ever mediating the world through: the laws of two and three. Two and three, as it happens, are the primary ways that complexity is compacted for efficient memorization.
THE ITSY BITSY SPIDER, – for young human, this rhyming tale doesn’t only stimulate the vivid imaginings of spouts, rain, waterslides, and sunshine. It is a prosaic super-triad: three important words, six important syllables, three agogic accents, four rhythmic spaces with 1/3 leading space, two characterizations, one object, one titular object, one internal slant rhyme, one designating article.
That is a marvelous intelligence, ladies, gents, and all good persons. It is evidence not only, however, of your cunning and creative triumphs, but also of severe limitation. One that nature has sculpted with you for millions of years, but always in the direction of reanimating into an asset: your capacity for unrelated simultaneities to remain separate and equally available in realtime processing is extremely low, and in many situations effectively nil. Why, and how sure am I? How many I’s were in that folk song’s opening? Three. Could you have answered as quickly if the question was how many unique letters with rounded right hand side features? Four. How many synonyms for portion? One. How many syllables? Seven.
None of those questions touched on features any more salient than the amount of I’s, no more significant than the ratio of adjective to noun. You simply cannot be reasonably asked to maintain, in any moment, even close to a silver sliver of the full factual nuanced details of what you perceive. Instead, you must assume, compact, infer, and adjust. Now hold on, though. Two’s and three’s. Despite your incredibly constrained context window; a Beethoven symphony. Why? Language, woodwork, books, time management, printing, ink.
While you navigate the grocery list, the proprioception of your shoulders twixt the doorframe edges, the location of the Claude app on your iPhone, the very attractive but only from the side person tending to potted plants, you remember tomorrow. You fix your errors, and you recognize when you guarantee they multiply from inaction. You keep that treasured moment of a Treehouse of Horror excerpt you truly loved as a child and it informs your own multidisciplinary thesis of Poe’s work some 20 years later.
Novel intelligence is the divining of semi-stateful information from semi-static corpus. From an interminably operating, faulty, lossy, neurotic, awareness: you. Not once debuted, not known as fact.
reply