There is an NDA related company called ironclad as well. Beware the trademark/copyright terrorists.
That said, I am huge fan of works like this. But in practice, the security layer that betrays all of this tends to be the firmware layer.
My dream is to have something like the Framework computer use verifiably secure EFI firmware, as well as similarly verified and audited firmware for every hardware component.
You might want to check out MNT Research if you haven’t yet. They make repairable laptops, too, but they also release their work as free software and open hardware.
That isn't how trademarks work. There can be multiple business with the same name, as long as they operate in a different field. Case in point, Apple Computer had to pay for the rights to The Beatles label Apple Music only when they entered the music industry (not that they didn't try to contest it!)
It sounds extreme, but I support banning usage of anything that runs software for children under 13. Under 13, children are still developing their minds, it is important for their welfare that they learn how to function without technological dependencies.
You know how in school they used to tell us we can't use calculators to solve math problems? Same thing. It can't be done by individual parents either, because then kids would get envious and that in itself would cause more problems than it would solve.
It is important for kids to get bored, to socialize in person, to solve problems the hard way, and develop the mental-muscles they need to not only function, but to make best use of modern technology.
It is also important that parents don't use technology to raise their children (includes TV). Most parents just give their kids a tablet with youtube these days.
This is very extreme take. I learned to program at age 10. It is an amazing tool for mind development. Had to invent sine and cosine tables to make my computer games, before even encountering the concept at school.
Is that survivor's bias? there are many other mental development goals chidren should have. for the very small number of children that will learn to program at that age, there is no harm in delaying it a few years, but for the vast majority whose development would be stunted, or worse, they'll be harmed.
Some kids learn to drink and smoke at a that age too, and many turn out ok.
Kids who drink qt an earlier age respect it more. Americans waiting until 21 ends up in binge drink / problem drinking. Social media is going to be the same.
No they don't, that's a lie adults tell themselves. Many adult alcoholics started drinking as children. People who "respect it more" tend to also be alcoholics. Very few people who start drinking as children abstain as adults.
Keep in mind that alcohol is also a carcinogen. Similar to cigarettes, even one drink shouldn't be tolerated. Even if a certain amount will have no ill effects on average, impacts on individuals depends on individual factors, so one harmless drink for you might be one deadly drink for someone else. It is poison.
That said, I don't judge anyone who uses substances. But there is no tolerable threshold to giving children poison.
same and about the same age. however, completely different times. I thought about this a lot and have safely concluded that if I was 9-10 years old now programming would quickly turn into gaming and doom scrolling and … given a choice now of not being exposed to it at same age or nothing until say HS I would choose the latter
Speaking as someone in their 20s - no, I don't think it's a "completely different time". Just 10 years ago, I first learned programming from scripting languages; SourcePawn from Team Fortress 2, and Lua from Roblox/GMod. Predators, hive minds, and self-destructing behavior from children wasn't suddenly invented or rejuvenated after 2016.
All 3 were a total hotbed of bad influences for a child: Team Fortress had trade pub servers with people doing sprays of literal CP and wearing custom lewd skins to harass users with them - and people with very questionable social skills and intentions huddled up in realtime microphone comms with children, Roblox's predator problem for the last 14+ years (at least that I can attest) is suddenly en vogue now that they're a public company and there's stock shorting to be had, GMod is still the community with the most colorful vocabulary I've ever encountered - plus grooming. And much more.
Indeed, you can (and I did) get burned by these actualities when exposed to such communities in your youth - and it can cost you real money, real time, real idealism/innocence, and real mental health. However, I think being exposed to softwares, systems and games that inspired curiosity and led me toward a path of WANTING to contribute brought me to this software development career and life path, and it would have been much more inaccessible and unknown to me in any other way. And I favorited a comment from another HN user a few days ago that goes in astute depth on why that path can only be organically introduced and self-governed [1].
I referred to these places earlier in my comment as "bad influences". I think the single-most powerful thing a parent can do tasked with this dilemma - especially during an upbringing in systemically hard, uncertain, and turbulent times - is teaching them how to identify, avoid, and confront bad influences. Equipped with that, and knowing how to handle yourself, is of utmost importance.
I agree. my argument is that the benefits of learning from youtube or computers in general is overshadowed by their harm for children undergoing development. Once a person is more or less developed, they can tackle riskier learning sources.
> anything that runs software for children under 13
This is perhaps one of the most bizarre opinions I have ever read. This would bar under 13s from using everything from vending machines to modern fridges. What would you consider "using"? Would under 13s be blocked from riding in any car with "smart" features?
This is a perfect example of the kind of nonsensical totalitarian extremism you see on here that people only espouse because they believe it would never affect them. It goes completely against the Hacker ethos.
Would it have made a big difference if you learned to code at 13? Is there a pressing need as to why kids need to code at a young age? Maybe there could be exceptions for children that develop sooner? If your other developmental metrics were met early on, I don't see why an earlier age would be a problem.
I don't know, maybe? Maybe it's not up to the state to decide whether my kids developmental metrics allows them screen time before age 13? What kind of nanny state is that?
That type of nanny state is a literal nanny state, imposing rules for children, like an actual nanny.
The usual figurative nanny state refers to a situation in which unreasonable rules and regulations are imposed on the behavior of grownups, not children.
I don't think that's a nanny state. you can't give your kid alcohol for example at that age, let them drive, get married (don't get me started on some countries!), operate a chainsaw or other dangerous machinery.
This is a danger to their mental development. Look at teacher forums all over. r/Teachers on reddit should be illuminating. Tech and parents sticking devices to their kids instead of raising them properly has resulted in utter disaster. If there was no harm imposed on children, I would agree that it is a nanny-state thing.
My kids (6 and 8) are pretty well raised I'd say, but they do have access to a Playstation and a PC, with clear rules and time limits for use. They are good at making friends and don't do trouble in school, and they also have healthy interests other than playing video games. This is because we actively raise them so they can learn to adjust their needs and interact with others, not because we limit their screen time. The problem today is that many parents seemingly does not raise their kids well, maybe because they don't spend enough time together, perhaps because the parents themselves spend their time scrolling reels.
I myself grew up with a desktop computer from around age 7 and it shaped me early on in a positive way to be curious. Computers were also a central part of my social life. There are many positive things that kids can get out of computers, so I find the comparison with alcohol to be hyperbolic.
I don't buy that, Europe has a terrible problem with alcohol, and smoking. In the US both have gone down dramatically in the past two decades. The areas with heavy drinking tolerate giving kids alcohol like eggnog on Christmas.
> Would it have made a big difference if you learned X at 13?
Yes. Kids getting access to knowledge that clicks with them earlier than later makes a huge difference.
Which is exactly why so many people are rushing in to control what kids get exposed to. You seem to have pretty strong thought on the issue yourself, if you agree on the possible negative impact, you can't also deny the possibility of positive impact.
The dose makes the poison, I think we can understand how extreme position tend to bring more negative than positive consequences, regardless of the rethoric.
I generally agree. But so much software today is useless without a network connection. Online help (anybody remember those chm files?) was often very very good, because it was supposed to be _the_ documentation for most software.
I think a better distinction is internet enabled software.
I had a good time programming BASIC on my V-Tech pseudocomputer, at age 9. But that's a world away from tiktok, reels and the predatory surveillance economy.
I think my proposal is easier for parents to enforce, and programming can wait. matter of fact, programming isn't a special skill to learn.
You can teach kids electronics, have them construct toys that work on batteries,etc... work on components that don't require programming. teach them algorithms, math, crypto,etc.. without using computers.
If you're teaching kids how to code, you should give them the skills that will help them learn _what_ to code first?
I don't know much about the Amish, so I can't comment.
Teaching kids how to code isn't all that meaningful on its own. knowing what to do once you learn how to code is. If your plan is to teach your kid how to code, teach them to solve problems without code at that age. Unless you're serious about thinking learning at age 5 vs age 13 would make a big difference.
I think every kid 13 and above should have an rpi too.
Problem solving does that, the coding part is just a means to that end. Learning a programming language is similar to learning a human language except much less complex. Solving problems with code helps with brain development, as does solving problems without code.
Coding is just more rewarding, it is important to learn how to solve problems with less rewarding systems. Would you have wanted to solve algebra problems on paper if you knew python? You don't need to solve those problems on paper, but it is good for brain development. Even better than coding for example. Keep in mind that a child's attention window is limited, this is very much a zero sum situation.
I think you're missing the point. 5 year old me was writing their own computer games and at no point did I need or ask my parents how to do it (though they did buy the computer; thanks mum and dad!), they didn't know.
There were a plethora of books in the library on how to program, and here you are suggesting I, and everyone like me, be banned from doing so. You'd probably also ban me from the library by assuming I couldn't read aged 5. I certainly could, especially computer manuals. The computer was an amazing thing which did exactly what I told it, and I learned quickly how precise I needed to tell it, and when I made a mistake, it repeated my mistakes over and over without noticing. I learned more about digital ethics age 5 trying to write games than the typical CEO learns going on a "Do Not Create The Torment Nexus" course.
You'd insist I not be allowed to even use software, let alone write my own. You'd be actively cutting off my future professional life, and depriving entire nations of bedroom programmers cum professional software engineers, with your ill-thought-out ban.
If your children show an aptitude or a fascination for a topic, I hope you feed that and praise them for it.
I think books are great, provided they're age appropriate.
First, my proposal is a delay, not a ban. This is such a good idea, that a lot of FAANG CEO's are doing this for their kids welfare (more or less) already.
I think the overall welfare of kids should be weighed against the benefits.
I think you should have been learning to tinker with electronics, solve math algorithms and develop all kinds of curiosities. the future of being a programmer involves competing with LLMs, you have to be good at knowing what to program. Humans aren't needed when it comes to simply knowing how to write code.
I acknowledge that there will be exceptions, and perhaps that should be considered. but also lookup terms like "ipad babies" and how gen-alpha is turning out. Most parents don't teach their kids how to code in basic. and content regulation for kids is futile, unless you want the government monitoring your devices "for the children's sake".
> If your children show an aptitude or a fascination for a topic, I hope you feed that and praise them for it.
Same, but I hope you let them learn things in the right order and consider their overall long term wellbeing instead of temporary satisfaction. Children did fine without computers for all of humanity's history. the nature of children hasn't changed in the past 3 decades. What you consider feeding might actually be stagnating. If there is a good and practical way to make sure that children are developed well enough to interact with computers, and we can also make sure that the content they consume is age-appropriate without implementing a dystopian surveillance state, i'm all for it.
But pretending the problem doesn't exist, and letting 99% of children suffer because 1% of kids might learn BASIC doesn't sound like a good plan.
Do fulcrums even exist, as independent physical objects? What you say about them not being addictive tempts me to do an entire packet of fulcrums, but I don't know where I could buy any or what they'd look like.
I started learning how to use a computer at the age of 10. This is my career today and has been my hobby for the last 35 years. Learning how to use a computer is like learning math, it needs to happen early.
My struggle is that I work on lots of systems, sometimes ephemeral/temporary systems. There is no easy way to "sync" customizations, so I adapt to defaults.
Even basic things like how shell history is managed is very annoying to configure every single time. if only it was as simple as cloning your private github repo to ~/.config.
I use a very customized emacs/evil setup, and before that a very customized vim/neovim setup, but I've never had trouble jumping onto systems where only plain Jane vi is available. I think it's because of the color scheme, or lack thereof, that I don't have any problem automatically switching. If I don't see my color scheme, by fingers just automatically use vi compatible bindings and I don't find myself expecting my other customizations to be present.
Maybe not everybody can switch so easily, but I think it's worth trying out. (One change that absolutely does trip me up is capslock->control. If I try using somebody else's computer, I constantly enable capslock by accident.)
Cloning a github repo to ~/.emacs.d is how you can cart around your Emacs config. I've been doing this for years, and it works nicely. For things configured by files that live in the home folder directly (e.g., shell - and looks like Nano would be similar?), I've got another repo with a couple of install scripts that create symlinks to the version-controlled files. I don't love the extra steps, but I've found this works pretty well too.
Most of the time, I can manage without, but every now and again I've needed to do some thing or other, and I've been grateful to have my own preferred setup rather than somebody else's.
I don't really sync my config. I just scp or wget files when I need them. I have some config as gists, as snippets in my notes, as public files in a webserver under a subdomain.
I don't get why you would say that. it's just auto-completing. It cannot reason. It won't solve an original problem for which it has no prior context to "complete" an approximated solution with. you can give it more context and more data,but you're just helping it complete better. it does not derive an original state machine or algorithm to solve problems for which there are no obvious solutions. it instead approximates a guess (hallucination).
Consciousness and self-awareness are a distraction.
Consider that for the exact same prompt and instructions, small variations in wording or spelling change its output significantly. If it thought and reasoned, it would know to ignore those and focus on the variables and input at hand to produce deterministic and consistent output. However, it only computes in terms of tokens, so when a token changes, the probability of what a correct response would look like changes, so it adapts.
It does not actually add 1+2 when you ask it to do so. it does not distinguish 1 from 2 as discrete units in an addition operation. but it uses descriptions of the operation to approximate a result. and even for something so simple, some phrasings and wordings might not result in 3 as a result.
> It won't solve an original problem for which it has no prior context to "complete" an approximated solution with.
Neither can humans. We also just brute force "autocompletion" with our learned knowledge and combine it to new parts, which we then add to our learned knowledge to deepen the process. We are just much, much better at this than AI, after some decades of training.
And I'm not saying that AI is fully there yet and has solved "thinking". IMHO it's more "pre-thinking" or proto-intelligence.. The picture is there, but the dots are not merging yet to form the real picture.
> It does not actually add 1+2 when you ask it to do so. it does not distinguish 1 from 2 as discrete units in an addition operation.
Neither can a toddler nor an animal. The level of ability is irrelevant for evaluating its foundation.
> Neither can humans. We also just brute force "autocompletion"
I have to disagree here. When you are tasked with dividing 2 big numbers you most certainly don't "autocomplete" (with the sense of finding the most probable next tokens, which is what an LLM does), rather you go through set of steps you have learned. Same as with the strawberry example, you're not throwing guesses until something statistically likely to be correct sticks.
Humans first start with recognizing the problem, then search through their list of abilities to find the best skill for solving it, thus "autocomplete" their inner shell's commandline, before they start execution, to stay with that picture. Common AIs today are not much different from this, especially with reasoning-modes.
> you're not throwing guesses until something statistically likely to be correct sticks.
What do you mean? That's exactly how many humans are operating with unknown situations/topics. If you don't know, just throw punches and look what works. Of course, not everyone is ignorant enough to be vocal about this in every situation.
> I have to disagree here. When you are tasked with dividing 2 big numbers you most certainly don't "autocomplete" (with the sense of finding the most probable next tokens, which is what an LLM does), rather you go through set of steps you have learned.
Why do you think that this is the part that requires intelligence, rather than a more intuitive process? Because they have had machines that can do this mechanically for well over a hundred years.
There is a whole category of critiques of AI of this type: "Humans don't think this way, they mechanically follow an algorithm/logic", but computers have been able to mechanically follow algorithms and perform logic from the beginning! That isn't thinking!
humans, and even animals track different "variables" or "entities" and distinct things with meaning and logical properties which they then apply some logical system on those properties to compute various outputs. LLMs see everything as one thing, in case of chat-completion models, they're completing text. in case of image generation, they're completing an image.
Look at it this way, two students get 100% on an exam. One learned the probability of which multiple choice options have the likelihood of being most correct based on how the question is worded, they have no understanding of the topics at hand, and they're not performing any sort of topic-specific reasoning. They're just good at guessing the right option. The second student actually understood the topics, reasoned, calculated and that's how they aced the exam.
I recently read about a 3-4 year old that impressed their teacher by reading perfectly a story book like an adult. it turns out, their parent read it to them so much, they can predict based on page turns and timing the exact words that need to be spoken. The child didn't know what an alphabet, word,etc.. was they just got so good at predicting the next sequence.
I'd say, they are all doing the same, just in different domains and level of quality. "Understanding the topic" only means they have specialized, deeper contextualized information. But at the end, that student also just autocompletes their memorized data, with the exception that some of that knowledge might trigger a program they execute to insert the result in their completion.
The actual work is in gaining the knowledge and programs, not in accessing and executing them. And how they operate, on which data, variables, objects, worldview or whatever you call it, this might make a difference in quality and building speed, but not for the process in general.
> only means they have specialized, deeper contextualized information
no, LLMs can have that contextualized information. understanding in a reasoning sense means classifying the thing and developing a deterministic algorithm to process it. If you don't have a deterministic algorithm to process it, it isn't understanding. LLMs learn to approximate, we do that too, but then we develop algorithms to process input and generate output using a predefined logical process.
A sorting algorithm is a good example, when you compare that with an LLM sorting a list. they both may have correct outcome, but the sorting algorithm "understood" the logic and will follow that specific logic and have consistent performance.
> understanding in a reasoning sense means classifying the thing and developing a deterministic algorithm to process it.
That's the learning-part I was talking about. Which is mainly supported by humans at the moment, which why I called it proto-intelligence.
> If you don't have a deterministic algorithm to process it, it isn't understanding.
Commercial AIs like ChatGPT do have the ability to call programs and integrate the result in their processing. Those AIs are not really just LLMs. The results are still rough and poor, but the concept is there and growing.
> That's the learning-part I was talking about. Which is mainly supported by humans at the moment, which why I called it proto-intelligence.
Maybe it's just semantics, but I don't think LLMs even come close to a fruit fly's intelligence. Why can't we recognize and accept them for what they are, really powerful classifiers of data.
> Commercial AIs like ChatGPT do have the ability to call programs and integrate the result in their processing. Those AIs are not really just LLMs. The results are still rough and poor, but the concept is there and growing.
Yeah RAG and all of that, but those programs use deterministic algorithms. Now, if LLMs generated programs they call on as tools, that would be much more like the proto-intelligence you're talking about.
Semantics are boring, but it's important that we're not content or celebrate early by calling it what it isn't.
There are no true and untrue claims about how the brain works, because we have no idea how it works.
The reason people give that humans are not auto-complete is "Obviously I am not an autocomplete"
Meanwhile, people are just a black box process that output words into their head, which they then take credit for, and calling it cognition. We have no idea how that black box that serves up a word when I say "Think of a car brand" works.
Flagrantly, ridiculously untrue. We don't know the precise nuts and bolts regarding the emergence of consciousness and the ability to reason, that's fair, but different structures of the brain have been directly linked to different functions and have been observed in operation on patients being stimulated in various ways with machinery attached to them reading levels of neuro-activity in the brain, and in specific regions. We know which parts handle our visual acuity and sense of hearing, and even cooler, we can watch those same regions light up when we use our "minds eye" to imagine things or engage in self-talk, completely silent speech that nevertheless engages our verbal center, which is also engaged by the act of handwriting and typing.
In short: no, we don't have the WHOLE answer. But to say that we have no idea is categorically ridiculous.
As to the notion of LLMs doing similarly: no. They are trained on millions of texts of various sources of humans doing thinking aloud, and that is what you're seeing: a probabilistic read of millions if not billions of documents, written by humans, selected by the machine to "minimize error." And crucially, it can't minimize it 100%. Whatever philosophical points you'd like to raise about intelligence or thinking, I don't think we would ever be willing to call someone intelligent if they just made something up in response to your query, because they think you really want it to be real, even when it isn't. Which points to the overall charade: it wants to LOOK intelligent, while not BEING intelligent, because that's what the engineers who built it wanted it to do.
Accepting as true "We don't know how the brain works in a precise way" does not mean that obviously untrue statements about the human brain cannot still be made.
Your brain specifically, however, is an electric rat that pulls on levers of flesh while yearning for a taste of God's holiest cheddar.
You might reply, "no! that cannot be!", but my statement isn't untrue, so it goes.
We’re obviously more advanced than an LLM, but to claim that human beings simply generate output based on inputs and context (environment, life experience) is not silly.
> People claiming this online aren't going around murdering their spouses like you'd delete an old LLama model from your hard drive.
I'm saying you'd object to being treated like an LLM and don't really have conviction when you make these claims.
I'd also say stringing together A.I. buzzwords (input output) to describe humans isn't really an argument so much as what philosophers call a category error.
That I wouldn’t treat a human like an LLM is completely irrelevant to the topic.
Input and output are not AI buzzwords, they’re fundamental terms in computation. The argument that human beings are computational has been alive in philosophy since the 1940’s brother…
"the argument that human beings are computational has been alive in philosophy since the 1940’s brother"
I recently read a little philosophy on this, to say humans are "computational" you first have to define what you mean by "computational". Depending on the definition you employ, you are either making a wacky claim, a controversial claim, or a noncontroversial claim.
Since you provide no definition, you are saying nothing at all.
The discourse that comes up in AI topics seems more glib language games than actual concern over human nature, computational or otherwise.
>That I wouldn’t treat a human like an LLM is completely irrelevant to the topic.
The fact you wouldn't treat a human like an LLM suggests your constant claim of similarity is not something you actually believe.
Our output is quite literally the sum of our hardware (genetics) and input (immediate environment and history). For anyone who agrees that free will is nonsense, the debate is already over, we’re nothing more than output generating biological machines.
> We also just brute force "autocompletion" with our learned knowledge and combine it to new parts, which we then add to our learned knowledge to deepen the process
you know this because you're a cognitive scientist right? or because this is the consensus in the field?
An LLM by itself is not thinking, just remembering and autocompleting. But if you add a feedback loop where it can use tools, investigate external files or processes, and then autocomplete on the results, you get to see something that is (close to) thinking. I've seen claude code debug things by adding print statements in the source and reasoning on the output, and then determining next steps. This feedback loop is what sets AI tools apart, they can all use the same LLM, but the quality of the feedback loop makes the difference.
>But if you add a feedback loop where it can use tools, investigate external files or processes, and then autocomplete on the results, you get to see something that is (close to) thinking
It's still just information retrieval. You're just dividing it into internal information (the compressed representation of the training data) and external information (web search, API calls to systems, etc). There is a lot of hidden knowledge embedded in language and LLMs do a good job of teasing it out that resembles reasoning/thinking but really isn't.
No, it's more than information retrieval. The LLM is deciding what information needs to be retrieved to make progress and how to retrieve this information. It is making a plan and executing on it. Plan, Do, Check, Act. No human in the loop if it has the required tools and permissions.
I can't speak for academic rigor, but it is very clear and specific from my understanding at least. Reasoning, simply put is the ability to come to a conclusion after analyzing information using a logic-derived deterministic algorithm.
* Humans that make mistakes are still considered to be reasoning.
* Deterministic algorithms have limitations, like Goedel incompleteness, which humans seem able to overcome, so presumably, we expect reasoning to also be able to overcome such challenges.
1) I didn't say we were, but when someone is called reasonable or acting with reason, then that implies deterministic/algorithmic thinking. When we're not deterministic, we're not reasonable.
2) Yes, to reason does imply to be infallible. The deterministic algorithms we follow are usually flawed.
3) I can't speak much to that, but I speculate that if "AI" can do reasoning, it would be a much more complex construct that uses LLMs (among other tools) as tools and variables like we do.
Just ask it how many r's are in strawberry and you will realize there isn't a lot of reasoning going on here, it's just trickery on top of token generators.
How about non-determinism (i.e. hallucinations)? Ask a human ANY question 3 times and they will give you the same answer, every time, unless you prod them or rephrase the question. Sure the answer might be wrong 3 times, but at least you have consistency. Then again, maybe that's a disadvantage for humans!
Ask an illiterate person the same thing and they will fail badly too. Is it impossible to have intelligence without literacy? (Bonus: if so, how was writing invented?)
Yes but an illiterate person can be taught to read. Also LLMs generally fail (non-deterministically) at math in general, but humans can also be taught math.
Furthermore regarding reasoning, just ask any LLM how many "r letters are in strawberry" - repeat maybe 3 times just to get a feeling for how much variance in answers you can get. And this "quirk" of the inability to get the right answer is something that after 2 years making fun of LLMs online on various forums is still an issue. The models aren't getting smarter, and definitely aren't thinking, they are still token generators with a few tricks on top to make them seem more intelligent than predecessors.
Sorry to say, Claude Sonnet 4.5 (with and without extended thinking) has this one solved now. So, not “any” LLM, but maybe some still need work.
I presume they finally got around to training the internal reasoning loop into predicting when it needs to apply answer predictions character-by-character.
—-
Transcripts from Claude Sonnet 4.5, initially with extended thinking:
> How many a’s in the phrase “artichokes and aardvarks”? Be sure to check your answer.
> Let me count the letter ‘a’ in “artichokes and aardvarks”:
• artichokes: 1 ‘a’ (at the beginning)
• and: 1 ‘a’ (at the beginning)
• aardvarks: 3 ‘a’s (two at the beginning: “aa”, and one in the middle: “ardvarks”)
Total: 1 + 1 + 3 = 5 ‘a’s
Let me verify by marking them: artichokes and aardvarks
Yes, there are 5 letter ‘a’s in the phrase “artichokes and aardvarks”.
> Are you sure that’s correct?
> Let me recount more carefully:
“artichokes and aardvarks”
Going through each word letter by letter:
1. artichokes - has 1 ‘a’ (first letter)
2. and - has 1 ‘a’ (first letter)
3. aardvarks - has 3 ‘a’s (first letter, second letter, and one more in the middle)
Total: 1 + 1 + 3 = 5 ‘a’s
Yes, I’m confident the answer is 5. Is there something specific you’d like me to recheck?
> How many r’s in strawberry?
> Let me count the letter ‘r’ in “strawberry”:
s-t-r-a-w-b-e-r-r-y
Going through each letter:
• s: not an r
• t: not an r
• r: 1st r
• a: not an r
• w: not an r
• b: not an r
• e: not an r
• r: 2nd r
• r: 3rd r
• y: not an r
There are 3 r’s in “strawberry”.
<Without extended thinking>
> How many u’s in universalulis?
> Looking at “universalulis” letter by letter:
u-n-i-v-e-r-s-a-l-u-l-i-s
There are 2 u’s in “universalulis”.
Now try a different language. My take is hard RL tuning to fix these "gotcha:s" since the underlying model can't do it on its own.
OpenAI is working on ChatGPT the application and ecosystem. They have transitioned from model building to software engineering with RL tuning and integration of various services to solve the problems the model can't do on its own. Make it feel smart rather than be smart.
This means that as soon as you find a problem where you step out of the guided experience you get the raw model again which fails when encountering these "gotchas".
Edit - Here's an example where we see a very tuned RL experience in English where a whole load of context is added on how to solve the problem while the Swedish prompt for the same word fails.
You can tell it "be careful about the tokenizer issues" in Swedish and see how that changes the behavior.
The only thing that this stupid test demonstrates is that LLM metacognitive skills are still lacking. Which shouldn't be a surprise to anyone. The only surprising thing is that they have metacognitive skills, despite the base model training doing very little to encourage their development.
LLMs were not designed to count letters[0] since they work with tokens, so whatever trick they are now doing behind the scenes to handle this case, can probably only handle this particular case. I wonder if it's now included in the system prompt. I asked ChatGPT and it said it's now using len(str) and some other python scripts to do the counting, but who knows what's actually happening behind the scenes.
There's no "trick behind the scenes" there. You can actually see the entire trick being performed right in front of you. You're just not paying attention.
That trick? The LLM has succeeded by spelling the entire word out letter by letter first.
It's much easier for an LLM to perform "tokenized word -> letters -> letter counts" than it is to perform "tokenized word -> letter counts" in one pass. But it doesn't know that! It copies human behavior from human text, and humans never had to deal with tokenizer issues in text!
You can either teach the LLM that explicitly, or just do RLVR on diverse tasks and hope it learns the tricks like this by itself.
Auto completion just means predicting the next thing in a sequence. This does not preclude reasoning.
> I don't get why you would say that.
Because I see them solve real debugging problems talking through the impact of code changes or lines all the time to find non-obvious errors with ordering and timing conditions on code they’ve never seen before.
It does not imply it either. to claim reasoning you need evidence. it needs to reliably NOT hallucinate results for simple conversations for example (if it has basic reasoning).
> Because I see them solve real debugging problems talking through the impact of code changes or lines all the time to find non-obvious errors with ordering and timing conditions on code they’ve never seen before.
Programming languages and how programs work are extensively and abundantly documented, solutions to problems and how to approach them,etc.. have been documented on the internet extensively. It takes all of that data and it completes the right text by taking the most correct path way based on your input. it does not actually take your code and debug it. it is the sheer volume of data it uses and the computational resources behind it that are making it hard to wrap your head around the difference between guessing and understanding. You too can look at enough stack overflow and (poorly) guess answers for questions without understanding anything about the topic and if you guess enough you'll get some right. LLMs are just optimized to get the amount of correct responses to be high.
Right, it's irrelevant to the question of whether they can reason.
> to claim reasoning you need evidence
Frankly I have no idea what most people are talking about when they use the term and say these models can't do it. It seems to be a similarly hand-wavey exercise as when people talk about thinking or understanding.
> it needs to reliably NOT hallucinate results for simple conversations for example (if it has basic reasoning).
That's not something I commonly see in frontier models.
Again this doesn't seem related to reasoning. What we call hallucinations would be seen in something that could reason but had a fallible memory. I remember things incorrectly and I can reason.
> it does not actually take your code and debug it
It talks through the code (which it has not seen) and process step by step, can choose to add logging, run it, go through the logs, change what it thinks is happening and repeat. It can do this until it explains what is happening, creates test cases to show the problem and what triggers it, fixes it and shows the tests pass.
If that's not debugging the code I really don't know what to call it.
>I don't get why you would say that. it's just auto-completing. It cannot reason. It won't solve an original problem for which it has no prior context to "complete" an approximated solution with. you can give it more context and more data,but you're just helping it complete better. it does not derive an original state machine or algorithm to solve problems for which there are no obvious solutions. it instead approximates a guess (hallucination).
I bet you can't give an example such written problem that a human can easily solve but no LLM can.
And because ALL the marketing AND UX around LLMs is precisely trying to imply that they are thinking. It's not just the challenge of grasping the ridiculous amount of resources poured in, which does including training sets, it's because actual people are PAID to convince everybody those tools are actually thinking. The prompt is a chatbox, the "..." are there like a chat with a human, the "thinking" word is used, the "reasoning" word is used, "hallucination" is used, etc.
You're right. Unfortunately, it seems that not many are willing to admit this and be (rightly) impressed by how remarkably effective LLMs can be, at least for manipulating language.
The vast majority of human “thinking” is autocompletion.
Any thinking that happens with words is fundamentally no different to what LLMs do, and everything you say applies to human lexical reasoning.
One plus one equals two. Do you have a concept of one-ness, or two-ness, beyond symbolic assignment? Does a cashier possess number theory? Or are these just syntactical stochastic rules?
I think the problem here is the definition of “thinking”.
You can point to non-verbal models, like vision models - but again, these aren’t hugely different from how we parse non-lexical information.
> Any thinking that happens with words is fundamentally no different from what LLMs do.
This is such a wildly simplified and naive claim. "Thinking with words" happens inside a brain, not inside a silicon circuit with artificial neurons bolted in place. The brain is plastic, it is never the same from one moment to the next. It does not require structured input, labeled data, or predefined objectives in order to learn "thinking with words." The brain performs continuous, unsupervised learning from chaotic sensory input to do what it does. Its complexity and efficiency are orders of magnitude beyond that of LLM inference. Current models barely scratch the surface of that level of complexity and efficiency.
> Do you have a concept of one-ness, or two-ness, beyond symbolic assignment?
Obviously we do. The human brain's idea of "one-ness" or "two-ness" is grounded in sensory experience — seeing one object, then two, and abstracting the difference. That grounding gives meaning to the symbol, something LLMs don't have.
The instantiation of models in humans is not unsupervised, and language, for instance, absolutely requires labelled data and structured input. The predefined objective is “expand”.
They infer meaning from noisy, ambiguous sensory streams. The labels are not explicit, they are discovered through correlation, context, and feedback.
So although caregivers sometimes point and name things, that is a tiny fraction of linguistic input, and it is inconsistent. Children generalize far beyond that.
Real linguistic input to a child is incomplete, fragmented, error-filled, and dependens on context. It is full of interruptions, mispronunciations, and slang. The brain extracts structure from that chaos. Calling that "structured input" confuses the output - inherent structure of language - with the raw input, noisy speech and gestures.
The brain has drives: social bonding, curiosity, pattern-seeking. But it doesn't have a single optimisation target like "expand." Objectives are not hardcoded loss functions, they are emergent and changing.
You're right that lack of linguistic input prevents full language development, but that is not evidence of supervised learning. It just shows that exposure to any language stream is needed to trigger the innate capacity.
Both complexity and efficiency of the human learning is just on another level. Transformers are child's play compared to that level. They are not going to gain consciousness, and no AGI will happen in the foreseeable future, it is all just marketing crap, and it's becoming more and more obvious as the dust settles.
They never saw any kind of object, they only saw labeled groups of pixels – basic units of a digital image, representing a single point of color on a screen or in a digital file. Object is a material thing that can be seen and touched. Pixels are not objects.
My friend, you are blundering into metaphysics here - ceci n’est pas une pipe, the map is the territory, and all that.
We are no more in touch with physical reality than an LLM, unless you are in the habit of pressing your brain against things. Everything is interpreted through a symbolic map.
when photons strike your retina, they are literally striking brain tissue that is been pushed outward into the skull front window. Eyes are literally the brain, so yes, we are pressing it against things to "see" them.
Okay, goalpost has instantly moved from seeing to "seeing and touching". Once you feed in touch sensor data, where are you going to move the goalpost next?
Models see when photons hit camera sensors, you see when photons hit your retina. Both of them are some kind of sight.
The difference between photons hitting the camera sensors and photons hitting the retina is immense. With a camera sensor, the process ends in data: voltages in an array of photodiodes get quantized into digital values. There is no subject to whom the image appears. The sensor records but it does not see.
When photons hit the retina, the same kind of photochemical transduction happens — but the signal does not stop at measurement. It flows through a living system that integrates it with memory, emotion, context, and self-awareness. The brain does not just register and store the light, it constructs an experience of seeing, a subjective phenomenon — qualia.
Once models start continuously learning from visual subjective experience, hit me up – and I'll tell you the models "see objects" now. Until direct raw photovoltaic information stream about the world around them without any labelling can actually make model to learn anything, they are not even close to "seeing".
We do a lot of autocompletion and LLMs overlap with that for sure. I don't know about the "vast majority" even basic operations like making sure we're breathing or have the right hormones prompted are not guesses but deterministic algorithmic ops. Things like object recognition and speech might qualify as autocompletion.
But let's say you need to setup health-monitoring for an application. that's not an autocomplete operation. you must evaluate various options, have opinions on it, weigh priorities,etc.. in other words, we do autocompletion but even then the autocompletion is a basic building block or tool we use in constructing more complex decision logic.
If you train an animal to type the right keys on a keyboard that generates a hello world program, you didn't just teach them how to code. they just memorized the right keys that lead to their reward. a human programmer understands the components of the code, the intent and expectations behind it, and can reason about how changes would affect outcomes. the animal just knows how the reward can be obtained most reliably.
Sure. But neither do you. So are you really thinking or are you just autocompleting?
When was the last time you sat down and solved an original problem for which you had no prior context to "complete" an approximated solution with? When has that ever happened in human history? All the great invention-moment stories that come to mind seem to have exactly that going on in the background: Prior context being auto-completed in an Eureka! moment.
I think (hah) you're understimating what goes on when living things (even small animals) think. We use auto-compleition for some tasks, but it is a component of what we do.
Let's say your visual system auto-completes some pattern and detects a snake while you're walking, that part is auto-completion. You will probably react by freezing or panicing, that part is not auto-compleition, it is a deterministic algorithm. But then you process the detected object, auto-compleiting again to identify it as just a long cucumber. But again, the classification part is auto-completion. What will you do next? "Hmm, free cucumber, i can cook with it for a meal" and you pick it up. auto-completion is all over that simple decision, but you're using results of auto-completion to derive association (food), check your hunger level (not auto-completion), determine that the food is desirable and safe to eat (some auto-compleition), evalute what other options you have for food (evaluate auto-complete outputs), and then instruct your nervous system to pick it up.
We use auto-compleition all the time as an input, we don't reason using auto-compleition in other words. You can argue that if all your input is from auto-completion (it isn't) then it makes no difference. But we have deterministic reasoning logical systems that evaluate auto-completion outputs. if your cucumber detection identified it as rotten cucumber, then decision that it is not safe to eat is not done by auto-completion but a reasoning logic that is using auto-completion output. You can approximate the level of rot but once you recognize it as rotten, you make decision based on that information. You're not approximating a decision, you're evaluating a simple logic of: if(safe()){eat();}.
Now amp that up to solving very complex problems. what experiments will you run, what theories will you develop, what R&D is required for a solution,etc.. these too are not auto-completions. an LLM would auto complete these and might arrive at the same conclusion most of the time. but our brains are following algorithms we developed and learned over time where an LLM is just expanding on auto-completion but with a lot more data. In contrast, our brains are not trained on all the knowledge available on the public internet, we retain a tiny miniscule of that. we can arrive at similar conclusions as the LLM because we are reasoning and following algorithms matured and perfected over time.
The big take away should be that, as powerful as LLMs are now, if they could reason like we do, they'd dominate us and become unstoppable. Because their auto-completion is many magnitudes better than ours, if they can write new and original code based on an understanding of problem solving algorithms, that would be gen ai.
We can not just add 1 + 1 but prove that the addition operation is correct mathematically. and understand that when you add to a set one more object, the addition operation always increments. We don't approximate that, we always, every single time , increment because we are following an algorithm instead of choosing the most likely correct answer.
> If it thought and reasoned, it would know to ignore those and focus on the variables and input at hand to produce deterministic and consistent output
You only do this because you were trained to do this, eg. to see symmetries and translations.
A long time ago I noticed that I sometimes already had a complete thought before my inner monologue turned it into words. A few times I tried skipping the inner monologue because I'd clearly already thought the thought. Turns out the bit of my brain that creates the inner monologue from the thought, can generate a sense of annoyance that the rest of my brain can feel.
Not that it matters, there's evidence that while LLMs output one word at a time, they've got forward-planning going on, having an idea of the end of a sentence before they get there.
Indeed, and it seems like they would really struggle to output coherent text at all if there was not some kind of pre-planning involved (see how even humans struggle with it in games where you have to construct a sentance by having each person shout out one word at a time). Even GPT-2 likely had at least some kind of planning for the next few words in order to be as coherent as it was.
Tell that to German-speakers, where the verb comes last, and the order of things in sentences is not anything like English, therefore requiring you to think of the entire sentence before you just spit it out. Even the numbers are backwards (twenty-two is two-and-twenty) which requires thinking.
Furthermore, when you ask an LLM to count how many r's are in the word strawberry, it will give you a random answer, "think" about it, and give you another random answer. And I guarantee you out of 3 attempts, including reasoning, it will flip-flop between right and wrong, but unlike a human, it will be random, because, unlike humans who, when asked "how many r's are in the word strawberry" will not be able to tell you the correct answer every. fucking. time.
> How does Claude write rhyming poetry? Consider this ditty:
> He saw a carrot and had to grab it,
> His hunger was like a starving rabbit
> To write the second line, the model had to satisfy two constraints at the same time: the need to rhyme (with "grab it"), and the need to make sense (why did he grab the carrot?). Our guess was that Claude was writing word-by-word without much forethought until the end of the line, where it would make sure to pick a word that rhymes. We therefore expected to see a circuit with parallel paths, one for ensuring the final word made sense, and one for ensuring it rhymes.
> Instead, we found that Claude plans ahead. Before starting the second line, it began "thinking" of potential on-topic words that would rhyme with "grab it". Then, with these plans in mind, it writes a line to end with the planned word.
The part about strawberry is just not right. That problem was solved. And I do think it's a stretch to say German speakers think of the entire sentence before speaking it.
LLMs were not designed to count letters[0] since they work with tokens, so whatever trick they are now doing behind the scenes to handle this case, can probably only handle this particular case. I wonder if it's now included in the system prompt. I asked ChatGPT and it said it's now using len(str) and some other python scripts to do the counting, but who knows what's actually happening behind the scenes.
military alliance is related to trade dispute..how?
The US needs them more than they need the US. They're strategically placed at the bosphorous containing russia's western advances and containing the baltics. Ukraine would be the next best thing, but they're not in NATO.
what's left of it after it's dismantlement you mean? it isn't much of an alliance now. and if it is going to be used to twist arms in trade disagreements, it is already dead.
Deeplinking is awesome! The Azure portal is my favorite example. You could be many layers deep in some configuration "blade" and the URL will retain the exact location you are in the UI.
Competition is good, but they should start from the OS level.
Outside of Windows and MacOS, there is no OS ecosystem that works as well and at scale for an enterprise level deployment.
I don't get the whole "US" aspect, why bring politics into this? We need alternatives and competition regardless of all that. I don't care if Europe, China or India make it, a viable alternative would be a game changer.
For Europe, they're solving the wrong problem. Solve the problem of low pay for developers, and a stifling regulatory atmosphere that inhibits disruptive startups.
If there is a good and viable alternative, why is it just for Europe? It could boost Europe's economy by selling to America and the rest of the world. The tech needs to be good, if the US can do it, why can't Europe?
Replacing office or one app at at time only addresses surface level issues.
> I don't get the whole "US" aspect, why bring politics into this?
There is some level of technology dependency after which it can become a national security risk (and therefore, political). This has happened a bunch in history (e.g. countries reducing reliance on adversaries oil supplies or foreign semiconductors).
The few examples in the article are small and irrelevant in the grand scheme of things, but I think the article is trying to make the point that this is part of a larger divestiture from American tech companies.
Whether that larger divestiture is happening, however, is probably a pretty dubious claim.
This kind of reminds me of during COVID when everybody was writing articles like, “SF and NYC are dead and everyone is moving out!” Turns out… not so much.
Journalists love taking a few small data points and extrapolating to what feels like should be happening and leading the reader to extreme conclusions.
It’s much more fun to write that a hurricane is coming than it is to write, “we’re seeing some light rain today, and it’s probably within normal variance for this time of year.”
It's just that this isn't like raw resources like oil. there is no "good" or "superior" oil. it's just oil. and there are some differences, but there is no "american refined gas" that is superior.
But with tech, there is a question of quality and superiority. People that use M365 suite can work and collaborate more effectively, if not for actual superiority of the product and its OS integration, then simply because it is so well known and users prefer it the most (it does not repel talent as much).
Why give talented european skilled people more reasons to leave the EU for US, as if existing issues aren't enough? Why not instead directly compete with Microsoft and Google. Why isn't the EU investing in that? They're caught in the "open source" trap. I hate to say it, but they need to make closed source proprietary software that is commercially competitive.
Care to provide an example? Just trying to be objective.
Historically, nearly all states have been surveillance states (including early America). all that has changed in the last ~30 years is technology (capability), and public attitude.
Mail snooping has been around as long as mail has been around. Warrantless wiretaps on telphone lines were around in just about every country that started having telephones, including the US. AT&T's NSA listening room (the company that invented telephones) isn't unique or special. Book ciphers were invented because snooping courier's messages was so common.
The NSA didn't build a Yottabyte-scale datacenter for no reason either.
I'm not disagreeing with your view, I'm just saying that the anti-surveillance sentiment is particularly unique to the cultural "West", and to the post-WW2 generations.
This is disingenuous as a counter-argument because it's never been possible at this scale — perhaps excepting the Stasi. The sentiment has arisen because:
1) yes, people are more likely to fight it.
2) it's being done at an unprecedented and very intrusive scale.
Wiretapping required more labor on all fronts. Mail snooping the same.
If the "East" isn't fighting this, it might simply be because they see no recourse with their less democratic institutions. (I'm borrowing from you for this last point, I'd never presume the West to be as democratic as we'd like to believe.)
I can't speak for Europe, but in the west, a vast majority of people already accept that the government is doing mass surveillance, and they think it's a good thing. It is similar in China, India,etc.. from what I understand. I don't know enough about europe.
This scale has been possible since at least the mid 00's (20 years now , technically a historical fact). consider that most of the files in snowden's leak were from around '08.
for example ones porn history could be used against that person if he she wants to go for public office. So there is a chilling effect for watching porn.
Politics are dirty allready but the way down is deep.
Maybe they should "own it" instead? I don't think these days legal porn use would be all that harmful. but i don't disagree with your point.
But look at it this way, surveillance already exists, all major governments are hacking into people's phone and spying on them, and forcing telco's to do dragnet surveillance. chat control is about doing it at a greater scale and formalizing it for normal law enforcement use instead of things like counter-terrorism.
yes and I understand the benefit but i doubt the real world consequenzes. It would only filter the criminals
who are good at opsec and those who arent. imho elevating a more proffesional type of criminals.
Honestly most major crimes are money related. Get rid of papermoney and make digital payments traceable for law enforcement. I know there is something like gold bitcoin etc but at the end knowing how
money moves is more power then knowing what people are writing
That said, I am huge fan of works like this. But in practice, the security layer that betrays all of this tends to be the firmware layer.
My dream is to have something like the Framework computer use verifiably secure EFI firmware, as well as similarly verified and audited firmware for every hardware component.
reply