If you follow the spirit of the essay, Ted Chiang suspects Anthropic is being cute with the idea. There's no good reason to suspect consciousness in LLMs, so the null hypothesis must be taken as default. And Anthropic sort of knows this, but for marketing purposes they are playing loose with definitions, hedging their bets and drawing up "constitutions" with terms for the "well being" and "happiness" of Claude, while at the same time -- and this is an important part of the essay -- being unethical (think slavery) if we assume they truly believe Claude could be conscious.
Of course Anthropic is being cute about this, they have a vested interest in hype and overpromising; even drumming up the "AI danger" is a way of hyping up the tech.
Ted Chiang is taking the default and honest position: "no, LLMs aren't conscious. If you truly believe they are, show us the really scientific and rigorous proof."
In practice this is a much weaker stance than saying "maybe they are conscious". It is the only honest scientific stance, really.
This is a bit of a side point, but you are severely misusing "null hypothesis".
1. A null hypothesis is part of a statistical test: it is the hypothesis whose consequences are used to compute a test statistic, p-value, confidence interval, or related error rates. There is no null hypothesis for arbitrary philosophical debates. You cannot compute a p-value for H_0 that LLMs aren't conscious. You might be using "null hypothesis" informally to mean "default position", but remember this has no relation to its scientific meaning in inferential statistics and doesn't lend any scientific legitimacy to implying we should assume LLMs aren't conscious.
2. It's up to the experimental design to choose a null hypothesis. Yes, normally it would be something like "this drug has no effect", but it doesn't have to be. You could set up an experiment where the null hypothesis is that the drug you're testing is equally effective as another drug. There is no objective truth of what the null hypothesis needs to be. It's up to you and your test setup.
3. There is no requirement that the null hypothesis be likely. You and everyone else on Earth can believe there's a 99.99% chance that the null hypothesis of an experiment is wrong. Assuming you could set up a statistical experiment to detect consciousness, whatever you pick for your null hypothesis has no bearing on whether Ted Chiang or Anthropic is more justified.
> There's no good reason to suspect consciousness in LLMs
That's the entire debate. One side thinks there are good reasons to suspect consciousness and presents their arguments for that. The other side thinks there aren't and presents their arguments against.
What happened here is you judged the no-consciousness side to be more likely, but then you try to pass off your judgement as an obvious prior that everyone should share.
It would be fine for you to say you think LLM consciousness is unlikely. That makes it clear it's your judgement on the debate. It's fine for you to say you would require extraordinary evidence because your priors are so low. It's fine for you to say you think everyone who thinks otherwise is wrong.
But there is no reason for anyone who has judged the arguments differently than you to accept that your priors should be the default. There is no meta-level principle that you can appeal to here. If you want to show consciousness in LLMs is unlikely, you have to use arguments about the issue itself.
> Ted Chiang is taking the default and honest position: "no, LLMs aren't conscious. If you truly believe they are, show us the really scientific and rigorous proof."
That would be a bad faith, double standard. No one has ever shown scientific and rigorous proof that humans are conscious. Philosophers call this the hard problem of consciousness.
That aside, the honest default position to anything should be "we don't know for sure", which can then be followed by considering the evidence and estimating the likelihood. If you think the sun will rise tomorrow, that's only because we've gathered extremely strong evidence that it will. If today was your first day hearing about the sun and you know nothing else about it, you have no reason to think it will or won't rise tomorrow.
Your pedantic nitpick is duly noted and dismissed as a red herring. You know what I meant and you know there's a real scientific principle behind my use of "null hypothesis".
Ted Chiang's is the only take that is honest and makes scientific sense: we cannot accept the extraordinary claim that LLMs "might" (or whatever hedging words Anthropic chooses) be conscious without extraordinary evidence, and experiments done by people who have no vested interest in hyping up the tech. Possibly something that would take years or decades, impossibly long for the needs of today's marketing.
If you (or Anthropic) want to claim "but this is philosophy (the subset unrelated to science" feel free, but that's like a religious belief. Not interested, there's no scientific claim to debate and that's not what I'm engaging with.
There's no real serious debate here. Everyone's just vibing, and worse, vibing on hype done by a business with vested interests, doing free marketing for them.
It's not just a nitpick, because this is really at the core of your position. Ignoring the terminology, you're trying to claim there is some principle that says everyone should have the default position that LLMs aren't conscious instead of uncertainty. And there isn't. You have to weigh actual arguments for or against LLM consciousness.
> There's no real serious debate here. Everyone's just vibing
Sure, but if everyone including Ted Chiang is just vibing, and no one can prove anything, that makes Ted's statement false and Anthropic's statement true.
> If you (or Anthropic) want to claim "but this is philosophy
I'd just like you or Ted Chiang to not pretend your vibes are science. You're welcome to have them. I also don't think LLMs are conscious. But it's our personal judgement, and not scientific fact.
