Given that this is 'research' carried out (and seemingly published) by a company with a direct interest in selling you a product (or, rather, getting investors excited/panicked), can we trust it?
This is just about seeing whether the model can accurately report on its internal reasoning process. If so, that could help make models more reliable.
They say it doesn't have that much to do with the kind of consciousness you're talking about:
> One distinction that is commonly made in the philosophical literature is the idea of “phenomenal consciousness,” referring to raw subjective experience, and “access consciousness,” the set of information that is available to the brain for use in reasoning, verbal report, and deliberate decision-making. Phenomenal consciousness is the form of consciousness most commonly considered relevant to moral status, and its relationship to access consciousness is a disputed philosophical question. Our experiments do not directly speak to the question of phenomenal consciousness. They could be interpreted to suggest a rudimentary form of access consciousness in language models. However, even this is unclear.
> They say it doesn't have that much to do with the kind of consciousness you're talking about
Not much but it likely has something to do with it, so experiments on access consciousness can still be useful to that question. You seem to be making an implication about their motivations which is clearly wrong, when they've been saying for years that they do care about (phenomenal) consciousness, as bobbylarrybobb said.
On what grounds do you think it likely that this phenomenon is at all related to consciousness? The latter is hardly understood. We can identify correlates in beings with constitutions very near to ours, which lend credence (but zero proof) to the claim they're conscious.
Language models are a novel/alien form of algorithmic intelligence with scant relation to biological life, except in their use of language.
Yes, they do care about it, and unlike many AI researchers they've bothered to learn something about philosophy of mind. They point out that "the philosophical question of machine consciousness is complex and contested, and different theories of consciousness would interpret our findings very differently. Some philosophical frameworks place great importance on introspection as a component of consciousness, while others don’t." Which would be one reason they point out that these experiments don't help resolve the issue.
They go further on their model welfare page, saying "There’s no scientific consensus on whether current or future AI systems could be conscious, or could have experiences that deserve consideration. There’s no scientific consensus on how to even approach these questions or make progress on them."
What would you title this article to make it less "clickbait"? This is one of the least clickbait headlines I've seen, it's literally just describing what's in the article.
> Would knowing that Claude is maybe kinda sorta conscious lead more people to subscribe to it?
For anyone having paid attention, it has been clear for the past two years that Dario Amodei is lobbying for strict regulation on LLMs to prevent new entrants on the market, and the core of its argument is that LLMs are fundamentally intelligent and dangerous.
So this kind of “research” isn't targeted towards their customers but towards the legislators.
The thing is, if he is right, or will be in the near future, regulators will get scared and ban the things outright, throwing the baby out with the bathwater. Yes, he benefits if they step in early, but it isn’t a given that we all don’t when this happens.
Well, maybe look at how many people worked in the agricultural sector is 1900 and how many do so today.
Automation of field labour has decreased the worker count by a factor 20 or something.
Same for the mining sector.
It's not necessarily a bad thing as working in the fields or in coal mines wasn't pleasant, but pretending automation doesn't reduce employment is nonsense.
No matter how hard the work conditions are, people don't usually accept its disappearance.
Automatization reducing work can actually be a good thing, as it is the reason why we can have vacations, retirement and long studies: because the society's need for work is lower than before.
I can't be exactly sure of the intended target, but it certainly helps to increase the sense of FOMO among investors even if as an unintended side effect (though I don't think it is unintended).
This is the worst possible objection to scientific research. All medication in the US is approved by research conducted by the company trying to sell it, because nobody else is motivated to do it. And if it's properly conducted and preregistered, this doesn't matter!
It basically just shows you're looking for a way to dismiss something that doesn't require you to understand it or check their work.
It seems completely obvious that AI companies benefit massively from (and in many cases likely only continue to stay afloat because of) 'research papers' like this.
I also don't think a scientist purely interested in the truth would be claiming anything about concepts like 'introspection' that are nebulous and only really serve to capture the imagination of the general public (and, of course, investors).
The difference between AI and the pharmaceutical industry should be clear: one produces products of undeniable value, and the other is largely built on hype and endless dreaming of what might come next, but so far hasn't.
> So you don't think it's relevant at all? Really?
It's relevant if it's not preregistered. I agree this one is not preregistered and they should release their model weights instead of doing random tinkering on it themselves.
It feels a little like Nestle funding research that tells everyone chocolate is healthy. I mean, at least in this case they're not trying to hide it, but I feel that's just because the target audience for this blog, as you note, are rich investors who are desperate to to trust Anthropic, not consumers.
Yes, I think: it was we can't be sure we can trust output form self-interested research, I believe. Please feel free to correct me :) If you’re curious about mine, it’s sort of a humbly self aware Jonathan Swift homage.
