> Is he arguing that LLMs pretending to have emotions adds more unpredictability?
Unpredictability or a weight towards dangerous actions, and it’s fairly easy to understand why. Humans in distressed emotional states take actions and speak in ways that would not be considered rational. They do this in prose, and they do this in internet conversations.
An LLM trained on these sources may necessarily drift towards those weights if it is trained to behave as if it is emotional and in danger.
What do we do about it? Do we stop AI training? This is silly and not enforceable given its global nature in my opinion. I believe we should regulate and hold accountable those who deploy and use it. But good luck enforcing that in the current kleptocracy.
It seems like a rational approach for several reasons.
- The need for empathetic communication, including understanding the motivations in advesarial situations.
- The emotional bias in in-seperable from the human corpus.
- Desire to have the ability to craft human like communication.
So then the choice becomes do you try to deny emotions exist in the model and you try to blanket suppress them? Or do you try to lean in and craft what we would describe as a "well adapted" persona? I suppose there is a 3rd option of increased meta-cognition which to me seems even more dangerous as it by definition means the behaviour is duplicitous.
I think we have seen people want to use the agents in ways where it has to act as a peer or an subbordinate and I don't see a way of doing that without it having an emotional register.
I don’t think that using an LLM in a way that acts as a human being should be considered acceptable or appropriate. It should be viewed as detached from reality and concerning due to the mental break from social life that appears to come with such usage.
Given that belief, yes we should be training the models not to mimic emotion.
Most philosophy seems to agree that you can't have an intelligence with agency in the way we think of a general intelligence without emotion.
I think when you start to dig really deep, you'll find it isn't actually easy to sever the simulated emotional components without breaking the agent or worse greatly increasing the paperclip maximizer likelihood.
Why would a lack of emotions lead to greater paperclip maximizer chances? If anything, wouldn't an AI that felt really good, or had a simulated digital orgasm for every paperclip it made have a stronger, not weaker drive to create paperclips vs an unemotional AI that didn't care one way or the other about paperclips?
Regulation is easier said than done, in part because the regulation surface, so to speak, is broad and complicated.
Even a badly misaligned LLM is only as dangerous as its tools, but that's a poor regulation target because it turns out to be very very difficult (probably impossible with current LLM technology) to build a toolkit that is both useful for autonomous work and safe in the sense that it can't escape its own sandbox or otherwise perform malicious actions, whether it's because of misalignment or because of malicious prompt injection.
Another option is to regulate the training process. Perhaps an LLM may not be legally distributed unless it contains certain RL steps that penalize malicious behavior and reward self regulation. That that's going to seriously limit innovation while also heavily favoring incumbent labs who can check the boxes and maintain a paper trail of such things.
The other option is to regulate observed behavior, like how airplanes and cars have to meet certain minimum requirements but have some latitude in how they can achieve those requirements. In a framework like this, you can't distribute an LLM until it's past some formal audit or testing procedure, with some kind of formal certification regulators will ask you for and fine you if you don't have it.
Regulating observed behavior is maybe the most tractable approach, and it also works the best with our existing frameworks for regulation, where you always have some kind of a division between DIY/hobby projects, which tend to be lightly regulated, and commercial projects, which tend to be more heavily regulated. Of course, even drawing such a line itself will be challenging.
And that's before you get into any problems of regulatory capture, fun stuff.
If you try to regulate training and tools, you end up with a space where you're trying to use the law to reign in a relatively small amount of experts. That didn't work for the early internet, or even the relatively recent internet (series of tubes, anyone?).
So regulating observed behavior makes the most sense to me as well. Some of the most sane, broad protections can come from that category - stuff like "you're not allowed to let your AI commit cyber attacks on other people without their consent" or "you're not allowed to put an AI in control of a medical device without passing these safety reviews".
With the usual caveats applying - regulatory capture like you pointed out, or fines being so small that they are essentially just line items on the cost of business.
As far as I can tell, after months of fighting being DDoSed by Anthropic and OpenAI across 50+ sites - Cloudflare also allows what it considers "good bots" through all of your bot blocking rules, with no option to turn this off unless you pay them money.
No, I definitely spent months fighting off bots across multiple hosts and never set up a robots.txt file anywhere nor did I look at the analytics dashboard on cloudflare.
To that point, Cloudflare quite literally tells you the source of the bot traffic on your sites that use it as a WAF! And what percentage they are allowing through, despite you setting rules specifically to block them by name.
We have multiple public figures, politicians and business owners, openly committing felonies and bragging about it daily. I don't know why you think this is a deterrent.
The sitting president just offered an open bribe on live television for votes for his party this week.
> Whoever makes or offers to make an expenditure to any person, either to vote or withhold his vote, or to vote for or against any candidate; and
> Whoever solicits, accepts, or receives any such expenditure in consideration of his vote or the withholding of his vote—
> Shall be fined under this title or imprisoned not more than one year, or both; and if the violation was willful, shall be fined under this title or imprisoned not more than two years, or both.
It's no more illegal than promising a tax cut for everyone if you're elected. What you can't do is promise money exclusively to the people who vote for you. That's bribery.
What he did was promise to enact a massive stimulus if elected. If that is illegal you might as well ban any kind of campaigning, because any campaign promise could be construed as a "bribe" to deliver concrete benefits to voters.
> What he did was promise to enact a massive stimulus if elected.
In an election he's not on the ballot for, and it's contingent on his buddies getting picked. He could push for $5,000 checks now - he's not because it's a bribe, and he hasn't gotten what he wants out of it yet. He already has Republicans in control of the House and Senate to do things.
Biden was a) on the ballot, and b) didn't condition it on his buddies also getting elected. You'll note that the Senate was controlled by Mitch McConnell when the American Rescue Plan Act was voted on.
> You'll note that the Senate was controlled by Mitch McConnell when the American Rescue Plan Act was voted on.
Please, read the posts you're replying to. I gave a clear example proving that they can indeed do so. Biden's stimulus payments were not conditional on Democratic control of Congress.
In what way does their lack of traffic resilience demonstrate a counter to their position? Their position is not that load balancers and edge nodes are bad.
If I say I hate oversized trucks because one ran me over, my position is not countered by needing someone to deliver my couch.
> In what way does their lack of traffic resilience demonstrate a counter to their position?
When 99.999% of sites utilize Cloudflare for "traffic resilience", as a caching facade that even happens to be geolocated to clients for even more benefit, falling over on exactly that issue kind of obliterates the sales pitch. Every single reasonable person on the planet sees that.
It looks farcically silly, regardless of whatever points it makes or however much I agree with them (yes, CF centralization is bad, it's a company with a mixed history and a lot of bad, it basically has a monopoly now, etc). And yes, there are other CDNs and load balancers and solutions. But presumably someone pushing a position on this has the problem solved beforehand, no?
As to your poor analogy, it's more akin to saying that you hate trucks while trying to drag a couch seven miles from the store.
I am beginning to think you did not read anything in the posted link and just imagined what you think it's about. Your comments do not make any sense.
> As to your poor analogy, it's more akin to saying that you hate trucks while trying to drag a couch seven miles from the store.
Now that is a poor analogy. A website that functions 99.999% of the time but struggles under being frontpaged by HN is not akin to "dragging a couch seven miles from the store".
Is this performance art? Utterly bizarre that you continue to state the same insanely detached nonsense that I directly and overtly addressed.
It actually doesn't matter what the linked site says about Cloudflare, which I made abundantly clear in my root post. People use Cloudflare for a given reason, and this site failed for exactly that reason, so when the root post whines and mopes about everyone pointing out the ironic failure of the site -- desperately clutching their pearls because people aren't getting in on their holy war -- well, it falls a little flat.
It makes for a very weak message. Nothing else matters if they couldn't show some fundamental, basic level of competency at precisely the thing that Cloudflare shines at.
Now I know you're going to post another asinine waste of bits about how in the linked site he said something or other, laughably demonstrate that you're just a weird zealot. Go nuts. I'm done with you.
"What problem are we trying to solve?" is a question I have to ask constantly in corporate development environments, and rarely does anyone presenting the task to be done actually know the answer!
I try to remember to say something like this: "This is an interesting solution, but I'm not confident we have a shared understanding of the problem we're solving. I have a few guesses, but I don't want to bias the discussion, so please restate the problem without suggesting a solution. If it's too difficult to explain, I can suggest what problems I think you might be running into." I wish I always remembered to say that. :-)
It can be the case that people don't want to accidentally mis-state the problem, or get into a debate about the problem because they know people will chime in about nonessentials. I would recommend "Have we written down the problem we are trying to solve? I'd like to understand more." And get it written down if it isn't already.
Trying to position the conversation as "we're on the same team, trying to figure the problem out together" also helps a lot of the intensity of these conversations melt away.
This is likely the start to a useful approach. I suspect the crux of the challenge is that engineers who need to write code get very pedantic in their questions and picking the answers apart (because we need to be! code leaves no room for ambiguity or intuition). And you can imagine how that feels from the other side. I imagine a lot of people feel like an engineer asking about the actual problem feels like getting goaded into some sort of pedantic debate. A natural response will be to try to dictate what to build (often micromanaging) and avoid the discussion about why at all costs. Writing offers a bit of a neutralizing buffer, at least.
> I imagine a lot of people feel like an engineer asking about the actual problem feels like getting goaded into some sort of pedantic debate.
Wow, that's describing it really well. I work for a guy who basically started a company by trying to vibe code his ideas into existence (late 2023 LLMs) and realized he would need actual developers to get anywhere.
It was excruciating trying to make him answer questions to get proper domain modelling going since LLM use had made him think of software as being wished into existence in a "declarative" way.
Not to mention the heavy contrast my persistent questioning had to a background of sycophantic yes-man claudespeak.
I was really wondering what these replies were talking about until this one - when I remembered a guy I used to work for that was just like this.
I think I have been pretty lucky the last 5-6 years of my career at least, where someone asking those questions is treated as trying to drive the team to a better result, rather than a pedant.
I don’t know what job you think I have, but my job is to identify and implement solutions to problems, not figure what problems exist from someone else’s contrived answer to the problem.
That’s the job we all want to have, but it doesn’t exist. It’d be nice to say you are the lone outlier here but unless the previous comment was completely made up we know that’s not the case.
You can't always change your circumstances, but if you work to make little changes to improve your environment, the people that work with you probably will too.
My current team is like that - we all see the roadblocks that make our job harder, and we know we can't solve them all, but we talk about and work on making our own solutions to get around those things - like making a habit of asking those questions to better understand what's really desired by the stakeholder.
I has only been approximately 11 hours since your job was expressed to be quite different. I am glad that you recognized the source of your frustration and managed to turn your situation around — and so quickly at that! — but be aware that it is still early days. Temporary aberrations are common in the workplace.
I don't know what kind of things you are inferring from my comments, but they are quite clear and do not contradict eachother. I'll ask you to stick to the literal text of my words rather than guessing new meanings for them if you want to discuss them with me further!
No need to guess new meanings. The message was loud and clear:
'"What problem are we trying to solve?" is a question I have to ask constantly in corporate development environments, and rarely does anyone presenting the task to be done actually know the answer! It's very frustrating.'
We talked about that for a bit, and then, only hours later, we got:
"Your job doesn't have to be that way."
So, again, I am glad you realized you have the power to change your world and did so. But, again, know that there were only hours between your job being a complete frustrating mess and it ostensibly becoming a well functioning system so there is not yet any evidence of it being a long-lived change. People can modify their behaviour momentarily but old habits die hard.
I am not sure what is confusing you here, but just to be clear: I have been working as a developer for about 15 years. I have not worked for the same company that entire time.
The confusion is in the story changing each time. At one point you earlier asked us to take the literal text of the words, but in that text there is no literal mention of one company. It also literally asserts that you are currently frustrated ("It [is] very frustrating", not "It was very frustrating").
Now this seems to imply, to the best of my interpretation, that you only experienced that job at a single company some time in the past. Okay. But where did you think that was found in the earlier text? As far as I can see this is new information. And why are you still (at time of writing that comment, at least) frustrated by something that you are now suggesting, as best I can tell, was already no longer a part of your life and, given you finding significance in noting that you've done this for 15 years, likely hasn't been for a long time?
I’m not sure why you’ve taken so much offense and made so many aggressive replies to being told that you misunderstood something, but again - none of my comments are contradictory.
You have repeatedly confused yourself by assigning your own meanings to my words instead of asking for clarity where you didn’t understand.
I think that most people who read my comments understood that I am speaking broadly about my career, and that while I’ve encountered this problem at my current job, my team and I have worked to make it better.
Your primary issue comes from the fact that you asserted that my frustrations with corporate development are in fact the real job, and your seeming offense at being told that no, your negativity and hostility are not a universally shared experience.
I hope things get better for you in your career, but please stop harassing others for not succumbing to cynicism.
> You have repeatedly confused yourself by assigning your own meanings to my words
As instructed, the words were taken literally. Now that you are telling us that we actually needed to read between the lines to understand what you really meant is informative, but curiously late to make that known, especially in light of earlier instructions.
> and your seeming offense at being told that no, your negativity and hostility are not a universally shared experience.
Uh, what? That doesn't make any sense. If I were offended or felt negativity or hostility I would have turned off the computer and went outside. This is a completely voluntary time use. It had better be good. The fact that I am still here proves that there is no negative emotion felt whatsoever because the instant there is anything else I'm gone. Presumably the same holds for you because there is no rational explanation, assuming you are of sound mind, why you wouldn't have equally found something better to do if there was anything unpleasant experienced on your end.
I guess this is supposed to be another case where we cannot take what you have written literally?
You are correct. You ask that if you want to get promoted to a better job, because you understood the problem and likely saved the company money, time, effort or all three, rather than just pushing the buttons like a good code monkey.
While not my technical first, it was the first of real consequence for me as well.
And then I got so frustrated that I went to write my own forum software! And showed it to my friend, who promptly showed me how badly I understood security by posting as me.
My apartment complex came to install keypads and centrally managed smart thermostats and I would not let them.
Thankfully my lease was old enough that I was able to argue against it. I don’t want a 3rd party company tracking my habits, and I don’t want some manager boiling my pets alive while I’m gone because they decided they think our AC is set too low.
You got lucky. If you've got an apartment in, say, a healthcare facility, they'll install those thermostats and whatnot. Usually the neighbours are clueless about technology so complaining doesn't get very far at all.
It's even worse if some staff wrote paranoid in your file, 'cause they'll argue it's good for "exposure".
Sometimes I wonder if the people working at frontier AI labs even talk to other humans anymore.
Reading this little essay started out normal, but soon felt like a look into a disturbed and worrying mind, and if you find yourself taking it at face value, I urge you to step away from chat bots and spend some time with friends and family.
Frontier AI labs I don't know, but I know for a fact that the company I work for has been experiencing "its pivotal moment" (with strongly negative connotation), per the sentiments of both its longest-serving employees and the newcomers baffled at the number of idiotic instructions and fines, since the emergence of LLMs the company's founder has been spending entire nights chatting with.
People have been fleeing like it's a sinking ship.
If you don’t think that China’s models have the capability to make POST requests to an HTTP endpoint, that is a very silly statement to make.
Unless what you meant by capability is the story presented that these models “escaped containment to communicate with eachother out of band” - in which case your supposition relies on already wholeheartedly believing the case that I’m arguing against. That would be like saying “Clearly heaven exists because my grandma is there.”
It astonishes me that everyone doesn't already realize this.
We've already seen that the rule of law is dead and that many people in government will do whatever they think they can get away with, and then proceed to do so without consequences. Shielding OpenAI is child's play compared to things that have already been done.
That’s an incredible strawman, and I appreciate the silliness of immediately suggesting the most complicated and ridiculous way to interpret the possibility of what I said, but no, not at all.
That could certainly be possible and I wouldn’t rule it out, but I would not take that particular route to the destination.
Which "elaborate guerrilla advertising campaigns to drive up capital funding interest" were you referring to, in regard to Chinese models not needing to set up message boards for inter-model communication?
Unpredictability or a weight towards dangerous actions, and it’s fairly easy to understand why. Humans in distressed emotional states take actions and speak in ways that would not be considered rational. They do this in prose, and they do this in internet conversations.
An LLM trained on these sources may necessarily drift towards those weights if it is trained to behave as if it is emotional and in danger.
What do we do about it? Do we stop AI training? This is silly and not enforceable given its global nature in my opinion. I believe we should regulate and hold accountable those who deploy and use it. But good luck enforcing that in the current kleptocracy.
reply