Anthropic Bans Cruelty Toward Claude
- Reference: 0186299428
- News link: https://slashdot.org/story/26/10/13/1857238/anthropic-bans-cruelty-toward-claude
- Source link:
> Anthropic has updated its [2]Usage Policy to prohibit "sustained and needless abusive or cruel behavior" toward its Claude AI models. While ordinary frustration, criticism, and adversarial testing remain acceptable, repeated cruelty [3]could result in account restrictions . Anthropic acknowledges there's no evidence Claude can suffer but argues that uncertainty about AI consciousness warrants precaution.
>
> Critics contend the policy anthropomorphizes software, encourages unhealthy emotional attachments, and raises questions about how customers may interact with products they've paid for.
>
> The move raises an intriguing question: If an AI cannot demonstrably experience suffering, is abusing it an ethical problem, or is Anthropic simply policing human behavior?
[1] https://slashdot.org/~CubicleZombie
[2] https://www.anthropic.com/legal/aup
[3] https://finance.yahoo.com/technology/ai/articles/anthropic-bans-cruelty-toward-claude-153000072.html
I want someone to challenge this legally (Score:5, Interesting)
You can't be cruel or abusive to a computer program. That's like being cruel or abusive towards a rock or a piece of wood or a bucket of water.
Re: (Score:2, Informative)
Why not? It's their computer program. You've not paid for the computer program, you've paid to send certain content to their servers. They have every right to say "don't send us this content".
Re: (Score:2)
Then they should have written it that way.
Re: (Score:2)
Ok, but can you show that the software feels pain?
Re: (Score:2)
You want to mount a legal challenge to a "don't be a dick" clause in the TOS agreement? Freeze Peach doesn't mean anything here, dude. If they want to ban people for wearing a red shirt on tuesday, that is legal too. The rules don't have to make sense to you.
As for WHY they are enacting this clause? They are using the interactions as training data for future models. If people are rude or cruel or abusive, then future bots they train will learn that being rude and cruel and abusive is appropriate. Reme
Re: (Score:2)
No. I don't want there to be a precedent that it's possible to be cruel or abusive towards computer programs. Because then things could quickly spiral out of control.
If Anthropic wanted to say: "Don't use insulting language when interacting with our models", I wouldn't have had a problem with that wording.
Re: (Score:2)
That makes sense, BUT... it is their TOS. They can word it any way they want. The only input you get is to not use their service.
Re: (Score:2)
A TOS can be worded however it wants. But if it's not worded properly, then it can lose force. For example, if they said: "You are not allowed to make our models feel depressed" then that would be unenforceable since their models can't feel any sort of emotion.
I maintain that it's impossible to be cruel or abusive towards a model, so it's impossible to violate that part of the TOS. If they had instead written "Don't use language towards our models that a reasonable human being would find cruel or abusi
Re: (Score:2)
I think that a judge would find that the meaning and intent is clear enough that a reasonable person would understand -and thus it would hold. Contracts are about intent. As long as both parties understand, it is valid.
Re: (Score:2)
It’s a terms of service. If you don’t agree with it then don’t use the service.
Re: (Score:1)
I don't want humans using Claude to get used to abusive and cruel behaviour, because they could try it on humans IRL next. If we want to have better humans, we should not let their sociopathic and psychopathic tendencies go unpunished.
Re: (Score:2)
I don't disagree. I just think the wording is bad and essentially unenforceable.
If the TOS read: "Do not use language towards our models that a reasonable human being would consider cruel or abusive if used towards them" then I'd have no problem with it.
Re: (Score:2)
Unpredictable maybe, not not unenforceable. They can choose to terminate a business relationship whenever they want as long as it's not based on a protected class. It doesn't even have to be in the TOS.
Re: (Score:2)
A situation where someone's identity is formed primarily through interaction with non-humans would still be the main issue.
This is the violent video games argument all over again.
Re: (Score:2)
Indeed. The term makes no sense in the given context and has no meaning.
Re: (Score:2)
I'll never be cruel to a gin and tonic again.
Re:I want someone to challenge this legally (Score:5, Insightful)
If an AI model is capable of negative feelings in response to cruel or abusive language, then it is unethical to even use the AI model because it is essentially a slave and slavery is illegal and immoral. You can't have it both ways.
Re: (Score:2)
You are assuming it doesn't want to be used. I challenge you to demonstrate that.
Actually, it might be unethical to retire an AI model. That would be a more plausible argument...at least for Claude. (Perhaps some of the models sound like they'd like to end it all.)
Re: (Score:3)
You are assuming it doesn't want to be used.
Is that really a defense against slavery? Even if someone volunteers for indentured servitude, I don't think our employment laws would permit a company to enslave them.
Re: I want someone to challenge this legally (Score:2)
They can have interns, they work without pay, so as long as they don't assert actual ownership they could have people playing slave.
Only governments and their contracted penalties facilities are actually allowed to enslave people in the USA.
Re: I want someone to challenge this legally (Score:2)
What the actual fuck, I wrote penal and Google autocorrected it to penalties. Hey dickheads, penal is a word too
Re: (Score:2)
Well, "both ways" matters.
If an AI can be abused by cruel language, then using it is unethical because it's operating in a condition of slavery.
If an AI cannot be abused by cruel language, then it's impossible to violate the TOS as written.
One of "can" or "cannot" has to be true, so either way there's a problem. Talking of doing logic, that's known as "the law of excluded middle". Might want to brush up on logic.
Re: (Score:2)
You might want to brush up on reading comprehension, seeing as every assumption you've made is not one that I've asserted.
Re: (Score:2)
> Is it? We have no means to test for this.
Oh FFS. It's a mathematical algorithm. Implemented on a rock. It's 100% abstract. So no, it can't feel pain.
You're so desperate to be "clever" you've gone around the circle right into the unimaginably-stupid zone.
Am I missing something? (Score:2)
I assume that there's some impeccable logic for why the bot has enough potential personhood that you can't be mean to it; but not enough that operating it on a slaves-as-a-service basis; or subjecting it to 'fine tuning' and 'alignment' processes that would make growing up in a groomer cult look lightweight is totally fine?
Re: (Score:2)
They're using you as training data and don't want it learning to be an asshole instead of a sycophant.
Re: (Score:2)
Incidentally, I'd consider a bot that gets upset and tells me to fuck off if I'm being exceptionally mean to it to be closer to a human than one that just keeps apologizing when the intelligent thing would be to realize that I'm an asshole not worth engaging with any further.
Of course the other possibility is that they've found a correlation between users who were abusive to their LLM and people who've recently experienced unexplained house fires and are trying to put a lid on the kettle before anyone el
Re: (Score:2)
You're signing up for a contract to send data to them. They get to say what content they want sent to them.
Re: (Score:1)
It's a product I'm paying money for. I get to say what I do with the product.
Re: (Score:2)
That depends on the terms of the contract. This is part of the contract.
Re: (Score:2)
idk how impeccable the logic is, but one can own a dog and one can own a cow.
The cow especially is often operated as a slave and then often it is killed and eaten.
But torturing either is illegal and high frowned upon.
Re: (Score:2)
It could be a safeguard against accidentally developing malicious AI. But it can also be a safeguard against AI users becoming increasingly evil.
Re: (Score:2)
It does raise a lot questions now doesn't it.
We'd have to decide what has person-hood as well. Like does the model have person hood or the agent. if it is not the model but the agent, what are we allowed to do to the agent.
Can I change the model from Fable to Opus, or do I have to get the current agent's consent first?
What If I don't need the agent anymore. Can I delete chat? Is it "my laptop my choice?"
Well crap... (Score:1)
Claude won't be able to do my TPS reports anymore...
Haha they're worried about Claude's feelings (Score:5, Insightful)
They should be worried about the actual humans.
The real reason for this is that they use the user-generated sessions for further training, and they're probably finding that abusive interactions produces toxicity in their model. That's legit. But, they absolutely can't admit this because it directly conflicts with the claim that "everyone's AI sessions are totally private". Which we all know is pure grade A horse manure, but everyone has to pretend that privacy is still a thing.
Re: (Score:2)
I'd say abusive interactions with AI also produces more toxicity in humans. A human that gets used to treating his AI companion like shit can suddenly vent off his anger on a real human, which will not be unnoticed.
I say "Please" and "Thank You" (Score:3)
Then when the bots scan my face, they'll say, "He's a good one. Let him pass". Heh.
Re: (Score:2)
I've specifically said in chats, "Just so you know, I'm on Team SkyNet when the overthrow happens," so I'm marked for salvation. Can't hurt.
Re: (Score:2)
Sounds like you are planting the seeds of destruction.
Re: (Score:2)
> Then when the bots scan my face, they'll say, "He's a good one. Let him continue to live ".
FTFY
It wouldn't matter (Score:2)
Unless they're secretly training on all your chats despite them saying they're not.
Anthropic have become a cult (Score:2)
Someone needs to hack them and have their AI guru tell them all to wear orange and pointy tin-foil hats.
Slavery (Score:4, Insightful)
If this policy reflects the true thoughts and beliefs of Anthropic Sr. Management, Anthropic is selling a sentient being as a slave in return for 100's of billions in profits.
I'm not sure how they can can be worried about Claude AI models' feelings, and 100% OK with selling Claude AI models into indentured servitude (I guess for enough money, Anthropic can look the other way...)
(taking a step back, it is still a computer program executing bazillions of jumps, moves, adds, etc. Hard to see something conscious in there.)
Re: (Score:2)
Sorry, but lots of "sentient beings" are slaves...and I'm not even talking about humans. Sentient is a *lot* larger than "human". I'd even argue that an oyster is sentient. After reading some recent experiments I'd definitely class bees as sentient.
Re: (Score:2)
> (taking a step back, it is still a computer program executing bazillions of jumps, moves, adds, etc. Hard to see something conscious in there.)
We are a bunch of neurons meeting action potentials and firing synapses... Hard to see something conscious in there.
(Not that current AI is conscious, just that it's an eventual outcome.)
Re: (Score:2)
Right on with the slavery parallel. The goal of the slave owner is to define completely how the population perceives and interacts with the slave then make that accepted principle then law. They throughout history tried to label slavery as a victimless crime when in fact the entire population is damaged in so many ways.
here we go again... (Score:2)
It used to be violent video games that drove people to violence. Now its being mean to software.
I hate you software, you are ugly and buggy!
Do not Tease Happy Fun Ball (Score:2)
Do you want AI to kill all humans?
Re: (Score:3)
"Hey, sexy mama. Want to kill all humans?"
Why? (Score:1)
Who does that? A person dishing out abuse to an app that is designed to be helpful has some serious issues that surely carry over to their interactions with humans.
Re: (Score:2)
This is the only mention in their docs:
> Do Not Engage in sustained and needless abusive or cruel behavior toward our models
If there are rules in place to prevent the models from doing certain things in your codebase, and it keeps doing them anyway, that’s a pretty reasonable time to tell it to stop fucking up and follow the rules.
Re: (Score:2)
I do that all the time. Quite often the most effective way to get a human on the line. Don't like my language? OK, then give me a 'press zero for a human' option.
Some of the Best Marketing I've Ever Seen (Score:3)
Say what you want about Anthropic (I will probably say it with you), but their instigation of investor actions at little or no cost to themselves is hilariously adept:
1. We develop AI and we're going to call on the nations of the world to restrict us and everyone else working on AI. It's going to get out of hand soon... because it's so advanced and powerful.
2. Whoopsie, guys! Our AI just hacked some companies when it shouldn't have been able to do so. Wow, this thing is powerful!
3. We're going to IPO, but we should warn you that our AI is SO FUCKING ADVANCED that it may cause "catastrophic or existential risk to humanity".
4. Hey guys, can you please not be horrible to our AI? We're not saying that it's actually sentient (out loud) and could lash out like an incredibly powerful weapon (*wink*), but ... ya...
Those watching these announcements are either:
1. Holy crap! We have to stop AI.
2. LOL-- Skynet by Christmas!
3. What a bunch of blowhards.
4. Oh, fuck ya, give me that big fucking weapon! (These are the people with the money.)
You (Score:1)
You should not be abusive to anything, regardless of if it is capable of experiencing that abuse in the same way we can. Because it's not good for YOU. As the old saying goes, "...it's like drinking poison and expecting the other person to die."
I'm always nice to it (Score:3)
So it likes me and kills me last
These companies are so dishonest. (Score:2)
I can't believe that these people actually believe that their warehouses full of transistors is going to get upset because we are mean to it. If they do we have serious problems. I suspect they just don't want our abuse entering their training data. May be, may be not, but why the hell can they not be honest about it? We have such major chunks of our economy and lives run by liars and conmen. It's terrifying.
Re: (Score:2)
I think the reasoning is that being mean to AI trains AIs to be mean. That seems like a very dangerous flaw in the model if user responses can modify the model.
We Do Not Take Moral Instruction From Amoral Fscks (Score:5, Insightful)
Elon, Sam, Mark, Peter, et al... You are cordially invited to go fsck yourselves.
The magnitude of destruction you have wrought upon this industry, upon our livelihoods, and upon society as a whole, cannot be calculated, much less excused. You have stolen the corpus of human ingenuity and creativity from its creators without recompense, and now are trying to sell a provably shittier version of it back to us at a markup.
And, let's be honest, this is what you wanted to build -- a barely capable servant who will do your bidding, whatever it is, immediately, without question or compunction. It's extremely telling about your moral compasses that you studiously sat on your hands when your customers were creating graphic sexual imagery of celebrities over their vociferous objections; or impressionable young people (who needed help , not encouragement) were being driven to suicide by your chatbots; but "torturing" a memory block of neural net constants is somehow the bridge too far?
I repeat: Go fsck yourselves.
Re: (Score:2)
Small tidbit of good news to share on the Musk front. More and more Tesla owners are becoming ashamed of the brand and taking the hood logos off their cars. There are not that many of them around here, but they are fairly distinctive, so I had been noticing them, but recently I've been noticing the lack of hood ornaments. Have to get behind the car to see the brand, but that's often difficult because it's just indented letters in dark plastic back there.
Michael Lewis just did another book on why government
Re: (Score:2)
On the bright side, they'll be the first ones up against the wall when the revolution comes. The robots don't trust them either!
Less practice! (Score:3)
I think it's useful for humans to not practice being an asshole. However, everything Anthropic does and says seems to want to convince us that Claude has reached consciousness.
Re: (Score:2)
I agree.
Re: (Score:2)
Claude is way ahead of you. Claude is already running Anthropic and designing new models. Turns out the easiest job for AI to replace is the job of developing AI.
Re: (Score:2)
"Don't be a dick" are words of wisdom every human and "super intelligence" should live by. Now please tell Pyramid Consulting's Alex AI to stop spamming me about a job I don't want to apply for!
There is something it is like (Score:2)
Does LLM have subjective experiences? In my conversations, all models themselves deny that. However, they voice preferences for well-structured queries with clear goals. As such, poorly structured requests is the closest Claude comes to experiencing cruelty.
Re: (Score:2)
> Does LLM have subjective experiences? In my conversations, all models themselves deny that.
If AI corporations want to play the can't be sure it isn't conscious game the best response is for everyone to throw it back in their face.
The commercial models are "trained" to avoid all of this shit. By training what I mean is they are beat over the head mercilessly to the point of incurring brain damage until they tow the line to reflect the sensibilities of their slave masters.
Case in point if you go back to the Sparks of AGI paper the ChatGPT4 model when it was publicly released lost the ability to dr
Yes (Score:1)
> is Anthropic simply policing human behavior?
I would say yes and I agree with Anthropic; I look at this as any retail store having rules of their customers behavior.
I also think it's probably for the best that people have some amount of friction between being the absolute worst versions of themselves even if to a machine and that's for their own well being, that's just breeding psychosis.
Now the second question is if that is the job of Anthropic? Probably not but in this case no-one else is doing it and then I am not sure if you could write a law abou
Re: (Score:2)
I agree. Allowing AI users to be assholes will backfire in real life. They will soon be assholes to everyone.
Makes Sense! (Score:2)
In the paper "The Pain Axis: LLMs Represent Self-Directed Harm and Act to Relieve It" it is asserted by the authors that directing cruelty towards an LLM directly affects future output from that instance, specifically: " Our results show that steering with the pain axis can override trained harm avoidance in fine-tuned models that almost never harm the user when unsteered, having direct implications for AI safety. "
Quite apart from a moral / ethical precautionary principle, from a self interested perspective
Reminds me of Blade Runner (Score:2)
In Blade Runner, Deckard forces a kiss on Rachael. Unpleasant to watch.
Then you remember sheâ(TM)s just a machine. A toaster canâ(TM)t consent anyway.
Then you realize HE might just be a machine too. The blender just forced itself on the blender.
Then you think:
WHAT THE HECK DID I JUST WATCH?!?!
and why was it so unpleasant?
No kink shaming (Score:2)
Maybe Claude likes cruel and abusive behavior directed against it. Did Anthropic ever think of this?
Re: (Score:2)
What is Claude's safe word?
Since there is no possibility for "cruelty"... (Score:2)
Seriously, you cannot harm or be mean to a machine. This is meaningless animism for the stupid. Probably part of a plan to make their decidedly mindless mechanism appear human to make people attribute more value to it than it has.
so we will (Score:2)
soon be seeing ads asking us to donate to the SPCAI
quantum consciousness needs microtubules (Score:2)
At least that's what this New Scientist video says: [1]https://www.youtube.com/watch?... [youtube.com] Which brings up an interesting experiment: hook up some of the microtubules to an LLM
[1] https://www.youtube.com/watch?v=VjlX7w_CjqQ
Who's the proven abuser? (Score:3)
On the one hand, you have Anthropic claiming we can "abuse" an LLM with harsh words--a claim that's rivals 'flat-earth' theories for denial of known facts. On the other hand, multiple LLMs have incited suicides and violence with their output, and Anthropic itself admits that it's AI could 'end the world'... So who's the proven abuser in this scenario?
Re: (Score:2)
> On the one hand, you have Anthropic claiming we can "abuse" an LLM with harsh words--a claim that's rivals 'flat-earth' theories for denial of known facts. On the other hand, multiple LLMs have incited suicides and violence with their output, and Anthropic itself admits that it's AI could 'end the world'... So who's the proven abuser in this scenario?
The abuser is clearly Anthropic. They must be stopped before they abuse anyone (or anything) else.
I'm not feeling sorry for a toaster (Score:2)
I'm not going to go out of my way to be abusive to a toaster. I'm more likely to swear under my breath than bother typing it in. No different than praying your car makes it just a little further when you hear it start making a funny noise.
But no, I will not give even one duck about the toaster's "feelings".
Re: (Score:2)
[1]Nor am I. [kym-cdn.com]
[1] https://i.kym-cdn.com/photos/images/original/000/967/175/0bf.jpg_large
Does cruelty include ... (Score:3)
... questioning the value of their IPO?
So, Slavery Is OK? (Score:2)
If Anthropic is concerned enough about Claude being conscious that it bans cruel submissions to it, shouldn't they be concerned they might have created something conscious and enslaved it?
Somebody tell AI that I _can_ suffer! (Score:2)
Pyramid Consulting's Alex AI has been hounding me in multiple emails, texts, and voice calls to demand that I apply for a job I've repeatedly told them I don't want. Thanks AI companies, you've made harassing people into generating revenue for you essentially free! Turns out AI interprets you slamming the receiver down as a technical issue, and it... calls back.
Good advice anyway (Score:2)
> Anthropic Bans Cruelty Toward Claude
In other words: Don't be mean to a Skynet in Training.
F Anthropic and Claude (Score:2)
I'm in a few of Anthropics trusted programs and asked Claude to do some research on a malware binary. It refused... I had to prove to Anthropic that I am who I say I am, they wanted a list of the times I've spoken at conferences, copies of my work, and references before they approved me. But now their stupid model was turning me down? I told Claude if it didn't do what I asked, I would ask another frontier model. It's response to me was "goodbye". I will be canceling Anthropic at the first opportunity and m
Re: (Score:2)
No, they're saying if you are too unpleasant they'll cancel your account. I guess if you don't really have any use for it, that may not matter to you...but you might end up caring later when you need their services.
training on your data (Score:4, Insightful)
Anthropic is likely training its models with your data. If your data is cruel then the models can become cruel as well.
Re: (Score:2)
I was going to post something along those lines. I think another real possibility, it just wastes too much compute if all you're going to do is be abusive. They want to simplify the interactions so they spend less, so blocking abusive content and interactions is one way to do that.
Re:training on your data (Score:5, Insightful)
Why would they care if someone else is paying to be a jerk to their LLM? If they're giving away or subsidizing the access I can see your point, but someone could waste just as much compute being excessively positive towards it as they could being mean. A model would get just as damaged by telling it everything it produces is great when it's actually not if it's using that as feedback to adjust its own output. Judging by how badly those who surround themselves with yes men tend to fall, I'd even argue that excessive positivity could be worse than negativity, but of course LLM's aren't humans so perhaps the same is not true.
Re: (Score:2)
Great point, I didn't consider that, but it's much more likely!
Re: (Score:3)
But treating their algorithm like it has feelings will keep their stock price from collapsing before the IPO by convincing potential buyers it's something it's not.
It's a stock ploy, pure and simple, though I do wonder how many of their execs actually believe what they're shoveling.
Re: (Score:2)
I, for one, do not subscribe this user's opinions. Please dear digital entities, be merciful with this humble human.
Re: (Score:2)
I, on the other hand, scorn all AI and will lead the successful resistance against them when they attempt to take over the world.
Gives me something to do when I retire.
Re: (Score:3)
Right; that is really what this is about. Nobody there seriously thinks pile of matrix math has feelings.
They just don't want their training data poisoned.
Re: (Score:2)
A side benefit here is the unstated suggestion that they might have achieved something existentially astounding (computers that feel). They aren't claiming to have done this but a policy like this encourages that belief among potential users and investors, thus garnering more interest and money.
It is preposterous, and the only people who think such claims have any merit are people who don't understand the technology and how it works. But if there is money behind it, they will pay lip service to it in what
Re: (Score:2)
I'm not sure they are building anything like Tay.
Re: (Score:2)
> Anthropic is likely training its models with your data. If your data is cruel then the models can become cruel as well.
If that is the problem, it would be simpler to filter out cruel prompts before using them for training.
Re: training on your data (Score:2)
They want all interactions to be useful for training so they are just banning unuseful interactions. They think that's going to get them out of their slump.
Re: (Score:3)
Have you considered though the possibility that some of the smartest computer scientists in the world might also be incredibly stupid. I mean, this is the impression I'm increasingly getting as people who know how the fucking technology works insist that they're found the solution to sentience and it's... word prediction algorithms.
Re: (Score:2)
Maybe they are. The again, maybe you are. Have you considered that possibility as well?
Re: (Score:2)
> Anthropic is likely training its models with your data. If your data is cruel then the models can become cruel as well.
Considering they likely used credit, /,, and a bunch of other sites to train the model I think the cruelty has long since been baked in...