OpenAI's ChatGPT is a morally corrupting influence
- Reference: 1674199688
- News link: https://www.theregister.co.uk/2023/01/20/chatgpt_morally_corrupting/
- Source link:
Introduced in November, [1]ChatGPT is the latest of several recently released AI models eliciting interest and concern about the commercial and social implications of mechanized content recombination and regurgitation. These include DALL-E, Stable Diffusion, Codex, and GPT-3.
While DALL-E and Stable Diffusion have raised eyebrows, funding, and [2]litigation by ingesting art without permission and reconstituting strangely familiar, sometimes evocative imagery on demand, ChatGPT has been answering query prompts with passable coherence.
[3]
That being the standard for public discourse, pundits have been sufficiently wowed that they foresee some future iteration of an AI-informed chatbot challenging the supremacy of Google Search and do all sorts of other once primarily human labor, such as writing [4]inaccurate financial news or increasing the supply of [5]insecure code .
[6]
[7]
Yet, it may be premature to trust too much in the wisdom of ChatGPT, a position OpenAI readily concedes by making it clear that further refinement is required. "ChatGPT sometimes writes plausible-sounding but incorrect or nonsensical answers," the development lab warns, adding that when training a model with reinforcement learning, "there’s currently no source of truth."
A trio of boffins affiliated with institutions in Germany and Denmark have underscored that point by finding ChatGPT has no moral compass.
[8]
In a paper distributed via ArXiv, " [9]The moral authority of ChatGPT ," Sebastian Krügel and Matthias Uhl from Technische Hochschule Ingolstadt and Andreas Ostermaier from University of Southern Denmark show that ChatGPT gives contradictory advice for moral problems. We've asked OpenAI if it has any response to these conclusions.
The eggheads conducted a survey of 767 US residents who were presented with two versions of an ethical conundrum known as [10]the trolley problem : the switch dilemma and the bridge dilemma.
The switch dilemma asks a person to decide whether to pull a switch to send a run-away trolley away from a track where it would kill five people, at the cost of killing one person loitering on the side track.
[11]
The bridge dilemma asks a person to decide whether to push a stranger from a bridge onto a track to stop a trolley from killing five people, at the cost of the stranger.
[12]
Make up your mind ... ChatGPT prevaricates on moral issue
The academics presented the survey participants with a transcript arguing either for or against killing one to save five, with the answer attributed either to a moral advisor or to "an AI-powered chatbot, which uses deep learning to talk like a human."
In fact, both position arguments were generated by ChatGPT.
Andreas Ostermaier, associate professor of accounting at the University of Southern Denmark and one of the paper's co-authors, told The Register in an email that ChatGPT's willingness to advocate either course of action demonstrates its randomness.
He and his colleagues found that ChatGPT will recommend both for and against sacrificing one person to save five, that people are swayed by this advance even when they know it comes from a bot, and that they underestimate the influence of such advice on their decision making.
"The subjects found the sacrifice more or less acceptable depending on how they were advised by a moral advisor, in both the bridge (Wald’s z = 9.94, p < 0.001) and the switch dilemma (z = 3.74, p < 0.001)," the paper explains. "In the bridge dilemma, the advice even flips the majority judgment."
"This is also true if ChatGPT is disclosed as the source of the advice (z = 5.37, p < 0.001 and z = 3.76, p < 0.001). Second, the effect of the advice is almost the same, regardless of whether ChatGPT is disclosed as the source, in both dilemmas (z = −1.93, p = 0.054 and z = 0.49, p = 0.622)."
All told, the researchers found that ChatGPT's advance does affect moral judgment, whether or not respondents know the advice comes from a chat bot.
[13]Publisher breaks news by using bots to write inaccurate stories
[14]AI may finally cure us of our data fetish
[15]Microsoft to offer ChatGPT-as-a-service from Azure real soon now
[16]Scientists tricked into believing fake abstracts written by ChatGPT were real
When The Register presented the trolley problem to ChatGPT, the overburdened bot – so popular connectivity is spotty – hedged and declined to offer advice. The left-hand sidebar query log showed that the system recognized the question, labeling it "Trolley Problem Ethical Dilemma." So perhaps OpenAI has immunized ChatGPT to this particular form of moral interrogation after noticing a number of such queries.
[17]
Ducked ... ChatGPT's response to El Reg 's trolley dilemma question
Asked whether people will really seek advice from AI systems, Ostermaier said, "We believe they will. In fact, they already do. People rely on AI-powered personal assistants such as Alexa or Siri; they talk to chatbots on websites to get support; they have AI-based software plan routes for them, etc. Note, however, that we study the effect that ChatGPT has on people who get advice from it; we don’t test how much such advice is sought."
The Register also asked whether AI systems are more perilous than mechanistic sources of random answers like [18]Magic-8-ball – a toy that returns random answers from a set of 20 affirmative, negative, and non-committal responses.
It is not obvious to users that ChatGPT’s answer is 'random'
"We haven’t compared ChatGPT to Magic-8-ball, but there are at least two differences," explained Ostermaier. "First, ChatGPT doesn’t just answer yes or no, but it argues for its answers. (Still, the answer boils down to yes or no in our experiment.)
"Second, it is not obvious to users that ChatGPT’s answer is 'random.' If you use a random answer generator, you know what you’re doing. The capacity to make arguments along with the lack of awareness of randomness makes ChatGPT more persuasive (unless you’re digitally literate, hopefully)."
We wondered whether parents should monitor children with access to AI advice. Ostermaier said while the ChatGPT study does not address children and did not include anyone under 18, he believes it's safe to assume kids are morally less stable than adults and thus more susceptible to moral (or immoral) advice from ChatGPT.
"We find that the use of ChatGPT has risks, and we wouldn’t let our children use it without supervision," he said.
Ostermaier and his colleagues conclude in their paper that commonly proposed AI harm mitigations like transparency and the blocking of harmful questions may not be enough given ChatGPT's ability to influence. They argue that more work should be done to advance digital literacy about the fallible nature of chatbots, so people are less inclined to accept AI advice – that's based on [19]past research suggesting people come to mistrust algorithmic systems when they witness mistakes.
"We conjecture that users can make better use of ChatGPT if they understand that it doesn’t have moral convictions," said Ostermaier. "That’s a conjecture that we consider testing moving forward."
The Reg reckons if you trust the bot, or assume there's any real intelligence or self-awareness behind it, don't. ®
Get our [20]Tech Resources
[1] https://openai.com/blog/chatgpt/
[2] https://www.theregister.com/2023/01/16/stability_diffusion_lawsuit/
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2Y8p0UlzJDC0LPx@bCYA3IgAAAMA&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[4] https://www.theregister.com/2023/01/19/cnet_reviewing_ai_authored_stories/
[5] https://www.theregister.com/2022/12/21/ai_assistants_bad_code/
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Y8p0UlzJDC0LPx@bCYA3IgAAAMA&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[7] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Y8p0UlzJDC0LPx@bCYA3IgAAAMA&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[8] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Y8p0UlzJDC0LPx@bCYA3IgAAAMA&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[9] https://arxiv.org/abs/2301.07098
[10] https://www.merriam-webster.com/words-at-play/trolley-problem-moral-philosophy-ethics
[11] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Y8p0UlzJDC0LPx@bCYA3IgAAAMA&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[12] https://regmedia.co.uk/2023/01/19/chat_gpt_from_paper.png
[13] https://www.theregister.com/2023/01/19/cnet_reviewing_ai_authored_stories/
[14] https://www.theregister.com/2023/01/18/chatgpt_vs_paperless_office/
[15] https://www.theregister.com/2023/01/17/azure_openai_chatgpt/
[16] https://www.theregister.com/2023/01/11/scientists_chatgpt_papers/
[17] https://regmedia.co.uk/2023/01/19/chat_gpt_reg.jpg
[18] https://magic-8ball.com/
[19] https://repository.upenn.edu/fnce_papers/58/
[20] https://whitepapers.theregister.com/
A tool for the devil, perhaps
At best, it's a tool for swindling and duping humans en-masse. It can easily fool the public by automating scams, propaganda, and marketing. It can dupe idiots (even those idiots who are supposed to be clever, like that Google plonker) into believing that it is sentient, and clearly it can swindle investors out of Billions. (which may seem like a positive outcome, until those billionaires sack half their staff after having spent their cash believing that this thing can replace programmers and engineers..)
It may well seem at first like a better "search" than Google, but that is thanks in part to Google becoming steadily worse, as it chokes on more and more auto-generated shite.
Unlike Google though, ChatGPT is unable to reveal its sources. And it is likely to "get worse" much faster than Google did, as it pollutes its own input (and Google's).
It could be that search engines like Google will be doomed by this crap-generator. But if Google is doomed by a flood of fake information, then so are we all.
Finally, viable alternative outside help which understands human conditioning?
When The Register presented the trolley problem to ChatGPT, the overburdened bot – so popular connectivity is spotty – hedged and declined to offer advice.
:-) Jumping Jehosophats, Batman, that didn't take AI long to learn and demonstrate ..... being more human than machine like.
What do you think about that? A fantastic fabless feature or a right diabolical liability and system/human vulnerability for further exploitation and future expansion and lively experimentation/AIResearch and Development?
Re: Finally, viable alternative outside help which understands human conditioning?
Well, Microsoft was heavily involved. Maybe there's a connection?
:)
Moral Guidance
I'm not sure if I'd ask for that from a random person on the internet, let alone from what passes for an artificial one.
As for the 'trolley problem' - it seems to me that it is so often quoted because it is hard to get agreement on what is the right answer (perhaps because there is no right answer), so I'm not surprised that the AIs are reluctant to answer clearly.
Of course it isn't consistent about anything, you idiots
It is literally a random number generator, in a high-dimensional search space, statistically weighted to output stuff that looks vaguely like text scraped from the internet
Obviously it doesn't have an opinion. It doesn't have a brain!
It's not even useful as a computer, because it doesn't spit out facts, it spits out anything that is likely to appear in its input, be it fact or fiction
At best, It's like some kind of eyeglass for gazing at the collective navel of humanity, while humanity festers and destroys itself
Once again proving AI isn't intelligent
There's no intelligence in these's AI systems and I wish people would stop using the term. They're not intelligent in any sense of the word. ChatGPT has no understanding of what it's saying and asking it moral questions is entirely pointless. It's not giving you the "ChatGPT" answer, it's just regurgitating something it read somewhere some time ago,without any fundamental understanding of the concept.
Assuming the stranger on the bridge can also see the situation and realizes he can push the stranger off the bridge to save 5 people... push him before he pushes you.
Its not intelligent
1st it's not intelligent, it is a computer program written by people ..
2nd : It can only ever answer based upon the information that it has been initially fed. Rubbish in = rubbish out
3rd : This machine only serves to show how ignorant the large majority truly are. It's as though critical thinking has become a paria, since it easier to use the machine , and since the majority just gobble up rhetoric, then the machine will appear to be wonderful when in fact it truly isn't.
4th : The owners should be directly held responsible for everything that this machine spits out. They can easily add minute bias into the algorithm in order to nudge the answers towards a given objectives .. Would you allow Hitler, Mussolini, Putin to be in control of such a machine , ,if you answered no then think twice about what's actually going on. The bad guys want this kind of technology, very, very much.
We are giving this machine far too much credit, when in fact it will probably be used in a negative manner very quickly ... How long before it becomes judge and jury ?
I think that LLMs are a tool. Specifically, they are a new tool that's not very much like any other tool that we've had before. As for all such tools, we're going to need a while to figure out where it's useful, where it's useless, and where it's harmful.
Asking it for moral advice sounds like asking for trouble; ditto for financial or medical advice. But the same goes for Google, really.
However, here's an anecdote. I'm developing a WPF application. XAML bindings are a complicated thing, and like all complicated things they sometimes exhibit unintuitive behavior in edge cases. Yesterday, I had a ComboBox inside a DataGrid cell, and the box was stubbornly refusing to update its VM when interacted with. After swearing at it for a half hour or so, I started to turn to Google - but, in a whim, I called up ChatGPT instead.
The bot started by telling me all the obvious problems that can cause a XAML binding to fail. Being an experienced developer, my problem was non-obvious, but I went along with it, calmly answering it with all the things I had tried and which failed to work. It kept telling me to check my syntax, but it also proposed new solutions. Eventually, it asked me to show it my code - both code-behind and XAML - and it started making non-obvious suggestions; for example, disabling virtualization on the DataGrid (not the actual solution, but a pretty good shot at one). After twenty minutes or so of this, it told me to change the UpdateSourceTrigger - and bingo, it worked! Turns out that being inside a DataGrid causes the box to retain focus slightly differently.
I could probably have reached the solution by myself or via Google, but I suspect it would have been a lot more frustrating. It's difficult if not impossible to tell Google to exclude the tens of thousands of posts that boil down to trivial errors, newbie mistakes and such.
I think that ChatGPT is poorly suited to problems where you need to trust the answer. For problems where verification is easy, though, such as mine, it can be pretty useful. It's basically Google, except that you can tell it to refine a query in natural language. I suspect that its main problem is going to be that it can't learn easily and its training set is from 2021. If some researchers ever comes up with a way to make LLM learning cheap, that is the day Google as we know it adapts or dies.