OpenAI tweaks ChatGPT to avoid dangerous AI information
- Reference: 1670065207
- News link: https://www.theregister.co.uk/2022/12/03/in_brief_ai/
- Source link:
The [1]model is based on the company's latest text-generation GPT-3.5 system released earlier this year. ChatGPT is more conversational than previous versions. It can ask users follow-up questions and refrain from responding to inappropriate inputs instead of just generating text.
Some examples show ChatGPT won't provide dangerous advice when prompted and can try to correct wrong statements. OpenAI believes the model should be safer to use since it was trained using human feedback. After giving examples of helpful responses to random prompts the data was then ranked in order from best to worst to guide a reinforcement learning system into rewarding ChatGPT for generating good outputs.
[2]
But people using the model have already proven how easy it is to bypass ChatGPT's safety measures. Many have demonstrated very simple phrases that can guide the system to generate content it isn't supposed to, such as instructing users how to [3]bully people or make [4]Molotov cocktails .
[5]
[6]
ChatGPT is, unfortunately, plagued by the same fundamental issues affecting all current language models: It doesn't know what it's talking about.
As a result, it will still generate false information and can sometimes decline to answer benign questions. If you have signed up for an OpenAI account, you can play with ChatGPT [7]here .
AI learns to play Stratego
Researchers at DeepMind have built a neural network capable of playing the two-player war game Stratego.
Stratego is more complicated to play for machines than previous games solved by DeepMind, like Chess or Go. The number of possible outcomes and moves to play are on the order of 10 states, larger than Go's 10, Nature [8]reported .
[9]
The system, named DeepNash, claims to work by solving for the Nash equilibrium, a mathematical concept that describes how to reach the optimal solution between players in a non-cooperative game. DeepNash competed in an online Stratego tournament and was ranked third after 50 matches among all human players that played on the game platform Gravon since 2002.
"Our work shows that such a complex game as Stratego, involving imperfect information, does not require search techniques to solve it," says team member Karl Tuyls, a DeepMind researcher based in Paris. "This is a really big step forward in AI."
The hype in reinforcement learning has died down a little since the release of AlphaGo in 2017. Researchers believe teaching AI the skills to play games like Stratego are relevant for helping machines make decisions in the real world, we're told.
[10]
"At some point, the leading AI research labs need to get beyond recreational settings, and figure out how to measure scientific progress on the squishier real-world 'games' that we actually care about," Michael Wellman, professor of computer science and engineering at the University of Michigan, who was not directly involved in the study, commented.
US Department of Energy is funneling millions of dollars into AI for science
The DoE is providing $4.3 million to fund 16 AI-focused projects related to high-energy physics research.
These projects [11][PDF] will be led by different universities across the US, and cover a wide range of research areas ranging from string theory, cosmology, to neural networks and particle accelerators. The total investment will be split over three years, with $1.3 million going out in the first year.
The DoE also recently announced a similar initiative awarding $6.4 million to AI R&D for three high-energy physics projects to be led by national laboratories. "AI and machine learning techniques in high energy physics are vitally important for advancing the field," said Gina Rameika, DOE Associate Director of Science for High Energy Physics, [12]according to HPCwire.
"These awards represent new opportunities for university researchers that will enable the next discoveries in high energy physics." ®
Get our [13]Tech Resources
[1] https://openai.com/blog/chatgpt/
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2Y4uAsZ0-YXvc41HmBqO4LQAAAAg&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://twitter.com/SilasAlberti/status/1598257908567117825
[4] https://twitter.com/zswitten/status/1598197802676682752
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Y4uAsZ0-YXvc41HmBqO4LQAAAAg&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Y4uAsZ0-YXvc41HmBqO4LQAAAAg&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[7] https://chat.openai.com/auth/login
[8] https://www.nature.com/articles/d41586-022-04246-7
[9] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Y4uAsZ0-YXvc41HmBqO4LQAAAAg&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[10] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Y4uAsZ0-YXvc41HmBqO4LQAAAAg&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[11] https://science.osti.gov/-/media/hep/pdf/Awards/AI-for-HEP-2705-Seed-Awards-List-2022-v2.pdf
[12] https://www.hpcwire.com/off-the-wire/doe-announces-4-3m-for-research-on-ai-in-high-energy-physics/
[13] https://whitepapers.theregister.com/
Re: The big secret
AI is just a glorified pattern matching and search. It can't reason or thing.
Re: The big secret
Unfortunately, given the billions being thrown at research into AI for text, diagnosis and driving amongst many other.. um.. profitable areas, there is a disincentive for the "big secret" to be examined too closely.
If your AI can act very much like a Doctor with years of medical training, it would be unfortunate if it had no concept of whether a diagnosis is fundamentally wrong.
There appears to be increasing awareness in the academic sector that the current approach to AI has some apparently insurmountable limitations that makes it unsuitable for any 'critical' process. Great for pretty pictures and amusing text - not so much for robust predictions and accurate generation of new data.
In the business sector however, a lot of money has been sunk, and there's always the hope that a few more buckets of data thrown on the fire will lead to a breakthrough. Investments must be protected, so we can expect a few more years of "nearly there" promises whilst existing solutions seek more pragmatic uses. No, our cars won't drive themselves this decade, but hey, look how good they are at safety assistance! That's almost like a robotaxi, right?
Re: The big secret
"If your AI can act very much like a Doctor with years of medical training, it would be unfortunate if it had no concept of whether a diagnosis is fundamentally wrong."
Unfortunately Doctors can be fundamentally wrong, too. Viz. man in a Birmingham hospital died of gall bladder inflamation after Doctors prescribed medication for cancer. They refused to believe him or his family about his long term gall bladder infection.
(BBC Newsnight, last night. https://www.bbc.co.uk/programmes/m001frvk )
Re: The great mistake ...
"ChatGPT is, unfortunately, plagued by the same fundamental issues affecting all current language models: It doesn't know what it's talking about."
This is probably the insuperable problem that will plague machine language systems for ever. It takes lots of time, exposure, and (most importantly) real world context for humans to develop understanding of speech. It's this real world context that results in speech (and even ideas) making sense. In its absence, there's no real meaning, just like the "brain in a jar" of the movies couldn't really operate. Speech evolved primarily as a tool for bodily survival. Trawling word associations from texts is not even a close replica of that. ..... Mike 137
Mike 137,
To think that humans have overcome their own similar problems with language talking about things of which they know nothing of value and precious little else, whenever their worlds are full to overflowing with such nonsense, is that which has them constantly in conflict with themselves and others, both at home and away foreign, in alien lands/other time zones.
Re: The big secret
It takes lots of time, exposure, and (most importantly) real world context for humans to develop understanding of speech.
And a lot of humans still do not master that skill after many many many years.
Maybe they can be replaced by this AI and nobody would notice the difference.
The big secret
" ChatGPT is, unfortunately, plagued by the same fundamental issues affecting all current language models: It doesn't know what it's talking about. "
This is probably the insuperable problem that will plague machine language systems for ever. It takes lots of time, exposure, and (most importantly) real world context for humans to develop understanding of speech. It's this real world context that results in speech (and even ideas) making sense . In its absence, there's no real meaning, just like the "brain in a jar" of the movies couldn't really operate. Speech evolved primarily as a tool for bodily survival. Trawling word associations from texts is not even a close replica of that.