If you're cautious about using ML and bots at work, that's not a bad idea
(2023/09/22)
- Reference: 1695364094
- News link: https://www.theregister.co.uk/2023/09/22/datagrail_generative_ai/
- Source link:
DataGrail Summit Generative AI is uncharted territory, and those wishing to explore it need to be aware of the dangers, privacy shop DataGrail warned at its summit this week in San Francisco.
Pretty much every business using technology is facing mounting pressure to exploit GenAI – and many fear that if they fall behind the trend, they could lose to more innovative competitors who have built some tool or product that will give them a competitive advantage. But along with its benefits generative AI introduces all sorts of issues that businesses don't yet understand or know how to solve.
Large language models can divulge sensitive, private information swept up in their training data, for example. In order to perform tasks like answering queries, summarizing documents, or writing code, they have to ingest huge amounts of content.
[1]
Law firms, banks, or hospitals will be training models on confidential data containing personal identifiable information, financial details, and health records. How can they protect customer information and their own proprietary intellectual property from accidentally leaking?
[2]
[3]
A developer at Samsung actually fed ChatGPT proprietary source code hoping it could help to find bugs. Executives at Apple, JPMorgan Chase, and Amazon have since banned workers from using ChatGPT and tools like it internally over concerns that the software's creator OpenAI could train on their data and expose their trade secrets.
"It's a little bit like the early days of software security," Alex Stamos, co-founder of the Krebs Stamos Group, a cybersecurity consultancy, and former chief security officer at Facebook, told The Register at the summit.
[4]
"You know, in the early 2000s, you'd have companies that had a line of business apps or even product teams with no centralized security control.
"And we're kind of back there with AI. If you ask the board of directors 'what do you think your risks are?' they have no idea. Or if they ask their executives to tell them what's going on, they can almost never get an accurate answer."
[5]GitHub Copilot, Amazon Code Whisperer sometimes emit other people's API keys
[6]Don't worry, folks. Big Tech pinky swears it'll build safe, trustworthy generative AI
[7]OpenAI pops an enterprise sticker on ChatGPT to give big biz some peace of mind
[8]3 of the most common reasons for analytics & AI project failure
Developers of generative AI models do offer a hope of slightly better security and privacy for enterprises – OpenAI promised to encrypt and not train on text in conversations between its chatbot and ChatGPT Enterprise customers, for example. But this doesn't solve other types of problems with the technology, such as its tendency to generate false information – a phenomenon known as "hallucination."
Joshua Browder, CEO of DoNotPay – a startup that has developed a so-called "robot lawyer" to automatically help consumers do things like contest parking tickets by using AI to draft emails or official complaints – said factual errors will make some services ineffective.
"Who's responsible for the hallucinations? I think that's the biggest thing," he told The Register . "Imagine if you're trying to appeal a charge on your internet bill and [the AI] said there were five service outages when there weren't. If you're using a tool and it lies, who's legally responsible for that? Is it the model provider? Or is it a company using it? Is it the consumer?"
[9]
Worse still, chatbots could end up spewing inappropriate responses that could be toxic or biased, which could damage the reputations of businesses using them for customer interaction. Safety measures like content filters can block swear words or harmful requests, but they don't always work.
"We don't really know what's gonna go wrong with AI yet," Stamos said. "It's like the '90s, when the basic vulnerabilities that exist in software weren't even discovered yet. We would have these situations where every year there'd be a BlackHat talk or one at DEF CON, that would totally revolutionize the industry, because all of a sudden, one person's research into one kind of flaw was found to be applicable to thousands of different products.
"And that's where we are with AI that you can come up with a new way to manipulate the system to do something harmful or break its promises and you can't prepare for that, because nobody knew that that kind of problem existed before.
"The other problem is these systems are so incredibly complicated – very few people understand how they actually work. These things are so non-deterministic, and so complex, that trying to predict what bad things can happen is impossible."
Finally, there are other risks in using generative AI – like copyright concerns. Many tools are trained on data scraped from the internet, and content creators are afraid they could get sued for generating content based on protected works – like books, songs, or art. Earlier this month, Microsoft [10]declared it would defend paying customers if they face any copyright lawsuits for using its Copilot tools.
Content generated by AI cannot be legally protected under current copyright laws in the US. Can a developer patent an invention if it contains code that was generated by a large language model? Coca-Cola recently released a soft drink flavor that was reportedly created by AI. The recipe is a secret – but if another company were able to replicate it, it would it be OK to sell?
These conundrums are difficult to solve. There are too many unknowns right now, and they touch on different departments within organizations that might not be used to working together – like legal, compliance, marketing, sales, and IT. The important thing is to be aware that the hazards are out there – and be ready when they emerge. ®
Get our [11]Tech Resources
[1] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2ZQ1lwyMJvmoYNn7ODZe0ZgAAABE&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZQ1lwyMJvmoYNn7ODZe0ZgAAABE&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZQ1lwyMJvmoYNn7ODZe0ZgAAABE&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZQ1lwyMJvmoYNn7ODZe0ZgAAABE&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[5] https://www.theregister.com/2023/09/19/github_copilot_amazon_api/
[6] https://www.theregister.com/2023/09/12/nvidia_adobe_palantir_ai_safety/
[7] https://www.theregister.com/2023/08/29/openai_chatgpt_enterprise_tier/
[8] https://www.theregister.com/2023/08/16/3_of_the_most_common/
[9] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZQ1lwyMJvmoYNn7ODZe0ZgAAABE&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[10] https://www.theregister.com/2023/09/07/microsoft_copilot_copyright/
[11] https://whitepapers.theregister.com/
Pretty much every business using technology is facing mounting pressure to exploit GenAI – and many fear that if they fall behind the trend, they could lose to more innovative competitors who have built some tool or product that will give them a competitive advantage. But along with its benefits generative AI introduces all sorts of issues that businesses don't yet understand or know how to solve.
Large language models can divulge sensitive, private information swept up in their training data, for example. In order to perform tasks like answering queries, summarizing documents, or writing code, they have to ingest huge amounts of content.
[1]
Law firms, banks, or hospitals will be training models on confidential data containing personal identifiable information, financial details, and health records. How can they protect customer information and their own proprietary intellectual property from accidentally leaking?
[2]
[3]
A developer at Samsung actually fed ChatGPT proprietary source code hoping it could help to find bugs. Executives at Apple, JPMorgan Chase, and Amazon have since banned workers from using ChatGPT and tools like it internally over concerns that the software's creator OpenAI could train on their data and expose their trade secrets.
"It's a little bit like the early days of software security," Alex Stamos, co-founder of the Krebs Stamos Group, a cybersecurity consultancy, and former chief security officer at Facebook, told The Register at the summit.
[4]
"You know, in the early 2000s, you'd have companies that had a line of business apps or even product teams with no centralized security control.
"And we're kind of back there with AI. If you ask the board of directors 'what do you think your risks are?' they have no idea. Or if they ask their executives to tell them what's going on, they can almost never get an accurate answer."
[5]GitHub Copilot, Amazon Code Whisperer sometimes emit other people's API keys
[6]Don't worry, folks. Big Tech pinky swears it'll build safe, trustworthy generative AI
[7]OpenAI pops an enterprise sticker on ChatGPT to give big biz some peace of mind
[8]3 of the most common reasons for analytics & AI project failure
Developers of generative AI models do offer a hope of slightly better security and privacy for enterprises – OpenAI promised to encrypt and not train on text in conversations between its chatbot and ChatGPT Enterprise customers, for example. But this doesn't solve other types of problems with the technology, such as its tendency to generate false information – a phenomenon known as "hallucination."
Joshua Browder, CEO of DoNotPay – a startup that has developed a so-called "robot lawyer" to automatically help consumers do things like contest parking tickets by using AI to draft emails or official complaints – said factual errors will make some services ineffective.
"Who's responsible for the hallucinations? I think that's the biggest thing," he told The Register . "Imagine if you're trying to appeal a charge on your internet bill and [the AI] said there were five service outages when there weren't. If you're using a tool and it lies, who's legally responsible for that? Is it the model provider? Or is it a company using it? Is it the consumer?"
[9]
Worse still, chatbots could end up spewing inappropriate responses that could be toxic or biased, which could damage the reputations of businesses using them for customer interaction. Safety measures like content filters can block swear words or harmful requests, but they don't always work.
"We don't really know what's gonna go wrong with AI yet," Stamos said. "It's like the '90s, when the basic vulnerabilities that exist in software weren't even discovered yet. We would have these situations where every year there'd be a BlackHat talk or one at DEF CON, that would totally revolutionize the industry, because all of a sudden, one person's research into one kind of flaw was found to be applicable to thousands of different products.
"And that's where we are with AI that you can come up with a new way to manipulate the system to do something harmful or break its promises and you can't prepare for that, because nobody knew that that kind of problem existed before.
"The other problem is these systems are so incredibly complicated – very few people understand how they actually work. These things are so non-deterministic, and so complex, that trying to predict what bad things can happen is impossible."
Finally, there are other risks in using generative AI – like copyright concerns. Many tools are trained on data scraped from the internet, and content creators are afraid they could get sued for generating content based on protected works – like books, songs, or art. Earlier this month, Microsoft [10]declared it would defend paying customers if they face any copyright lawsuits for using its Copilot tools.
Content generated by AI cannot be legally protected under current copyright laws in the US. Can a developer patent an invention if it contains code that was generated by a large language model? Coca-Cola recently released a soft drink flavor that was reportedly created by AI. The recipe is a secret – but if another company were able to replicate it, it would it be OK to sell?
These conundrums are difficult to solve. There are too many unknowns right now, and they touch on different departments within organizations that might not be used to working together – like legal, compliance, marketing, sales, and IT. The important thing is to be aware that the hazards are out there – and be ready when they emerge. ®
Get our [11]Tech Resources
[1] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2ZQ1lwyMJvmoYNn7ODZe0ZgAAABE&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZQ1lwyMJvmoYNn7ODZe0ZgAAABE&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZQ1lwyMJvmoYNn7ODZe0ZgAAABE&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZQ1lwyMJvmoYNn7ODZe0ZgAAABE&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[5] https://www.theregister.com/2023/09/19/github_copilot_amazon_api/
[6] https://www.theregister.com/2023/09/12/nvidia_adobe_palantir_ai_safety/
[7] https://www.theregister.com/2023/08/29/openai_chatgpt_enterprise_tier/
[8] https://www.theregister.com/2023/08/16/3_of_the_most_common/
[9] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZQ1lwyMJvmoYNn7ODZe0ZgAAABE&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[10] https://www.theregister.com/2023/09/07/microsoft_copilot_copyright/
[11] https://whitepapers.theregister.com/
Re: "Who's responsible for the hallucinations?"
Doctor Syntax
"Go get a lawyer to do the job."
We've already had the case of a lwyer using an AI to write his documents. It didn't go well because of hallucinations.
Generative AI has Charts for Charters and Chapters Championing Uncharted Territories
amanfromMars 1
"We don't really know what's gonna go wrong with AI yet," Stamos said.
Methinks you have no other choice than to accept, if you haven't already realised, that AI and ITs Virtual Machine Pioneers give not a jot about that ethereal concern, as they race way out ahead of opposition and competition at the front of novel leading progress justly earning and enjoying all the emerging and expanding benefits, whilst also likely trying to resist all the iffy temptations which are so readily available via any of those platforms happily designed to churn out extremely excessive reward for a place for their snouts in the trough of otherworldly profit [money for nothing].
Strangely enough, does that not suggest and support AI as having hacked/cracked a Sub-Prime Primary Human Driver Code ...... with such human-centric code being for some sad/rad/bad/glad souls, their Prime Primary Driver.
Indeed, such may very well be, for it surely cannot be proven otherwise, also an Emergent Singularity introducing the Human Machine/Machine Human Complex?
"Who's responsible for the hallucinations?"
Only one possible answer : the entity whose name is on the document. If it's a website, the owner of the website.
Stop splitting hairs in two. If a company builds a false-facts-spewing tool and makes it available, the company is responsible for the fallout. If I buy a second-hand vehicle and cause an accident, I can't go say it's the previous owner's fault. Companies that build these hallucinating tools should be responsible for the hallucinations.
Maybe, once the company has been dragged to court and found guilty, it can then turn and slap a lawsuit on the company that built the drunken slob for it, but it's the company that is using the tool that is responsible.
What about someone who uses such a tool to make legal documents for personal situations ? Well don't, first of all. Go get a lawyer to do the job. Second, the individual used a website, so he can attack the owner of the website.
Responsibility is simple in this case. Except for the obvious fact that companies will be dragging their feet as hard as they can, and lobbying like mad to get out of paying for any damages - as they have always done. But that doesn't last. Even the tobacco industry has had to bow in the end. Bil Oil will too, some day. Let's not start messing around with Big AI without laying down the ground rules firmly.