Harmed by a decision made by a poorly trained AI? You should be able to sue for damages, says law prof
- Reference: 1612877590
- News link: https://www.theregister.co.uk/2021/02/09/legal_fines_ai/
- Source link:
In a [1]virtual lecture organised by the University of California, Irvine, Professor Frank Pasquale, of the Brooklyn Law School, described how America's laws should be expanded to hold machine-learning companies to account. The talk was based on an article
[2]PDF
published in the Columbia Law Review.[3]
“Data can have massive adverse impacts, and therefore I think there really should be tort liability for many uses where there has been inaccurate or inappropriate data,” Prof Pasquale said. But in order for victims to build a case against vendors that have been training AI systems in a reckless manner, there has to be some way to obtain and investigate the training data.
Want to let an AI-powered doctor loose on patients? Try slapping a food-label-like sticker on it, says Uncle Sam [4]READ MORE
[5]
For example, he said, initial results from machine-learning research show that the technology is promising in healthcare, and there are numerous studies claiming that machines are as good as, if not better, than professionals at diagnosing or predicting the onset of a range of diseases.
Yet the data used to train such systems is often flawed, Prof Pasquale argued. The datasets can be, for instance, unbalanced, where the samples aren’t diverse enough to represent people of various ethnicities and genders The biases in the datasets are carried forward in the performance of these models; they are often less accurate and less effective for women or people of darker skin, for example. In the worse case scenario, patients could be mistakenly diagnosed or overlooked.
New laws must be passed at the state or federal level to force companies to be transparent about what data their systems have been trained on, and how that data was collected, he state. Next, federal organizations, such as the Food and Drug Administration, the National Institute of Standards and Technology, the Department of Health and Human Services, or Office for Civil Rights, should launch efforts to audit the datasets to analyze their potential biases and effects.
“Such regulation not only provides guidance to industry to help it avoid preventable accidents and other torts. It also assists judges assessing standards of care for the deployment of emerging technologies,” Pasquale wrote in the aforementioned article. He said that regulation should not impede progress and innovation in AI healthcare.
[6]
“Development is really exciting and should be applauded in these areas, I don’t want to lose the lead but [the technology] will only be fair and just if there is tort law,” he concluded. ®
Get our [7]Tech Resources
[1] https://calendar.law.uci.edu/event/artificial_intelligence_law_colloquium_frank_pasquale
[2] https://columbialawreview.org/wp-content/uploads/2019/11/Pasquale-Data_informed_duties_in_AI_development.pdf
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_emergent_tech/artificial_intelligence&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YCK-piJben3CLsDwU8uU@gAAANY&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[4] https://www.theregister.com/2021/01/13/ai_healthcare_regulation/
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_emergent_tech/artificial_intelligence&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YCK-piJben3CLsDwU8uU@gAAANY&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_emergent_tech/artificial_intelligence&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YCK-piJben3CLsDwU8uU@gAAANY&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[7] https://whitepapers.theregister.com/
Theory and Reality
That's all very nice, but you'd need to prove AI was somehow involved, and that its decision resulted from "poor training" and not from a conscious choice of the company (now that's a subtle difference!). The biggest problem is that "poor training" is a very vague and subjective notion, which most of the time will be almost impossible to legally prove.
In short, "snowflake's chance in hell" comes to mind...
Re: Theory and Reality
That all may be true but then it is the developer's responsibility to prove that the technology works as intended before application of the technology as a solution. I don't see why "AI" gets a special dispensation regarding this in comparison to other creations; from medical to the latest gadget, things get tested before introduction lest the consumer sues for anything from false advertising to criminal liability.
The consumer should never be the alpha or even beta-testers. The world tried that before, thousands upon thousands of times - remember Thalidomide??
Re: Theory and Reality
> I don't see why "AI" gets a special dispensation regarding this
Because it's a black box: It's not like it gets a dispensation, it's just so much more difficult to find and prove there is a problem, unless it's utterly obvious. A slight bias for instance will mostly go unnoticed, especially if it matches the bias of the people using/testing that AI. Is that AI "poorly trained", or working as expected? It will depend entirely on your personal opinion on that issue, and IMHO it will be hard to legally prove in court there is a training problem . All you can prove is that the decision doesn't suit you.
Re: Theory and Reality
> not like it gets a dispensation, it's just so much more difficult to find and prove there is a problem
True
> Is that AI "poorly trained", or working as expected?
Well, that's where the law tends to be ahead of technology, because that's not really the question that will be asked once you're looking at court.
Was the outcome equitable?
I.e. if your AI has started flagging 1:20 black person for intimate searches, and let all the whites through, the outcome wasn't equitable.
So as the AI developer, the court would probably (hopefully) find against you.
Even for more nuanced cases, the question is the same.
If your design/training decisions have led to an unreasonable outcome and harmed someone else (harm being financial as well as physical) then the equitable outcome is that you carry some liability for that.
There's no need to show that it specifically was a training problem, only that your system's output led to harm when it should not have. Or, as you put it "All you can prove is that the decision doesn't suit you.", much as a court's decision likely wouldn't suit you very much :)
EDIT:
Just to hammer the point home: the fact that AI is a black box, and you can't really discern why a neural net made the decision it did should not be a problem for society at large. It's very much an issue for the developer to deal with, because it is they that should carry liability when their creation starts causing harm.
That's the basic principle that's followed with almost everything else we produce, so it seems unlikely a court will ultimately consider AI too differently. The alternative is that the purchaser (i.e. the company using it) holds the buck for liability rather than being able to sue the developer. At which point, you've got to ask how much of a customer base you're actually going to have after the first lawsuit.
Re: Theory and Reality
Remember Windows 10?
Re: Theory and Reality
FWIW, a chef's kitchen can be held liable if just 1 utensil has a spec of numerous substances irrespective of training, thus 1 tool with any amount of bugs creates liability. For the chef, there is no "ifs", "ands" or "buts" when health is of concern.
Note to self: Create a "AI kitchen cleaning" program to help restaurants avoid liability (McDonald's will love this).
Re: Theory and Reality
> 1 tool with any amount of bugs creates liability
Yes, but you can prove there is dirt on that tool. You can not prove an AI has been "poorly trained", unless of course it's really blatant.
Unlike your kitchen, you can't assess the inner workings of an AI, once trained it's a black box. You put something in, something comes out, but you don't really know why.
Re: Theory and Reality
Which renders its use in *any* circumstance dubious at best, surely? Unless each and every decision it makes is supervised by a responsible adult...
One sees so many examples (claims) of 'AI performs better than humans' but in what is essentially a statistical issue, what matters are the effects of both false positives and false negatives. What happens, for example, when two AI systems trained on different data give differing results? And of course, there are huge differences in the importance of the decision: e.g. medical tests, or self-driving systems, job screening, or immigration decisions might have more of a knock-on effect than deciding if the washing is finished, or food is cooked.
Computer says no?
Medical training, and Facial Recognition
I cannot help feeling that holding those using AI to account for the decisions is a good idea, but getting them to reveal their training data is fraught with problems, particularly if that data is personal information of other people, such as medical information, or the AI is a military system.
There was an internet petition in the UK recently to get medical textbooks to include examples people of varying skin colour suffering from specific diseases, particularly skin diseases, so that medical students would recognise them. Facial recognition systems were trained generally using white faces (see several other El Reg articles). And there is now a debate on autonomous AI weapons systems 'deciding' for themselves whether to kill a target without human intervention or approval.