California suggests taking aim at AI-powered hiring software
(2022/04/08)
- Reference: 1649369499
- News link: https://www.theregister.co.uk/2022/04/07/california_ai_hiring/
- Source link:
A newly proposed amendment to California's hiring discrimination laws would make AI-powered employment decision-making software a source of legal liability.
The proposal would make it illegal for businesses and employment agencies to use automated-decision systems to screen out applicants who are considered a [1]protected class by the California Department of Fair Employment and Housing. Broad language, however, means the law could be easily applied to "applications or systems that may only be tangentially related to employment decisions," lawyers Brent Hamilton and Jeffrey Bosley of Davis Wright Tremaine [2]wrote .
Automated-decision systems and algorithms, both fundamental to the law, are broadly defined in the draft, Hamilton and Bosley said. The lack of specificity means that technologies designed to aid human decision-making in small, subtle ways could end up being lumped together with hiring software, as could third-party vendors who provide the code.
[3]
Strict record keeping requirements are included in the proposed law that double record retention time from two to four years, and require anyone using automated-decision systems to retain all machine-learning data generated as part of its operation and training.
[4]
[5]
Training datasets leave vendors responsible, too: "Any person who engages the advertisement, sale, provision, or use of a selection tool, including but not limited to an automated-decision system, to an employer or other covered entity must maintain records of the assessment criteria used by the automated-decision system," the proposed text says. It specifically mentions it must maintain records for each customer it trains models for, too.
A big target
Applicant tracking systems (ATS) and Recruiting management systems (RMS) are used nearly universally, with one 2021 study finding that [6]more than 90 per cent of businesses use such software to rank and filter candidates.
That same study suggests that HR software of the kind covered by the proposed California law is one of the reasons why employers are having trouble filling roles, too. The study concluded that data points often serve as proxies for personal traits that an employer may want to filter out, but personality and CV don't always map perfectly, leading to the exclusion of viable candidates.
[7]Fraud detection system with 93% failure rate gets IT companies sued
[8]Amazon's sexist AI recruiter, Nvidia gets busy, Waymo cars rack up 10 million road miles
[9]It's bizarre we're at a point where reports are written on how human rights trump AI rights
[10]Sure, check through my background records… but why are you looking at my record collection?
Unintentional filtering isn't covered by the newly proposed California law, which focuses on the ways in which software can discriminate against certain types of people, unintentionally or otherwise.
AI and automation tools have had [11]issues with [12]bias for [13]years . California's newly proposed law offers no solutions, and that could leave California businesses grappling with how to react, if at all.
Hamilton and Bosley suggest that California employers review their ATS and RMS software to ensure it conforms to the proposal, enhance their understanding of how the algorithms they use function, be prepared to demonstrate that the results of their process is fair and speak with vendors to ensure they are doing what they need to do to comply.
[14]
The 45-day public commentary period for the proposed changes is not yet open, meaning there's no timetable for the changes to be reviewed, amended and submitted for passage. ®
Get our [15]Tech Resources
[1] https://www.senate.ca.gov/content/protected-classes
[2] https://www.dwt.com/blogs/employment-labor-and-benefits/2022/04/california-ai-employment-laws
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2Yk@zcH6hpSBfxRd375YVAwAAAJM&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Yk@zcH6hpSBfxRd375YVAwAAAJM&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Yk@zcH6hpSBfxRd375YVAwAAAJM&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[6] https://www.theregister.com/2021/09/08/harvard_jobs_report/
[7] https://www.theregister.com/2017/03/08/fraud_detection_system_with_93_failure_rate_gets_it_companies_sued/
[8] https://www.theregister.com/2018/10/13/ai_roundup_131018/
[9] https://www.theregister.com/2021/09/16/human_rights_ai/
[10] https://www.theregister.com/2020/02/28/sure_check_my_background_records/
[11] https://www.theregister.com/2020/06/24/ai_image_tool/
[12] https://www.theregister.com/2021/09/08/harvard_jobs_report/
[13] https://www.theregister.com/2017/05/08/algorithmic_bias/
[14] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Yk@zcH6hpSBfxRd375YVAwAAAJM&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[15] https://whitepapers.theregister.com/
The proposal would make it illegal for businesses and employment agencies to use automated-decision systems to screen out applicants who are considered a [1]protected class by the California Department of Fair Employment and Housing. Broad language, however, means the law could be easily applied to "applications or systems that may only be tangentially related to employment decisions," lawyers Brent Hamilton and Jeffrey Bosley of Davis Wright Tremaine [2]wrote .
Automated-decision systems and algorithms, both fundamental to the law, are broadly defined in the draft, Hamilton and Bosley said. The lack of specificity means that technologies designed to aid human decision-making in small, subtle ways could end up being lumped together with hiring software, as could third-party vendors who provide the code.
[3]
Strict record keeping requirements are included in the proposed law that double record retention time from two to four years, and require anyone using automated-decision systems to retain all machine-learning data generated as part of its operation and training.
[4]
[5]
Training datasets leave vendors responsible, too: "Any person who engages the advertisement, sale, provision, or use of a selection tool, including but not limited to an automated-decision system, to an employer or other covered entity must maintain records of the assessment criteria used by the automated-decision system," the proposed text says. It specifically mentions it must maintain records for each customer it trains models for, too.
A big target
Applicant tracking systems (ATS) and Recruiting management systems (RMS) are used nearly universally, with one 2021 study finding that [6]more than 90 per cent of businesses use such software to rank and filter candidates.
That same study suggests that HR software of the kind covered by the proposed California law is one of the reasons why employers are having trouble filling roles, too. The study concluded that data points often serve as proxies for personal traits that an employer may want to filter out, but personality and CV don't always map perfectly, leading to the exclusion of viable candidates.
[7]Fraud detection system with 93% failure rate gets IT companies sued
[8]Amazon's sexist AI recruiter, Nvidia gets busy, Waymo cars rack up 10 million road miles
[9]It's bizarre we're at a point where reports are written on how human rights trump AI rights
[10]Sure, check through my background records… but why are you looking at my record collection?
Unintentional filtering isn't covered by the newly proposed California law, which focuses on the ways in which software can discriminate against certain types of people, unintentionally or otherwise.
AI and automation tools have had [11]issues with [12]bias for [13]years . California's newly proposed law offers no solutions, and that could leave California businesses grappling with how to react, if at all.
Hamilton and Bosley suggest that California employers review their ATS and RMS software to ensure it conforms to the proposal, enhance their understanding of how the algorithms they use function, be prepared to demonstrate that the results of their process is fair and speak with vendors to ensure they are doing what they need to do to comply.
[14]
The 45-day public commentary period for the proposed changes is not yet open, meaning there's no timetable for the changes to be reviewed, amended and submitted for passage. ®
Get our [15]Tech Resources
[1] https://www.senate.ca.gov/content/protected-classes
[2] https://www.dwt.com/blogs/employment-labor-and-benefits/2022/04/california-ai-employment-laws
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2Yk@zcH6hpSBfxRd375YVAwAAAJM&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Yk@zcH6hpSBfxRd375YVAwAAAJM&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Yk@zcH6hpSBfxRd375YVAwAAAJM&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[6] https://www.theregister.com/2021/09/08/harvard_jobs_report/
[7] https://www.theregister.com/2017/03/08/fraud_detection_system_with_93_failure_rate_gets_it_companies_sued/
[8] https://www.theregister.com/2018/10/13/ai_roundup_131018/
[9] https://www.theregister.com/2021/09/16/human_rights_ai/
[10] https://www.theregister.com/2020/02/28/sure_check_my_background_records/
[11] https://www.theregister.com/2020/06/24/ai_image_tool/
[12] https://www.theregister.com/2021/09/08/harvard_jobs_report/
[13] https://www.theregister.com/2017/05/08/algorithmic_bias/
[14] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Yk@zcH6hpSBfxRd375YVAwAAAJM&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[15] https://whitepapers.theregister.com/
I guess I will have to wait then
Anonymous Coward
Before I publicly comment that they should probably scrap this and start over. Not because the basic idea is bad, just that this is exactly how idiots over-regulate things without actually accomplishing their goals. This proposal is a mountain of bureaucratic paperwork with no teeth to prevent abuse. As sreynolds said, good luck proving this. Having the model data does not mean the regulator will understand any of it. The way it's written it is only useful in a prolonged investigation, which is to say after someones rights have been violated.
Scrap this and replace it with language requiring companies that want to use this kind of technology to show that they understand and can control it's biases BEFORE it's deployed in the first place.
We make them do drug trials, we should make them do something similar for machine learning algorithms.
The burden is as alaways...
Good luck proving this. I guess it depends on how the legislation is written but good luck anyways.