Excel recruitment time bomb makes top trainee doctors 'unappointable'
(2023/10/12)
- Reference: 1697102115
- News link: https://www.theregister.co.uk/2023/10/12/excel_anesthetist_recruitment_blunder/
- Source link:
Exclusive Computer errors, bad technology choices, and flawed processes have disrupted the recruitment of trainee anesthetists in England and Wales.
In autumn 2021, candidates seeking their third-level specialist training position (ST3) were looking forward to hearing where they would end up in one of the NHS's most sought-after medical disciplines.
However, the body responsible for their selection and recruitment – the Anaesthetic National Recruitment Office (ANRO) – told all the candidates for positions in Wales they were "unappointable," despite some of them achieving the highest interview scores.
[1]
Only when one of the candidates challenged the decision did ANRO realize its error. A subsequent Significant Incident Review showed a complex and confused approach to using spreadsheets led to the disaster.
[2]
[3]
"The interview scores are stored in an Excel spreadsheet. Each of the seven [UK] recruitment regions creates a separate spreadsheet, but these have no standardised template, naming convention or structure. After being manually amended, all of the various scores are entered into a Master spreadsheet. This is carried out row-by-row and takes several days, likely to be subject to interruptions," the report said.
In the process, a ranking column in the Wales Region Spreadsheet had been wrongly transferred to the Master National Spreadsheet, erroneously appearing as an interview score. After their interviews, candidates were ranked 1 to 24 – with 24 actually being the total number of candidates interviewed in the region. But even the highest possible "interview" score of 24 was much lower than candidates' true scores, and because the candidates had been ranked in order of performance, the best candidates were deemed weakest and vice versa.
[4]
"As a consequence of this all the candidates from the Wales Region did not score highly enough when all candidate scores were ranked nationally and all candidates from the Wales Region were 'unappointable'," the report said.
The report – only published in July following a Freedom of Information request – reveals that poor choice of technology for organization-wide decision-making was compounded by inconsistent practice, including the erratic use of Excel's " [5]VLOOKUP " function designed to transpose data from one spreadsheet or data source to another.
The function was used by some members of the ANRO team to minimize the opportunity for errors in manual cut and paste. But "not all the ANRO Team were using the 'V-LOOKUP' function," the report said.
[6]
"The issue was not identified by the team and therefore not escalated to the Team Leader for further investigation," it added.
Although the regional scoring issue hit Wales, potentially all the candidates across the cohort of 400 could have been affected, as replacements were required for the vacant Welsh positions. ANRO closed the application process while it investigated. It found 35 candidates had been affected.
[7]Test crash dummies: Pearson VUE broke half-way into all-day exam
[8]Building Excel-like UI for Uber's China ops exposed Microsoft calculation quirks
[9]Election Excel blunder declared a 'low point' for Austrian social democracy
[10]Excel Hell: It's not just blame for pandemic pandemonium being spread between the sheets
In attempting to tell candidates about the problems with the scoring system, ANRO then found a bug in the messaging system of Oriel, the recruitment platform from vendor HiCom.
Oh sorry, we meant...
ANRO decided to honor the 10 job offers it had made by mistake and used Oriel to tell the candidates. Unfortunately, a system error in Oriel meant it then erroneously sent that communication to an additional 16 candidates. ANRO decided to honor these 16 additional offers too, and find the candidates posts.
"There was a serious issue with the messaging functionality within the Oriel recruitment system which HiCom initially were unable to confirm was a system fault. It was only when a similar event occurred in a different location at a later date that the cause was identified and changes to the system implemented to prevent a similar issue in the future," the report said.
ANRO was previously part of Health Education England (HEE), which has now been merged with NHS England, the government quango running the NHS in England. The Register contacted NHS England to offer it the opportunity to respond.
Doctors not pleased
Meanwhile, doctors close to the recruitment process have expressed their concerns about the lack of transparency throughout the debacle.
"We need to understand from ANRO and HEE what went wrong and the full scale of any underlying issues, along with assurances of fair and equitable mitigating steps that ANRO is putting in place, clarification on their timescales and assurance of rapid and clear communications with those affected," the Royal College of Anaesthetists said in a 2021 statement.
However, the report uncovering the problems within ANRO's systems was not published until after Richards Marks, a former consultant anesthetist in the NHS, made the FoI request.
Speaking to The Register , he said doctors were expected to be open and honest when they make mistakes, and ANRO should do the same.
"We want to know what they've done to get their act together and how this has been allowed to happen," he said.
Next week, the Royal College of Anaesthetists – a professional accreditation and care standards body – is set to hold an Extraordinary General Meeting where it will "consider whether or not it still has confidence in the leadership and senior management of the ANRO" in a resolution proposed by Dr Marks.
The NHS suffers from a chronic shortage of anesthetists. Last year, the [11]Association of Anaesthetists found a shortfall of 1,400 staffers – or 14 percent of roles – while finding that 25 percent of the specialists currently working for the org plan to leave the NHS in five years. ®
Get our [12]Tech Resources
[1] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2ZSfDw5rIIFjwtG3H24ke3wAAAJY&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZSfDw5rIIFjwtG3H24ke3wAAAJY&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZSfDw5rIIFjwtG3H24ke3wAAAJY&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZSfDw5rIIFjwtG3H24ke3wAAAJY&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[5] https://support.microsoft.com/en-us/office/vlookup-function-0bbc8083-26fe-4963-8ab8-93a18ad188a1
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZSfDw5rIIFjwtG3H24ke3wAAAJY&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[7] https://www.theregister.com/2018/02/20/pearson_vue_test_crash/
[8] https://www.theregister.com/2023/09/19/matt_uber_china_excel_clone/
[9] https://www.theregister.com/2023/06/06/austria_election_excel_blunder/
[10] https://www.theregister.com/2020/10/06/excel/
[11] https://anaesthetists.org/Home/News-opinion/News/Action-must-be-taken-to-help-retain-older-anaesthetists-to-prevent-staff-shortage-of-11-000-by-2040-preventing-more-than-8-million-operations
[12] https://whitepapers.theregister.com/
In autumn 2021, candidates seeking their third-level specialist training position (ST3) were looking forward to hearing where they would end up in one of the NHS's most sought-after medical disciplines.
However, the body responsible for their selection and recruitment – the Anaesthetic National Recruitment Office (ANRO) – told all the candidates for positions in Wales they were "unappointable," despite some of them achieving the highest interview scores.
[1]
Only when one of the candidates challenged the decision did ANRO realize its error. A subsequent Significant Incident Review showed a complex and confused approach to using spreadsheets led to the disaster.
[2]
[3]
"The interview scores are stored in an Excel spreadsheet. Each of the seven [UK] recruitment regions creates a separate spreadsheet, but these have no standardised template, naming convention or structure. After being manually amended, all of the various scores are entered into a Master spreadsheet. This is carried out row-by-row and takes several days, likely to be subject to interruptions," the report said.
In the process, a ranking column in the Wales Region Spreadsheet had been wrongly transferred to the Master National Spreadsheet, erroneously appearing as an interview score. After their interviews, candidates were ranked 1 to 24 – with 24 actually being the total number of candidates interviewed in the region. But even the highest possible "interview" score of 24 was much lower than candidates' true scores, and because the candidates had been ranked in order of performance, the best candidates were deemed weakest and vice versa.
[4]
"As a consequence of this all the candidates from the Wales Region did not score highly enough when all candidate scores were ranked nationally and all candidates from the Wales Region were 'unappointable'," the report said.
The report – only published in July following a Freedom of Information request – reveals that poor choice of technology for organization-wide decision-making was compounded by inconsistent practice, including the erratic use of Excel's " [5]VLOOKUP " function designed to transpose data from one spreadsheet or data source to another.
The function was used by some members of the ANRO team to minimize the opportunity for errors in manual cut and paste. But "not all the ANRO Team were using the 'V-LOOKUP' function," the report said.
[6]
"The issue was not identified by the team and therefore not escalated to the Team Leader for further investigation," it added.
Although the regional scoring issue hit Wales, potentially all the candidates across the cohort of 400 could have been affected, as replacements were required for the vacant Welsh positions. ANRO closed the application process while it investigated. It found 35 candidates had been affected.
[7]Test crash dummies: Pearson VUE broke half-way into all-day exam
[8]Building Excel-like UI for Uber's China ops exposed Microsoft calculation quirks
[9]Election Excel blunder declared a 'low point' for Austrian social democracy
[10]Excel Hell: It's not just blame for pandemic pandemonium being spread between the sheets
In attempting to tell candidates about the problems with the scoring system, ANRO then found a bug in the messaging system of Oriel, the recruitment platform from vendor HiCom.
Oh sorry, we meant...
ANRO decided to honor the 10 job offers it had made by mistake and used Oriel to tell the candidates. Unfortunately, a system error in Oriel meant it then erroneously sent that communication to an additional 16 candidates. ANRO decided to honor these 16 additional offers too, and find the candidates posts.
"There was a serious issue with the messaging functionality within the Oriel recruitment system which HiCom initially were unable to confirm was a system fault. It was only when a similar event occurred in a different location at a later date that the cause was identified and changes to the system implemented to prevent a similar issue in the future," the report said.
ANRO was previously part of Health Education England (HEE), which has now been merged with NHS England, the government quango running the NHS in England. The Register contacted NHS England to offer it the opportunity to respond.
Doctors not pleased
Meanwhile, doctors close to the recruitment process have expressed their concerns about the lack of transparency throughout the debacle.
"We need to understand from ANRO and HEE what went wrong and the full scale of any underlying issues, along with assurances of fair and equitable mitigating steps that ANRO is putting in place, clarification on their timescales and assurance of rapid and clear communications with those affected," the Royal College of Anaesthetists said in a 2021 statement.
However, the report uncovering the problems within ANRO's systems was not published until after Richards Marks, a former consultant anesthetist in the NHS, made the FoI request.
Speaking to The Register , he said doctors were expected to be open and honest when they make mistakes, and ANRO should do the same.
"We want to know what they've done to get their act together and how this has been allowed to happen," he said.
Next week, the Royal College of Anaesthetists – a professional accreditation and care standards body – is set to hold an Extraordinary General Meeting where it will "consider whether or not it still has confidence in the leadership and senior management of the ANRO" in a resolution proposed by Dr Marks.
The NHS suffers from a chronic shortage of anesthetists. Last year, the [11]Association of Anaesthetists found a shortfall of 1,400 staffers – or 14 percent of roles – while finding that 25 percent of the specialists currently working for the org plan to leave the NHS in five years. ®
Get our [12]Tech Resources
[1] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2ZSfDw5rIIFjwtG3H24ke3wAAAJY&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZSfDw5rIIFjwtG3H24ke3wAAAJY&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZSfDw5rIIFjwtG3H24ke3wAAAJY&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZSfDw5rIIFjwtG3H24ke3wAAAJY&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[5] https://support.microsoft.com/en-us/office/vlookup-function-0bbc8083-26fe-4963-8ab8-93a18ad188a1
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZSfDw5rIIFjwtG3H24ke3wAAAJY&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[7] https://www.theregister.com/2018/02/20/pearson_vue_test_crash/
[8] https://www.theregister.com/2023/09/19/matt_uber_china_excel_clone/
[9] https://www.theregister.com/2023/06/06/austria_election_excel_blunder/
[10] https://www.theregister.com/2020/10/06/excel/
[11] https://anaesthetists.org/Home/News-opinion/News/Action-must-be-taken-to-help-retain-older-anaesthetists-to-prevent-staff-shortage-of-11-000-by-2040-preventing-more-than-8-million-operations
[12] https://whitepapers.theregister.com/
Re: Excel for dodgy databases
Headley_Grange
The name for the thing you're describing is "database". Spreadsheets are not databases. You can mess around in Excel to sort of give it the functionality, integrity, security, etc., of a database, but it's still a spreasheet at heart.
"The NHS suffers from a chronic shortage of anesthetists"
Mike 137
Also from a chronic shortage of people even minimally competent in IT at all levels, including centralised decision making.
In our area, all GP practices have changed to appointment booking solely via an online portal, and all the practice web sites appear to use the same template, seemingly devised by the NHS centrally. The snag is that booking an appointment is very hard because there is no clear indication on the web sites of how to access the portal. My GP stated (when eventually contacted with difficulty by phone) that they "would show me" how to access the booking portal. The assumption of course being that a patient in need of potentially urgent care or in pain would still remember what they had been shown maybe months or more earlier.
I used to think that a key attribute of interactive IT was that it should be intuitive to use but apparently I was wrong, even where the implications of failure might be life critical.
Re: "The NHS suffers from a chronic shortage of anesthetists"
Mike 125
> I used to think that a key attribute of interactive IT was that it should be intuitive to use but apparently I was wrong, even where the implications of failure might be life critical.
I used to think that a key attribute of a national health system was to allow me to occasionally... you know, see and talk to, like, an actual doctor. But apparently I was wrong, even where the implications of failure might be life critical.
Re: "The NHS suffers from a chronic shortage of anesthetists"
elsergiovolador
Also from a chronic shortage of people even minimally competent in IT at all levels, including centralised decision making.
There is no shortage of competent IT people. The issue is that they don't want to work for peanuts.
Excel
elsergiovolador
That's how you Excel at your job.
Excel for dodgy databases
Excel is a wonderful tool for a knowledgeable user. But it is a disaster when clerical staff are expected to use it directly from their keyboards.
At the very least there should be an Access screen, or equivalent, which will do some checking on what is typed in. Excel sheets should be in the background, after being professionally designed and at least with data-types to distinguish dates, numbers, and text.