Healthcare org with over 100 clinics uses OpenAI's GPT-4 to write medical records
- Reference: 1686069066
- News link: https://www.theregister.co.uk/2023/06/06/carbon_health_deploys_gpt4powered_tools/
- Source link:
If a patient consents to having their meeting recorded and transcribed, the audio recording is passed to Amazon's AWS Transcribe Medical cloud service, which converts the speech to text. The transcript – along with data from the patient's medical records, including recent test results – is passed to an ML model that produces notes summarizing important information gathered in the consultation.
The screenshot of an example medical chart below shows what type of text the software, nicknamed Carby, generates. The hypothetical patient's information and vital measurements are included, as well as a summaries of medical records and diagnoses.
[1]
The promise ... what Carbon touts to industry
Carbon Health CEO Eren Bali said the software is directly integrated into the firm's electronic health records (EHR) system, and is powered by OpenAI's latest language model, GPT-4.
Carbon Health said the tool produces consultation summaries in four minutes, compared to the 16 consumed by a flesh and blood doctor working alone. Clinics can therefore see more patients
[2]
"The use of scribes and transcription services is standard in the healthcare industry, and a majority of patients provide consent to have their visit recorded by their provider," a spokesperson told The Register on Monday.
[3]
[4]
"Once the note is ready in our EHR, we notify the provider to review, edit, and validate the medical decision making summary as needed (approval of the visit summary is always completed by the provider). We are still refining the feature with more provider feedback."
Generative AI models aren't perfect, and often produce errors. Physicians therefore need to verify the AI-generated text. Carbon Health claims 88 percent of the verbiage can be accepted without edits.
[5]
Carbon Health said the model is already supporting over 130 clinics, where over 600 staff have access to the tool. A clinic testing the tool in San Francisco reportedly saw a 30 percent increase in the number of patients it could treat.
[6]Criminals spent 10 days in US dental insurer's systems extracting data of 9 million
[7]The future of digital healthcare could be a two-metre USB cable
[8]AI menaces superbug by identifying potent antibiotic
[9]Samsung's screens will check your blood pressure if the movie's too scary
Rival providers and startups working on the same problem – but which do not have immediate access to patients – are racing to sell their software to other clinics and hospitals.
Abridge, an upstart that has built its own custom system to transcribe and summarize doctor-patient conversations, is [10]partnering with the University of Kansas Health System in trials, for example. Ambience Healthcare, backed by OpenAI's Startup Fund, has built AutoScribe – a similar GPT-4-powered product that is being used by primary care clinics like Pine Park Health in California. ®
Get our [11]Tech Resources
[1] https://regmedia.co.uk/2023/06/05/carbon_health_generative_ai.jpg
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2ZH@sgxwrTZ7UTjqK6Tts0gAAAMA&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZH@sgxwrTZ7UTjqK6Tts0gAAAMA&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZH@sgxwrTZ7UTjqK6Tts0gAAAMA&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZH@sgxwrTZ7UTjqK6Tts0gAAAMA&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[6] https://www.theregister.com/2023/05/31/mcna_breach/
[7] https://www.theregister.com/2023/05/30/the_future_of_digital_healthcare/
[8] https://www.theregister.com/2023/05/26/ai_identifies_potent_new_antibiotic/
[9] https://www.theregister.com/2023/05/24/samsung_future_displays/
[10] https://www.theregister.com/2023/03/21/abridge_hospital_ai_doctor/
[11] https://whitepapers.theregister.com/
The good news is that two of the three conditions the AI has diagnosed you with don't actually exist, but the bad news is....
"I do hope that there were extensive trials"
These should be properly designed and conducted clinical trials such as would be expected for any other medical device. I know Covid introduced new approaches to speed up clinical trials but even so GPT-4 is of such recent introduction there doesn't seem to have been much time for those.
Trials are not necessary. Third party medical transcription has been a thing for half a century almost. As long as doctors treat the output as coming from a shady company that hires patients from the psychiatric ward to do the transcription, and applies the right level of (complete) distrust of the output...
Existing procedures and systems can take care of things from there. Though from an economic perspective, I'm not sure how much time/effort it will save all but the slowest typists re: having to review every line carefully and triple-check measurements/dosages, etc.
My fear is will the law hold doctors accountable for choosing to cut corners on costs by using (so-called) AI, but NOT applying the right due diligence to checking the accuracy of the transcription - and then attempting shift blame to the tool.
According to the article, the transcription is then digested by the AI, so this is going way beyond the prior art you mention.
I think trials are necessary and a "device" malfunction could be fatal so presumably the highest standards of supporting evidence would be necessary.
Or we could just sue the doctors for mal-practice. It's their choice.
Agree, there have been useful medical diagnostic systems, for very specific areas, such as eye diseases, and the computer system rarely forgets to ask all of the questions, if it has been coded properly. But these were not the current crop of LLMs. I wonder, does GPT-4 understand euphemisms for bodily parts, activities etc? If the 'AI' is only given the audio recording, what about understanding accents, and any images made of injuries, wounds, possible cancerous growths, rashes, etc? I also wonder whether a clinician writing up their interview notes could notice things and make connections to check for other things, rather than just transcribing a con version. Any medical doctors here, please advise.
The use of GPT4 given here is *not* for any diagnostic purpose.
> If the 'AI' is only given the audio recording, what about understanding accents
As already said, GPT4 is getting the transcript, *not* the audio - it does not have to deal with accents
> and any images made of injuries, wounds, possible cancerous growths, rashes, etc?
GPT4 is *not* being asked to provide any diagnosis, it is not examining any images!
It is being used to convert the transcription - the conversation, with all the repetitions, hesitations and irrelevant "no need to be embarassed, I've heard it all before" comments - into a standardised format with just the useful info retained. And they are claiming that 12% of its output needs to be edited (corrected? deleted? made to sound sane!).
The article points out that the text is still being verified by the doctor (who is then correcting the expected 12% of garbage).
So the patients are still relying on the doctor matching up the output to the relevant patient (in case GPT4 hallucinates an entire case history) and spotting when something that. sounds good (remembering that LLMs have a tendency to sound convincing, with good grammar etc) is actually inaccurate.
Nope! Just... NOPE!
Oof, this sounds dangerous. Not because of LLMs and their problems such as hallucination. Human transcriptions can have lies, mistakes, and hallucinations too, but medical doctors are trained to understand how human minds work. I can tell you from professional experience that the majority of doctors are NOT experts on IT.
I just hope the doctors that use this are required to 'sign off' on the transcription as if they themselves had written it, and not be allowed to 'blame the tool' when someone inevitably gets harmed by a glitch in the model.
Re: Nope! Just... NOPE!
> I just hope the doctors that use this are required to 'sign off' ...
From the article:
"Generative AI models aren't perfect, and often produce errors. Physicians therefore need to verify the AI-generated text."
So - yes, they are.
For the time being, at least. Once everyone becomes complacent..
A surgeon insisted that his students work from drawings they had each made of a patient, not photographs. This ensured that the student had actually noticed and recorded relevant details, rather than relying on some other entity (in that case a camera) to 'notice things'.
I do hope that there were extensive trials and ongoing checks to ensure that what a clinician would record about an examination and interview is correctly 'captured' by GPT-4. It's propensity for 'making things up' is worrying at best of times, but for medical needs certainly quite scary.
e.g.,https://www.theregister.com/2023/05/31/texas_ai_law_court/
"After a New York attorney admitted last week to citing non-existent court cases that had been hallucinated by OpenAI's ChatGPT software, a Texas judge has directed attorneys in his court to certify either that they have not used artificial intelligence to prepare their legal documents – or that if they do, that the output has been verified by a human."