No, AI can't tell if you've got COVID-19 by listening to your coughs
- Reference: 1672924509
- News link: https://www.theregister.co.uk/2023/01/05/ai_covid_cough/
- Source link:
Claims that AI could detect the difference in cough sounds between those with and without COVID-19 with up to 98.5 per cent accuracy were first reported in a [2]paper from researchers led by the Massachusetts Institute of Technology. The result led to efforts to build an app powered by the algorithms to provide people with a cheap and easy method to test for the novel coronavirus.
The UK's Department of Health and Social Care even went so far as to award two contracts, collectively worth over £100,000, to Fujitsu to develop the government's so-called "Cough In A Box" initiative in 2021, Politico [3]reported . The software would collect audio recordings of coughs from users to analyse on its COVID-19 app.
[4]
But tests performed by a team of researchers led by the Alan Turing Institute and the Royal Statistical Society found the technology doesn't quite work after all. They collected and examined a dataset of audio recordings from over 67,000 people recruited from the National Health Service's Test and Trace and REACT-1 programs, which asked a random portion of the population to perform and send back nose and throat swabs to test for COVID-19.
[5]
[6]
Participants were asked to record samples of them coughing, breathing and talking as well as the results of their swab tests. Over 23,000 of them had tested positive for the respiratory disease. The team trained a machine learning model on these sounds, comparing them with people's COVID-19 test results to see whether coughs could act as an accurate biomarker.
[7]Palantir's Covid-era UK health contract extended without competition
[8]Tech supply chains brace for impact as China shifts from zero-COVID to rampant COVID
[9]China bins its COVID tracking app
[10]Boffins find COVID changed the way sysadmins work – probably for the worse
"But as we continued to analyse the results, it appeared that the accuracy was likely due to an effect in statistics called confounding – where models learn other variables which correlate with the true signal, as opposed to the true signal itself," [11]explained Kieran Baker, a research assistant at the Alan Turing Institute.
The confounding was due to recruitment bias in the [12]Test and Trace system, which required participants to have at least one symptom in order to take part. The researchers performed more tests grouping participants of the same ages and genders into pairs, with only one of them having COVID-19.
"When we evaluated these models on the matched data, the models failed to perform well, and so we conclude that our models cannot detect a COVID-19 bio-acoustic marker from this data," Baker said.
[13]
Chris Holmes, lead author of the paper released last month, professor of biostatistics at the University of Oxford, and program director for health and medical Sciences at The Alan Turing Institute, said: "Finding new ways to quickly and easily diagnose viruses like COVID-19 is really important to stop its spread. While it's disappointing that this technology doesn't work for COVID-19, it may still work for other respiratory viruses in the future," the UK Authority [14]reported . ®
Get our [15]Tech Resources
[1] https://arxiv.org/abs/2212.08570
[2] https://www.theregister.com/2020/10/31/ai_covid_cough/
[3] https://www.politico.eu/article/uk-ai-coughs-coronavirus-fujitsu/
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2Y7cCL0eMYkdF1bFqE45PVwAAANc&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Y7cCL0eMYkdF1bFqE45PVwAAANc&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Y7cCL0eMYkdF1bFqE45PVwAAANc&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[7] https://www.theregister.com/2023/01/04/palantirs_covidera_uk_health_contract/
[8] https://www.theregister.com/2022/12/19/china_supply_chain_living_with_covid/
[9] https://www.theregister.com/2022/12/13/china_bins_covid_app/
[10] https://www.theregister.com/2022/11/09/sysadmin_work_changed_by_covid/
[11] https://www.turing.ac.uk/blog/can-sound-someones-cough-be-used-detect-covid-19
[12] https://www.theregister.com/2021/09/06/accenture_gets_another_5m_nhs/
[13] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Y7cCL0eMYkdF1bFqE45PVwAAANc&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[14] https://www.ukauthority.com/articles/study-shows-limits-of-coughing-tech-for-covid-19/
[15] https://whitepapers.theregister.com/
Yes, decent research is nothing to be sneezed at. (sorry)
:)
Yes, investigated...
But the original claim already had severe problems passing the sniff test. It was a too good to be true story from the beginning and it seemed to cough up results that were better than the medical profession's collective wisdom.
But hey, AI/ML systems are so good at blinding. They have blinkenlight so the results must be fantastic.
/s(igh)
So basically, the ML didn't work, the statistics were sketchy, and peer review failed to spot the sketchy statistics.
Sadly, a triple-fail.
Weirdly, my AI Engine
predicted this would never work.
But then it never was about working or not working, was it ? It was how to find a(nother) way to shovel money to mates.
There are many reasons why people cough
Covid is just one of them. Listening to some people around here, they sound like they'll be corpses in a matter of days, but they've just been out for (yet another) fag.
That they come back sounding like a plague victim is just their body failing at telling them to stop.
Ironically, the people that I know around here who had Covid tended not to cough. They just ran a fever, felt awful, and looked like they'd pass out any minute. At least before they realised they really ought not stay at work... maybe the coughing comes later?
Re: There are many reasons why people cough
The most annoying thing is the time the cough lasts because it trains your body into coughing for the slightest tickle in your throat afterwards. I discovered that after having to suppress it during video calls (not for the noise, but once it starts it takes a few seconds) - it got easier after a while.
I guess there's truth after all in the use of a strong laxative as cough medicine* :)
* Because you wouldn't dare coughing then..
What a disappointment!
But [1]Simon said ... And he's got a government contract...
[1] https://www.theregister.com/2022/12/23/bofh_2022_episode_24/
Too bad we don't have such efficient tools in politics
" But as we continued to analyse the results, it appeared that the accuracy was likely due to an effect in statistics called confounding – where models learn other variables which correlate with the true signal, as opposed to the true signal itself "
I'm reminded of that chapter in Asimov's Foundation, where the declarations of some diplomat were passed through a logical analysis procedure to conclude that said diplomat had said absolutely nothing of substance or significance.
I still think we need a tool like that.
Re: Too bad we don't have such efficient tools in politics
I believe we're endowed with that tool already. It's honed by listening to Sir Humphrey!
Can you imagine the legislation MPs would wanted pushed through if that tool were proposed for the Commons, or used by broadcasters for politicians' gibberings? Non-partisan cries of "parliamentary privilege"!
"Cough In A Box"
Hey! Let's make some easy money. We can easily show what dicks we are while we're at it, let's mock those that are in mourning, we'll call it "Coffin: A Box", but we'll write it as...
Good to see they properly investigated this method irrespective of the outcome.