Chan Zuckerberg org to spin up 1,000+ H100 GPU cluster for AI medical research
- Reference: 1695205926
- News link: https://www.theregister.co.uk/2023/09/20/chan_zuckerberg_initiative_to_spin/
- Source link:
The duo believe large language models will be key to understanding what causes disease at the cellular level, and want to help researchers train systems that can predict how cells operate and can change to become pathogenic.
Large language models are computationally intensive to train, and require top-of-the-range hardware to build, test, and tweak effectively. The initiative hopes to set up a cluster of more than 1,000 of Nvidia's H100 GPUs, making it one of the more powerful computing systems for nonprofit life science research in the world, or so we're told.
[1]
Supply for H100s are tight and in high demand right now; even the largest cloud providers have struggled to secure chips for their own servers. A spokesperson for CZI declined to comment to The Register on the amount of funding required to build the cluster, but told us the system should be up and running in 2024.
[2]Can AI transformer models help design drugs and treat incurable diseases?
[3]ZuckerChan cash dump seals first biz gobble: A research paper slurper
[4]Zuckerberg to spend $3bn+ to rid world of all disease by 2100 (Starting with Facebook, right?)
"AI is creating new opportunities in biomedicine, and building a high-performance computing cluster dedicated to life science research will accelerate progress on important scientific questions about how our cells work," Zuckerberg [5]said in a statement.
"AI models could predict how an immune cell responds to an infection, what happens at the cellular level when a child is born with a rare disease, or even how a patient's body will respond to a new medication," Chan, a former pediatrician, added.
Google DeepMind has released a catalog of millions of DNA strings generated by its [6]AlphaMissense AI model detailing genetic mutations that affect the function of proteins in the human body.
The 71 million mutations are designed to help medical researchers deal with illnesses like cystic fibrosis, sickle-cell anaemia, or cancer.
Scientists and engineers at CZI have been experimenting with large language models like ChatGPT to generate descriptions for cells in a [7]virtual encyclopedia that also lists other types of information, such as the genes expressed or tissue it forms. The latest tools they want to build, however, will be more complex and incorporate multiple datasets obtained from lab experiments and microscopic images.
CZI has invested in numerous projects mapping the different types of cells in various organisms, such as mice, fruit flies, mouse lemurs, and humans. It believes AI can be trained to simulate and predict how cellular function is altered in healthy and diseased states to help scientists develop drugs and therapeutics.
[8]
The new AI cluster will be launched by a team of developers working in CZI's CZ Biohub Network HPC team. ®
Get our [9]Tech Resources
[1] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2ZQsXJOA9UKt1AOsBa9AhYwAAAII&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[2] https://www.theregister.com/2022/05/09/ai_drug_design/
[3] https://www.theregister.com/2017/01/24/zuckerchan_cash_dump_first_acquisition/
[4] https://www.theregister.com/2016/09/21/chanzuckerberg_to_rid_world_of_disease/
[5] https://chanzuckerberg.com/newsroom/czscience-builds-ai-gpu-cluster-predictive-cell-models/
[6] https://www.deepmind.com/blog/alphamissense-catalogue-of-genetic-mutations-to-help-pinpoint-the-cause-of-diseases
[7] https://cellxgene.cziscience.com/
[8] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZQsXJOA9UKt1AOsBa9AhYwAAAII&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[9] https://whitepapers.theregister.com/
LLMs for modelling disease?
> large language models will be key to understanding what causes disease at the cellular level
Um, the difference between an LLM and any other large Neural Net is that the input[1] is pre-processed to help pick out features of human language, in textual form[2], and the rest of the code used for training and "playback" is intended to be better at handling that data than any other random data set (cue many, many papers where CompSci bods work to refine these processes for this particular sort of data).
If you want to tackle disease modelling, perhaps use some (equally large) 'Nets, but ones that can be fed something more useful than plain text? Maybe images, some stats about rates of infection, raw feeds from lab instruments (to spot patterns our existing post-processing misses?) etc.
Unless, of course, the people doing all these systems really only know about putting GPU cards into racks and running whatever software they grabbed from AI labs and haven't actually got the knowledge to understand the difference?[3]
[1] and output, although that varies with the goal of the system: e.g. Stable Diffusion versus ChatGPT have different output systems (and the one for SD is yet another 'net, to do the actual image; crudely speaking, of course.
[2] and that processing may well be a.n.other 'net converting speech into text
[3] I've suggested before that these big money projects are more ops guys than research guys, hence why their "safeguards" sound tacked on the front rather than "built-in" to their models.
Re: LLMs for modelling disease?
Here's a [1]link to the source CZI article . LLMs are mentioned only in the third paragraph, which talks in generic terms about "[...] creates a unique opportunity to apply advances in large language models (LLMs) to biomedicine[...]" etc, etc. The rest, and in particular the quotes from Zuck and Chan (and also the video), do not mention LLMs, they feature terms like "AI", "AI models", "generative AI", "AI-driven cellular models", etc. Never forget such articles come through a chain of press officers and journalists who may or may not have a thorough grasp of the subject matter they are covering.
[1] https://chanzuckerberg.com/newsroom/czscience-builds-ai-gpu-cluster-predictive-cell-models/?utm_campaign=fullarticle&utm_medium=referral&utm_source=inshorts£
An aquaintance...
...of mine, let's call him Guy, was driving home from work this past spring. Another driver lost control in the rain and there was a collision. Guy's vehicle was damaged but it wasn't catastrophic. He went to the hospital because he was in pain and expected something minor. They found that his liver was failing. It turns out he had in addition to being bruised in the crash Guy had undiagnosed cancer of the liver. He was told that it would take six to nine months to get a liver transplant. Guy didn't even live six weeks.
How many cancer screenings can you get for 1000 H100s? How many clinicians can you train for the money it costs to run 1000 H100's or a year? The vast majority of illnesses at not just treatable these days, they are curable. When are people in the US going to get real health care and not government mandated health insurance?
When are the people building gigantic computer systems to bang virtual rocks to gather going to notice that there people dying around them every day that could be saved if they had money for treatments and screenings there insurance doesn't cover?