Micron, SK-Hynix's shipping bandwidth-boosting LPDDR5 for on-device AI
(2023/10/27)
- Reference: 1698361213
- News link: https://www.theregister.co.uk/2023/10/26/micron_skhynix_lpddr5/
- Source link:
Memory vendors Micron and SK Hynix this week began shipping their first LPDDR5 memory modules capable of achieving speeds up to 9,600MT/s.
For reference, that's technically 12 percent faster than the LPDDR5 [1]spec , and between 30-50 percent faster than the memory found in most thin and light notebooks.
That speed translates into higher memory bandwidth, something that's become increasingly [2]important as chipmakers have boosted core counts and embedded ever faster GPUs, neural processing units, and other co-processors into their system on chips (SoCs).
[3]
For instance, with the announcement of Qualcomm's Snapdragon 8 Gen 3 system on chip (SoC) this week, the silicon slinger is betting on a future where customers run machine learning and large language models, like Meta's Llama 2 or Stable Diffusion, entirely on their personal devices.
[4]
[5]
Most GPUs and accelerators used to run AI workloads use speedy GDDR or high-bandwidth memory (HBM) modules. However in a slim laptop, tablet, or smartphone this isn't always practical, and the CPU, GPU and other co-processors must often share a common pool of DDR5.
One of the techniques to prevent bandwidth from becoming a bottleneck is co-packaging memory alongside their compute dies. Apple's M-series processors are a prime example of that approach, with the memory modules on the same die as the CPU and co-processors.
[6]
Apple's [7]M2 Max — for the moment, its most powerful notebook SoC — can deliver 400 GB/s of memory bandwidth to the CPU and GPU. To put that in perspective, that's just shy of the 460GB/s of bandwidth AMD's Epyc 4 datacenter CPU can manage when all 12 of its memory channels are full up.
If Apple were to move to Micron or SK-Hynix's latest 9,600 MT/s memory, the company might just be able to eke out another 200GB/s of bandwidth.
[8]SK hynix puts the boot into Kioxia-Western Digital merger
[9]Qualcomm's claims its X Elite PC parts can go toe-to-toe with Apple, Intel
[10]AMD gives 7000-series Threadrippers a frequency bump with Epyc core counts
[11]A cheap Chinese PC with odd components. What could go wrong?
Intel is also [12]rumored to be working on a version of its Meteor Lake processors with on-package LPDDR memory. However, it's not in space constrained mobile devices that we're seeing chipmakers go this route. Nvidia's 144-core Grace CPU Superchip [13]uses LPDDR5X memory to keep the processors fed with 1TB/s of bandwidth.
One of the downsides to LPDDR memory is you can't really upgrade the device by tossing in a higher capacity SODIMM. This isn't really a problem for smartphones and tablets but may be a turn off for prospective laptop buyers. LPDDR modules are designed to be soldered down to the motherboard or co-packaged alongside the SoC, so taking advantage of LPDDR5's higher operating frequencies means forgoing upgradability.
Having said that, we won't have to wait long for [14]SK-Hynix and [15]Micron's latest memory modules hit the market. The companies claim that Qualcomm's Snapdragon 8 Gen 3 will be among the first to support their 9,600 MT/s memory modules. ®
Get our [16]Tech Resources
[1] https://www.jedec.org/news/pressreleases/jedec-publishes-new-and-updated-standards-low-power-memory-devices-used-5g-and-ai
[2] https://www.theregister.com/2022/11/14/amd_intel_nvidia_ram_bandwidth/
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/systems&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2ZTs16wMIZj97aYVHtb0hDAAAAJM&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/systems&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZTs16wMIZj97aYVHtb0hDAAAAJM&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/systems&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZTs16wMIZj97aYVHtb0hDAAAAJM&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/systems&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZTs16wMIZj97aYVHtb0hDAAAAJM&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[7] https://www.theregister.com/2023/01/17/apple_m2_max_pro/
[8] https://www.theregister.com/2023/10/26/sk_hynix_wd_kioxia/
[9] https://www.theregister.com/2023/10/24/qualcomm_x_elite/
[10] https://www.theregister.com/2023/10/20/amd_7000_threadripper_cpus_revealed/
[11] https://www.theregister.com/2023/10/24/kamrui_gk3_plus_review/
[12] https://www.tomshardware.com/news/intel-demos-meteor-lake-cpu-with-on-package-lpddr5x
[13] https://www.theregister.com/2022/03/22/nvidia_reveals_epyccrushing_144core_grace/
[14] https://news.skhynix.com/sk-hynix-lpddr5t-completes-compatibility-validation-with-qualcomm/
[15] https://investors.micron.com/news-releases/news-release-details/micron-collaborates-qualcomm-accelerate-generative-ai-edge
[16] https://whitepapers.theregister.com/
For reference, that's technically 12 percent faster than the LPDDR5 [1]spec , and between 30-50 percent faster than the memory found in most thin and light notebooks.
That speed translates into higher memory bandwidth, something that's become increasingly [2]important as chipmakers have boosted core counts and embedded ever faster GPUs, neural processing units, and other co-processors into their system on chips (SoCs).
[3]
For instance, with the announcement of Qualcomm's Snapdragon 8 Gen 3 system on chip (SoC) this week, the silicon slinger is betting on a future where customers run machine learning and large language models, like Meta's Llama 2 or Stable Diffusion, entirely on their personal devices.
[4]
[5]
Most GPUs and accelerators used to run AI workloads use speedy GDDR or high-bandwidth memory (HBM) modules. However in a slim laptop, tablet, or smartphone this isn't always practical, and the CPU, GPU and other co-processors must often share a common pool of DDR5.
One of the techniques to prevent bandwidth from becoming a bottleneck is co-packaging memory alongside their compute dies. Apple's M-series processors are a prime example of that approach, with the memory modules on the same die as the CPU and co-processors.
[6]
Apple's [7]M2 Max — for the moment, its most powerful notebook SoC — can deliver 400 GB/s of memory bandwidth to the CPU and GPU. To put that in perspective, that's just shy of the 460GB/s of bandwidth AMD's Epyc 4 datacenter CPU can manage when all 12 of its memory channels are full up.
If Apple were to move to Micron or SK-Hynix's latest 9,600 MT/s memory, the company might just be able to eke out another 200GB/s of bandwidth.
[8]SK hynix puts the boot into Kioxia-Western Digital merger
[9]Qualcomm's claims its X Elite PC parts can go toe-to-toe with Apple, Intel
[10]AMD gives 7000-series Threadrippers a frequency bump with Epyc core counts
[11]A cheap Chinese PC with odd components. What could go wrong?
Intel is also [12]rumored to be working on a version of its Meteor Lake processors with on-package LPDDR memory. However, it's not in space constrained mobile devices that we're seeing chipmakers go this route. Nvidia's 144-core Grace CPU Superchip [13]uses LPDDR5X memory to keep the processors fed with 1TB/s of bandwidth.
One of the downsides to LPDDR memory is you can't really upgrade the device by tossing in a higher capacity SODIMM. This isn't really a problem for smartphones and tablets but may be a turn off for prospective laptop buyers. LPDDR modules are designed to be soldered down to the motherboard or co-packaged alongside the SoC, so taking advantage of LPDDR5's higher operating frequencies means forgoing upgradability.
Having said that, we won't have to wait long for [14]SK-Hynix and [15]Micron's latest memory modules hit the market. The companies claim that Qualcomm's Snapdragon 8 Gen 3 will be among the first to support their 9,600 MT/s memory modules. ®
Get our [16]Tech Resources
[1] https://www.jedec.org/news/pressreleases/jedec-publishes-new-and-updated-standards-low-power-memory-devices-used-5g-and-ai
[2] https://www.theregister.com/2022/11/14/amd_intel_nvidia_ram_bandwidth/
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/systems&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2ZTs16wMIZj97aYVHtb0hDAAAAJM&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/systems&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZTs16wMIZj97aYVHtb0hDAAAAJM&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/systems&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZTs16wMIZj97aYVHtb0hDAAAAJM&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/systems&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZTs16wMIZj97aYVHtb0hDAAAAJM&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[7] https://www.theregister.com/2023/01/17/apple_m2_max_pro/
[8] https://www.theregister.com/2023/10/26/sk_hynix_wd_kioxia/
[9] https://www.theregister.com/2023/10/24/qualcomm_x_elite/
[10] https://www.theregister.com/2023/10/20/amd_7000_threadripper_cpus_revealed/
[11] https://www.theregister.com/2023/10/24/kamrui_gk3_plus_review/
[12] https://www.tomshardware.com/news/intel-demos-meteor-lake-cpu-with-on-package-lpddr5x
[13] https://www.theregister.com/2022/03/22/nvidia_reveals_epyccrushing_144core_grace/
[14] https://news.skhynix.com/sk-hynix-lpddr5t-completes-compatibility-validation-with-qualcomm/
[15] https://investors.micron.com/news-releases/news-release-details/micron-collaborates-qualcomm-accelerate-generative-ai-edge
[16] https://whitepapers.theregister.com/