You're right not to rush into running AMD, Intel's new manycore monster CPUs
- Reference: 1728318733
- News link: https://www.theregister.co.uk/2024/10/07/risk_of_manycore_cpus/
- Source link:
That’s because Intel, and AMD, will encourage you to put a lot of eggs in their manycore baskets. I’ve heard both argue that parts such as the [1]72-to-128-core 6900P processor family , the [2]144-core Sierra Forrest Xeon 6 , a promised 288-core monster Xeon, and the imminent [3]192-core Turin Epyc offer the chance for a new round of server consolidation by packing more cores into a single machine.
The chipmakers suggest that replacing your current servers with machines running their monster silicon will free as much as half your rack space and slash your power bills. They are all but evoking a moment in which datacenter ops folk who put this new tech will bask in the glow of a job well done, a planet protected, and a bonus pocketed.
[4]
You don’t have to do this. And if you didn't want to do this, hold firm. Plenty of orgs have standardized on modest hardware and done just fine. But if the boss reads about the chance for a fresh wave of server consolidation in an airline magazine, have them consider a few items.
[5]
[6]
One is the concentration of risk: A manycore box can run so many workloads that its failure would be catastrophic.
Yes, failover to another server is a mature art.
[7]
Making memory remains a less certain process, which is why it's still so expensive. A server running hundreds of cores is going to need huge amounts of RAM to handle all the workloads it runs, and that memory would end up costing more than the server itself.
That might explain why memory-maker Micron is [8]so excited about the prospect of manycore servers driving up demand for its products.
But CFOs won’t be excited if you buy RAM-crammed servers that only ever achieve low utilization rates, so they have enough capacity left for DR duties when your other manycore-equipped servers fall over.
[9]
Next, ponder whether your DR rig is set up to quickly handle 128 cores worth of workload. Failover and VM teleportation tech like VMware’s vMotion remains almost miraculous. But the DR practices you built for your current fleet may not work well when moving more data. Data protection and storage vendors will claim they’re ready, but their reference architectures won’t survive contact with the enemy.
Check your software licenses, too. Does your software vendor let you pay for fewer cores than are present in the box you’re using? Some don’t on bare metal or insist on minimum core counts for VMs. You’ll need to plan carefully to ensure these big new boxes don’t complicate licensing.
[10]With Granite Rapids, Intel is back to trading blows with AMD
[11]Chinese server-maker Inspur claims it's on track for better liquid cooling with 'railway sleeper' design
[12]AMD's Victor Peng: AI thirst for power underscores the need for efficient silicon
[13]All the datacenter roadmap updates Intel, AMD, Nvidia teased at Computex
Consider, too, that handling hardware risk at this scale is not a core competency for many organizations.
But I can think of a few for which it is absolutely essential: AWS, Microsoft, Google, Oracle, Alibaba, and a handful of other hyperscalers.
Those orgs can buy servers by the boatload and understand how to make them pay without tying up capital. They’re also masters of resilience and redundancy and have built predicted hardware failure rates into their pricing and plans.
It’s not your job to match them. Nor can the managed service provider who you trust to tend your colo, or small clouds.
Hyperscale clouds are therefore the natural destination for manycore machines, which look less like a new wave consolidation opportunity and more like a current tugging you into the cloud.
And the cloud is an environment we’ve come to understand brings with it cost uncertainty and lock-in risk.
So by all means, join The Register as we gaze in awe at the astounding CPUs coming our way in 2025. Then pick your jaw off the floor and do some real thinking about whether you will ever be ready to operationalize such monsters. If you’re not, that’s fine. And it’s far better to be fine than offline. ®
Get our [14]Tech Resources
[1] https://www.theregister.com/2024/09/24/intel_xeon_6p/
[2] https://www.theregister.com/2024/06/04/intel_xeon_epyc/
[3] https://www.nextplatform.com/2024/06/03/amd-previews-turin-epyc-cpus-expands-instinct-gpu-roadmap/
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/systems&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2ZwRaBArroCZoV3csRxcm6AAAAJM&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/systems&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZwRaBArroCZoV3csRxcm6AAAAJM&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/systems&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZwRaBArroCZoV3csRxcm6AAAAJM&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[7] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/systems&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZwRaBArroCZoV3csRxcm6AAAAJM&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[8] https://www.theregister.com/2024/09/26/micron_q4_2024/
[9] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/systems&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZwRaBArroCZoV3csRxcm6AAAAJM&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[10] https://www.theregister.com/2024/09/24/intel_xeon_6p/
[11] https://www.theregister.com/2024/09/23/inspur_sleeper_liquid_cooling_design/
[12] https://www.theregister.com/2024/08/29/ai_thirst_for_power/
[13] https://www.theregister.com/2024/06/05/chipmakers_computex_roadmaps/
[14] https://whitepapers.theregister.com/
Re: Many cores on power-limited package = poor single-thread performance?
My understanding is that GPUs scale well because they're only doing vector work.
That is "single instruction multiple data", where one instruction is applied to multiple numbers.
Think of it this way - if in a game you move a little away from a light source, you now need to recalculate the lighting. You could do this in a linear manner across the frame - which is what a CPU would have to do. Or you could stuff all the data into a bunch of GPU cores and ask them to calculate the light levels all at once.
The CPU method can only be faster if you lack the bus bandwidth for the GPU approach. That's why - back in the day - 3D cards were never produced for the slower bus types. They can't be, they need high I/O to keep the cores fed.
This has been a highly contrived and incomplete example, but hopefully serves to demonstrate things.
Why do we still have CPUs? GPUs only compute. They rarely do branch evaluation - IF this THEN that. Looking at the results and deciding which branch to take is still the domain of the CPU in most cases. Good luck running an OS on only a graphics card, it lacks the branching logic to make it worthwhile.
A suitable visualisation would be a factory floor full of people at calculating machines - these are the GPU cores. And a manager - or several if you like - controlling what numbers they will be calculating. The manager is of course the CPU core.
And yes, you get diminishing returns by adding managers, and you need better communications if you're to make use of more calculators - or even more managers. But bad managers everywhere have already taught us this. ;-)
Re: Many cores on power-limited package = poor single-thread performance?
GPU branching is rather ridiculous.
It works by executing both branches, but marking the "branch not taken" as not being able to write to anything.
Which of course indicates that loops are even more stupendously expensive, as they have to be faked by unrolling them...
Re: Many cores on power-limited package = poor single-thread performance?
The game is finding the solution with the best combination of single core performance and core count, and that is *really* tricky without spending all sorts of effort wading through the huge list of random alphanumeric CPU models that are options now.
Plus the prices become ridiculous very quickly as you climb the spec sheet.
Last time I played this game the sweet spot gave more cores *and* more single core performance for less money than what was initially specced, but at the cost of having to add other now necessary options into the base box.
If you really want to get into the power limited game it's monster laptops where it becomes comical - very few have enough cooling to run flat out. I've found that 'full fat' luggable desktop replacement machines could in reality outperform their 4 year newer nominally much faster but more compact workstation cousins because they weren't throttling anywhere near as much.
Risk, interesting, but there's more
That "risk" factor is very interesting, needs some further treatment.
On the RAM question I think that's off, RAM should be about constant per-app wherever it runs.
And failover, in my experience, is less than magical, and recovery from failover worse than the original fail.
Certainly these are meant for clouds. And I strongly question whether they gain or lose performance for individual apps. I hope everyone has the sense to go slowly and carefully on these.
And I'm way out of touch for licensing issues, which were a major pain, Microsoft doing only per-core licensing last I looked into it, almost ten years ago, whether they're idle or not. SHOULD offer volume discount, if not just per-processor flat charge to small users, big users get flat pricing anyway.
I've spent a lot of time on performance and scalability issues, it's great to have more cores to use for peak loads, if you can afford to have them sit idle 50% or 90% or 99% of the time. But most people don't even try such fancy stuff, they just hope paying for cores is the last thing they have to think about. Nope, doesn't work that way.
Re: Risk, interesting, but there's more
RAM may be somewhat constant, but everything running on the server needs it.
And it needs power.
Adding all those cores implies more RAM. I'm suddenly reminded of a decade ago when I was speccing a machine that needed (according the vendor's recommendations) half a terabyte of RAM. Which is getting to be a mundane figure today, but back then it had to sit on its own dedicated daughterboard. Which required its own PSU.
My manager balked at the cost and halved the PSUs, thinking he was smart. This of course meant that the failure of a single PSU would kill the whole machine. I have no idea if the hardware team let him get away with that - as he wasn't the best boss, I was gone before the machine actually arrived.
But it did teach me that RAM likes power a lot more than you'd think.
And it scales linearly per Mb. CPU power consumption doesn't quite do that per Mhz, nor does GPU. And storage - whether spinning rust or solid state - doesn't quite scale linearly either.
If you're planning on just doubling the cores for your existing VMs, this isn't a problem. But if you want to run more VMs, it may well be a problem...
Compute in memory is coming for us all.
This minder is a nice resetting of the 'one does not simply refresh into a €€230k cpu' (or four, plus I suppose a proofing oven and/or district heating,) and improved schedulers that get along nicely.
Missing the point
High CPU-core systems excel at scaling a single large task. Sometimes the static resources of an application (code, data tables, cache) is large compared to the dynamic resources (state, buffers, cache, compute, I/O). This is where high core counts are a huge money saver by not replicating static resources. The argument of a painful failure is silly. You should always have spare processing power and graceful fault tolerance.
Lower core systems are best when you have many smaller tasks that don't share static resources.
More cores the merrier. . .
. . . at least for me.
I have a Ryzen Threadripper 3990X 64-Core Processor machine (128 thread), which is getting a little long in the tooth, sitting next to me at the moment happily running Ubuntu. I don't need all those cores all the time but I don't need a car that will get up to 110 MPH 1 without complaining but in both cases it's darned nice to have it when you need it 2 .
But I'm admittedly the lunatic fringe , at least when it comes to computing. . . we won't tell anyone about my driving, now, will we. . .
________________
1 Unprofessional driver on a (somewhat) closed course to paraphrase all the car ads on the telly.
2 Gignudo database jobs and passing a slow moving semi and a RV on US 95 in Nevada, respectively.
Many cores on power-limited package = poor single-thread performance?
I've often been skeptical whether a 200W TDP 16-core CPU can perform as well as a 200W TDP 4-core CPU for single-threads (let alone 64+ cores). I wonder what the maximum "TDP" of a single-core is. It will be more than 200/16 of course, but how much more?
Going to very high core count, that's what GPUs are for, not CPUs?