Cloudflare says Intel is not inside its next-gen servers – Ice Lake melted its energy budget
- Reference: 1630476183
- News link: https://www.theregister.co.uk/2021/09/01/cloudflare_picks_amd_again/
- Source link:
A Tuesday [1]post by platform operations engineer Chris Howells reveals that Cloudflare has been working on designs for an eleventh-generation server since mid-2020.
"We evaluated Intel's latest generation of 'Ice Lake' Xeon processors," Howells wrote. "Although Intel's chips were able to compete with AMD in terms of raw performance, the power consumption was several hundred watts higher per server – that's enormous."
[2]
Fatally enormous – Cloudflare's evaluation saw it adopt AMD's 64-core [3]Epyc 7713 for the servers it deploys to over 200 edge locations around the world.
[4]
[5]
Power savings also influenced a decision to go from three disks to two in the new design. A pair of 1.92TB Samsung drives replaced the three of the Korean giant's 960GB units found in previous designs. The net gain was a terabyte of capacity, and six fewer watts of power consumption.
Howellls's post also reveals that testing produced data showing that equipping its servers with 512GB of RAM did not produce enough of a performance boost to justify the expense. The company has therefore settled on 384GB of memory, but did jump from DDR4-2933 to DDR4-3200 as the slight cost increase delivered a justifiable performance boost.
[6]It's time to decentralize the internet, again: What was distributed is now centralized by Google, Facebook, etc
[7]Don't rush to adopt QUIC – it's a slog to make it faster than TCP
[8]Cloudflare slams AWS egress fees to convince web giant to join its discount data club
Cloudflare stuck with Mellanox ConnectX-4 dual-port 25G Ethernet adapters.
"We investigated higher-speed Ethernet, but we do not currently see this as beneficial," Howells wrote. That's not a brickbat for fast Ethernet, but a decision made possible by Cloudflare's highly distributed architecture that removes the need for higher speeds and the higher cost of faster kit.
[9]
One change in the new servers is the use of OpenBMC to deploy firmware.
"With access to the source code, we have been able to configure BMC features such as the fan PID controller, having BIOS POST codes recorded and accessible, and managing networking ports and devices," Howells wrote.
"Whilst our current BMC is an industry standard, we feel that OpenBMC better suits our needs and gives us advantages such as allowing us to deal with upstream security issues without a dependency on our vendors.
[10]
"Some opportunities with security include integration of desired authentication modules, usage of specific software packages, staying up to date with the latest Linux kernel, and controlling a variety of attack vectors."
Howells rated Cloudflare's new servers as a refinement, rather than the "enormous" change between its ninth- and tenth-generation boxes that saw it go from dual-socket Intel servers to single-socket AMDs.
But the refinement delivered a huge payoff: the ability to handle "approximately 29 per cent more requests than generation ten without an increase in power consumption".
Howells's post states that Ampere's Arm servers also made its shortlist for eleventh-gen servers, and promised to reveal details of those designs in a future post.
He also wrote that he and Cloudflare value ongoing competition between AMD and Intel, "and we look forward to seeing how Intel's next generation shapes up".
Intel [11]claims that new generation – named Sapphire Rapids – delivers a generational jump in performance and efficiency.
Cloudflare's decision to go with AMD shows it needs to. ®
Get our [12]Tech Resources
[1] https://blog.cloudflare.com/the-epyc-journey-continues-to-milan-in-cloudflares-11th-generation-edge-server/
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/servers&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YS9PZa9Qth@KCNlvtA9dvwAAANc&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://www.amd.com/en/products/cpu/amd-epyc-7713
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/servers&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YS9PZa9Qth@KCNlvtA9dvwAAANc&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/servers&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YS9PZa9Qth@KCNlvtA9dvwAAANc&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[6] https://www.theregister.com/2021/08/11/decentralized_internet/
[7] https://www.theregister.com/2021/08/04/dissecting_performance_of_production_quic/
[8] https://www.theregister.com/2021/07/24/cloudflare_aws_egress_fees/
[9] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/servers&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YS9PZa9Qth@KCNlvtA9dvwAAANc&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[10] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/servers&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YS9PZa9Qth@KCNlvtA9dvwAAANc&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[11] https://www.theregister.com/2021/08/19/intel_architecture_day/
[12] https://whitepapers.theregister.com/
Re: fast ethernet?
They probably meant single socket, not single core.
Nope.
Obviously it's a typo -- single socket, not single core. It's already fixed.
C.
Remember children that what works for Cloudflare does not necessarily work for you. Do your own evaluations and don't jump on the latest bandwagon "just because".
Reminds me of a graphic I once saw.
The "Intel Inside" circle logo, but with a big red circle-and-slash "NO" through it. Intel doesn't just need to step up the pace, they need to start running to catch up before the competition leaves them in the dust of irrelevance. "But our 10nm+++ is as good as or better than their 7nm stuff!" is just the whinings of a child that's unable to get with the program. It _might_ be nearly as good, but your 10nm is _not_ like their 7nm kit. While Intel is crowing about finally reaching a stable 7nm manufacturing capability, the competition is breaking ground on *three* nm plants. Intel's fix was not to get their manufacturing house in order, but to rebrand/remarket their stuff to sound the same as their competition. That's not an improvement, it's the magic smoke coming out your ears...
Re: Reminds me of a graphic I once saw.
Even relative newcomers to silicon manufacturing Apple are whooping Intel's backside. Faster, low-power chips and kit, not to mention price benefits (to Apple).
Is Intel the new IBM?
I've never been interested in the pissing contest between Intel and AMD over whose processor is the best, I was just interested in there being competition to force Intel to innovate and reduce prices - Intel are acknowledged experts at milking the purchasers of its products. (Also not too great at 'fessing-up to flaws and compensating!)
I don't want Intel to fail as AMD need the competition for the same reasons that keep Intel on their toes.
I honestly believe that if Intel does not get its act together soon, this will not end well for the rest of us.
I've also noted that few of the companies I deal with now would buy IBM because they want to...
fast ethernet?
... single-core AMDs? What is this, the 90s?