SmartNICs power the cloud, are enterprise datacenters next?
(2022/05/20)
- Reference: 1653067966
- News link: https://www.theregister.co.uk/2022/05/20/smartnic_cloud_datacenter/
- Source link:
SmartNICs have the potential to accelerate enterprise workloads, but don't expect to see them bring hyperscale-class efficiency to most datacenters anytime soon, ZK Research's Zeus Kerravala told The Register .
SmartNICs are widely [1]deployed in cloud and hyperscale datacenters as a means to offload input/output (I/O) intensive network, security, and storage operations from the CPU, freeing it up to run revenue generating tenant workloads. Some more advanced chips even offload the [2]hypervisor to further separate the infrastructure management layer from the rest of the server.
Despite relative success in the cloud and a flurry of innovation from the still-limited vendor SmartNIC ecosystem, including Mellanox (Nvidia), Intel, Marvell, and Xilinx (AMD), Kerravala argues that the use cases for enterprise datacenters are unlikely to resemble those of the major hyperscalers, at least in the near term.
[3]
"When cloud was first coming around, the hyperscalers went into their engagements with their customers with the assumption that everybody wanted to run IT like they did," he said. "Businesses aren't hyperscalers. They don't run IT like hyperscalars. They're never going to."
[4]
[5]
Kerravala believes for the SmartNIC market to move beyond the high-performance, efficiency-obsessed world of the cloud, the industry needs to reevaluate how SmartNICs can accelerate enterprise workloads.
No shortage of opportunity
With that said, Kerravala sees a plethora of use cases for SmartNICs in enterprise datacenters and the edge.
SmartNICs are particularly well suited to processor-intensive workloads that involve large quantities of data, he said. "Traditional servers weren't really meant for that kind of overhead."
"There's not a company that I talk to that's not telling me that they've got more data on their hands than ever," he added.
[6]
This, Kerravala argues, makes SmartNICs particularly appealing to the growing number of industries looking for ways to use data to a competitive advantage.
This includes markets you might not expect to need high-speed networking and I/O offload capabilities, like retail, hospitality, and entertainment.
Kerravala highlighted sporting events, where a large number of video feeds are streamed live alongside telemetry on the player's performance, as one example of where SmartNICs could prove especially useful.
[7]Marvell CXL roadmap goes all-in on composable infrastructure
[8]To beat the cloud, datacenter operators must embrace automation, multiple vendors: Juniper VP
[9]Why Marvell bought interconnect upstart Tanzanite
[10]VMware says server sprawl is back, and SmartNICs are the solution
Another area where SmartNICs may be beneficial to enterprise customers is security. Because security is becoming so analytics driven, Kerravala believes SmartNICs have the potential to put security inspection much closer to the data.
Palo Alto Networks has already [11]demoed this functionality by deploying its virtualized firewalls on Nvidia's BlueField-2 SmartNICs.
[12]
Supporting these data-intensive workloads means carefully balancing compute, storage, networking, and security to avoid bottlenecks. "You can't have a fast network and slow storage, just like you can't have fast storage and a slow processor," Kerravala said, adding that letting the CPUs do what they're meant to do rather than having to process network transport or encrypt traffic, will save companies money in the long term.
[13]To beat the cloud, datacenter operators must embrace automation, multiple vendors: Juniper VP
[14]VMware says server sprawl is back, and SmartNICs are the solution
[15]SmartNICs, IPUs, DPUs de-hyped: Why and how cloud giants are offloading work from server CPUs
[16]Intel sticks with FPGAs and ASICs for next-gen IPUs
Barriers to adoption abound
For all the potential SmartNICs have to offer, there remains substantial barriers to overcome. The high price of SmartNICs relative to standard NICs being one of many.
Networking vendors have been chasing this kind of I/O offload functionality for years, with things like TCP offload engines, Kerravala said. "That never really caught on and cost was the primary factor there."
Another challenge for SmartNIC vendors is the operational complexity associated with managing a fleet of SmartNICs distributed across a datacenter or the edge.
"There is a risk here of complexity getting to the point where none of this stuff is really usable," he said, comparing the SmartNIC market to the early days of virtualization.
"People were starting to deploy virtual machines like crazy, but then they had so many virtual machines they couldn't manage them," he said. "It wasn't until VMware built vCenter, that companies had one unified control plane for all their virtual machines. We don't really have that on the SmartNIC side."
That lack of centralized management could make widespread deployment in environments that don't have the resources commanded by the major hyperscalers a tough sell. Most enterprises simply don't have the IT resources, Kerravala argued.
Several companies are working closely with SmartNIC vendors to address these challenges. Juniper, for example, has been working with Intel and Nvidia to extend its [17]Contrail orchestration platform to their SmartNICs. Similarly, VMware plans to address this challenge through its [18]Project Monterey initiative.
"Like most technologies, management of the stuff usually gets developed a couple of years after the stuff," Kerravala said.
Where to invest?
Right now, Kerravala believes enterprise datacenters can benefit from SmartNICs, but should deploy them sparingly to accelerate their highest-demand applications.
"If they've got a very high-performance workload, use it there. I wouldn't deploy it everywhere," he said. "I would be putting them in for low-hanging fruit use cases. Use it for all the new stuff, but leave the legacy stuff the way it is."
He also recommends customers consider fully-integrated platforms that pair software, management, and hardware in a single package.
[19]Fungible is one such example. The company offers a suite of storage and compute appliances built around its embedded data processing unit (DPU) — a trendy name for SmartNICs — and managed via a common software layer.
Until recently, the company has focused much of its attention on high-throughput storage applications, but recently applied the technology to compute pooling with a network-addressable appliance that allows GPU resources to be [20]composed on the fly.
However, long term, Kerravala believes SmartNIC vendors need to provide more than hardware and a loose SDK. He highlighted Nvidia's efforts to build out an ecosystem of both hardware and software libraries necessary to build support into enterprise software.
"That kind of complete systems approach reduces the complexity for the developer, helps to build an ecosystem a lot faster, and really turns what is essentially silicon into a platform," Kerravala said. ®
Get our [21]Tech Resources
[1] https://www.theregister.com/2021/11/24/infrastructure_processing_units/
[2] https://www.theregister.com/2022/04/27/vmware_project_monterey_server_sprawl/
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/networks&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YogPhdHTGBFc@buKP-iDuQAAANE&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/networks&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YogPhdHTGBFc@buKP-iDuQAAANE&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/networks&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YogPhdHTGBFc@buKP-iDuQAAANE&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/networks&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YogPhdHTGBFc@buKP-iDuQAAANE&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[7] https://www.theregister.com/2022/05/18/marvell_cxl_composable/
[8] https://www.theregister.com/2022/05/11/juniper_datacenter_cloud/
[9] https://www.theregister.com/2022/05/10/marvell_tanzanite_cxl/
[10] https://www.theregister.com/2022/04/27/vmware_project_monterey_server_sprawl/
[11] https://blogs.nvidia.com/blog/2021/07/12/palo-alto-networks-cyber-defenses/
[12] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/networks&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YogPhdHTGBFc@buKP-iDuQAAANE&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[13] https://www.theregister.com/2022/05/11/juniper_datacenter_cloud/
[14] https://www.theregister.com/2022/04/27/vmware_project_monterey_server_sprawl/
[15] https://www.theregister.com/2021/11/24/infrastructure_processing_units/
[16] https://www.theregister.com/2022/05/10/intel_ipu_smartnic/
[17] https://www.theregister.com/2022/05/11/juniper_datacenter_cloud/
[18] https://www.theregister.com/2022/04/27/vmware_project_monterey_server_sprawl/
[19] https://blocksandfiles.com/2020/08/18/fungible-inc-dpu-launch/
[20] https://blocksandfiles.com/2022/04/06/ts-graphic-fungible-composes-gpu-as-a-service-offering/
[21] https://whitepapers.theregister.com/
SmartNICs are widely [1]deployed in cloud and hyperscale datacenters as a means to offload input/output (I/O) intensive network, security, and storage operations from the CPU, freeing it up to run revenue generating tenant workloads. Some more advanced chips even offload the [2]hypervisor to further separate the infrastructure management layer from the rest of the server.
Despite relative success in the cloud and a flurry of innovation from the still-limited vendor SmartNIC ecosystem, including Mellanox (Nvidia), Intel, Marvell, and Xilinx (AMD), Kerravala argues that the use cases for enterprise datacenters are unlikely to resemble those of the major hyperscalers, at least in the near term.
[3]
"When cloud was first coming around, the hyperscalers went into their engagements with their customers with the assumption that everybody wanted to run IT like they did," he said. "Businesses aren't hyperscalers. They don't run IT like hyperscalars. They're never going to."
[4]
[5]
Kerravala believes for the SmartNIC market to move beyond the high-performance, efficiency-obsessed world of the cloud, the industry needs to reevaluate how SmartNICs can accelerate enterprise workloads.
No shortage of opportunity
With that said, Kerravala sees a plethora of use cases for SmartNICs in enterprise datacenters and the edge.
SmartNICs are particularly well suited to processor-intensive workloads that involve large quantities of data, he said. "Traditional servers weren't really meant for that kind of overhead."
"There's not a company that I talk to that's not telling me that they've got more data on their hands than ever," he added.
[6]
This, Kerravala argues, makes SmartNICs particularly appealing to the growing number of industries looking for ways to use data to a competitive advantage.
This includes markets you might not expect to need high-speed networking and I/O offload capabilities, like retail, hospitality, and entertainment.
Kerravala highlighted sporting events, where a large number of video feeds are streamed live alongside telemetry on the player's performance, as one example of where SmartNICs could prove especially useful.
[7]Marvell CXL roadmap goes all-in on composable infrastructure
[8]To beat the cloud, datacenter operators must embrace automation, multiple vendors: Juniper VP
[9]Why Marvell bought interconnect upstart Tanzanite
[10]VMware says server sprawl is back, and SmartNICs are the solution
Another area where SmartNICs may be beneficial to enterprise customers is security. Because security is becoming so analytics driven, Kerravala believes SmartNICs have the potential to put security inspection much closer to the data.
Palo Alto Networks has already [11]demoed this functionality by deploying its virtualized firewalls on Nvidia's BlueField-2 SmartNICs.
[12]
Supporting these data-intensive workloads means carefully balancing compute, storage, networking, and security to avoid bottlenecks. "You can't have a fast network and slow storage, just like you can't have fast storage and a slow processor," Kerravala said, adding that letting the CPUs do what they're meant to do rather than having to process network transport or encrypt traffic, will save companies money in the long term.
[13]To beat the cloud, datacenter operators must embrace automation, multiple vendors: Juniper VP
[14]VMware says server sprawl is back, and SmartNICs are the solution
[15]SmartNICs, IPUs, DPUs de-hyped: Why and how cloud giants are offloading work from server CPUs
[16]Intel sticks with FPGAs and ASICs for next-gen IPUs
Barriers to adoption abound
For all the potential SmartNICs have to offer, there remains substantial barriers to overcome. The high price of SmartNICs relative to standard NICs being one of many.
Networking vendors have been chasing this kind of I/O offload functionality for years, with things like TCP offload engines, Kerravala said. "That never really caught on and cost was the primary factor there."
Another challenge for SmartNIC vendors is the operational complexity associated with managing a fleet of SmartNICs distributed across a datacenter or the edge.
"There is a risk here of complexity getting to the point where none of this stuff is really usable," he said, comparing the SmartNIC market to the early days of virtualization.
"People were starting to deploy virtual machines like crazy, but then they had so many virtual machines they couldn't manage them," he said. "It wasn't until VMware built vCenter, that companies had one unified control plane for all their virtual machines. We don't really have that on the SmartNIC side."
That lack of centralized management could make widespread deployment in environments that don't have the resources commanded by the major hyperscalers a tough sell. Most enterprises simply don't have the IT resources, Kerravala argued.
Several companies are working closely with SmartNIC vendors to address these challenges. Juniper, for example, has been working with Intel and Nvidia to extend its [17]Contrail orchestration platform to their SmartNICs. Similarly, VMware plans to address this challenge through its [18]Project Monterey initiative.
"Like most technologies, management of the stuff usually gets developed a couple of years after the stuff," Kerravala said.
Where to invest?
Right now, Kerravala believes enterprise datacenters can benefit from SmartNICs, but should deploy them sparingly to accelerate their highest-demand applications.
"If they've got a very high-performance workload, use it there. I wouldn't deploy it everywhere," he said. "I would be putting them in for low-hanging fruit use cases. Use it for all the new stuff, but leave the legacy stuff the way it is."
He also recommends customers consider fully-integrated platforms that pair software, management, and hardware in a single package.
[19]Fungible is one such example. The company offers a suite of storage and compute appliances built around its embedded data processing unit (DPU) — a trendy name for SmartNICs — and managed via a common software layer.
Until recently, the company has focused much of its attention on high-throughput storage applications, but recently applied the technology to compute pooling with a network-addressable appliance that allows GPU resources to be [20]composed on the fly.
However, long term, Kerravala believes SmartNIC vendors need to provide more than hardware and a loose SDK. He highlighted Nvidia's efforts to build out an ecosystem of both hardware and software libraries necessary to build support into enterprise software.
"That kind of complete systems approach reduces the complexity for the developer, helps to build an ecosystem a lot faster, and really turns what is essentially silicon into a platform," Kerravala said. ®
Get our [21]Tech Resources
[1] https://www.theregister.com/2021/11/24/infrastructure_processing_units/
[2] https://www.theregister.com/2022/04/27/vmware_project_monterey_server_sprawl/
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/networks&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YogPhdHTGBFc@buKP-iDuQAAANE&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/networks&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YogPhdHTGBFc@buKP-iDuQAAANE&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/networks&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YogPhdHTGBFc@buKP-iDuQAAANE&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/networks&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YogPhdHTGBFc@buKP-iDuQAAANE&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[7] https://www.theregister.com/2022/05/18/marvell_cxl_composable/
[8] https://www.theregister.com/2022/05/11/juniper_datacenter_cloud/
[9] https://www.theregister.com/2022/05/10/marvell_tanzanite_cxl/
[10] https://www.theregister.com/2022/04/27/vmware_project_monterey_server_sprawl/
[11] https://blogs.nvidia.com/blog/2021/07/12/palo-alto-networks-cyber-defenses/
[12] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/networks&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YogPhdHTGBFc@buKP-iDuQAAANE&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[13] https://www.theregister.com/2022/05/11/juniper_datacenter_cloud/
[14] https://www.theregister.com/2022/04/27/vmware_project_monterey_server_sprawl/
[15] https://www.theregister.com/2021/11/24/infrastructure_processing_units/
[16] https://www.theregister.com/2022/05/10/intel_ipu_smartnic/
[17] https://www.theregister.com/2022/05/11/juniper_datacenter_cloud/
[18] https://www.theregister.com/2022/04/27/vmware_project_monterey_server_sprawl/
[19] https://blocksandfiles.com/2020/08/18/fungible-inc-dpu-launch/
[20] https://blocksandfiles.com/2022/04/06/ts-graphic-fungible-composes-gpu-as-a-service-offering/
[21] https://whitepapers.theregister.com/
another issue..
is these things will bring more bugs with them. Take for example iSCSI offload HBAs. They're quite common, perhaps almost universal on storage arrays, but on servers themselves they are rarely used(even if the capability is present, often is on recent systems regardless). While my systems primarily run on fibre channel storage(so I don't have a whole lot of recent iSCSI experience), I have read almost universally over the years the iSCSI offload HBAs are bug ridden and the general suggestion when using iSCSI is to use a software initiator(which by contrast gives far better results in most cases).
I remember my first experience with hardware iSCSI on a 3PAR E200 storage array in 2006. The NIC on the array (Qlogic dual port 1Gbps) had an issue where it would lock up under high load. I managed to mitigate the problem for a long time by manually balancing paths with MPIO (this was before ESX had round robin). Then maybe a month before I quit that job I rebooted the ESX hosts for software updates, and forgot to re balance the paths again. Major issues after a couple of weeks. I remember my last day at the company I had to hand the support case off to a co-worker as the issue was not yet resolved(and was pretty critical impacting production). A couple of weeks later that company replaced all the iSCSI with fibre channel to solve the issue(a patch ended up being made available a few weeks after that). Felt bad to leave them hanging but my next job started pretty quick I couldn't stick around.
I have read several complaints over the years about network cards with TCP offload enabled causing other unexpected issues as well and in many cases the suggestion is to disable the offload. It makes it more difficult to diagnose when you run a packet capture on the VM or on the host and the data differs from what is actually going over the wire because the NIC is doing stuff to the packets before it goes over the wire.
So beyond cost, these Smart NIC people need to be sure their stuff is really robust for enterprises to consider adopting them. Hyperscalers have the staff and knowledge to deal with that extra complexity. Given history I am not really holding my breath that these vendors will be up to the task. But they could find use cases in tightly vertically integrated solutions that are sold to enterprises, rather than a generic component that you can put in your average server.