Alibaba Cloud claims K8s service meshes can require more resources than the apps they run
- Reference: 1723435269
- News link: https://www.theregister.co.uk/2024/08/12/alibaba_microservices_mesh_canal/
- Source link:
The Chinese cloud leader revealed the existence of Canal Mesh at last week's Association for Computing Machinery SIGCOMM conference in Sydney, Australia, in a [1]presentation and [2]paper [PDF]. The prez opened with an explanation of how microservices rely on service meshes to connect Kubernetes pods, how those meshes rely on a proxy "sidecar" to handle and mediate network communication between microservices, and to collect telemetry on traffic, so that applications don't need their own networking plumbing.
But in Alibaba Cloud's estimation, sidecars "cause numerous problems, including intrusion into the user pod, excessive resource occupation, significant overhead in managing many sidecars, and performance degradation caused by passing traffic through the sidecar."
[3]
The Chinese cloud considered Istio's impact on a customer that ran a Kubernetes cluster comprising 500 nodes and 15,000 pods, and found it consumed 1,500 cores and 5,000 gigabytes of memory – ten percent of hardware resources.
[4]
[5]
In other scenarios, Alibaba Cloud claimed the sidecar's CPU and memory requirements "grow even higher than that of the app."
That's bonkers, and clearly untenable. And in 2022 Google did something about it by [6]introducing Ambient Mesh – an Istio data plane mode that offered Istio users the chance to park sidecars.
[7]
Alibaba Cloud's paper notes that Ambient Mesh improved performance and reduced demands on resources – but still required some proxies to reside within the user cluster.
The Chinese cloud felt complete decoupling of service mesh from user clusters would be more effective – and built Canal Mesh to prove it.
[8]Alibaba Cloud reveals its datacenter design, homebrew network used for LLM training
[9]Alibaba Cloud closing Australian and Indian datacenters
[10]Faulty instructions in Alibaba's T-Head C910 RISC-V CPUs blow away all security
[11]Tencent Cloud's home-grown traffic-tamer halves WAN latency
The paper claims Alibaba Cloud succeeded, handsomely, and produced the following results:
Throughput 12.3x and 2.3x higher than Istio and Ambient, with latency 1.7x and 1.3x lower;
CPU consumption 12x~19x and 4.6x~7.2x lower than Istio and Ambient;
Configuration completion time for creating hundreds of pods 1.5x~2.1x and 1.2x~1.5x smaller than Istio and Ambient;
Southbound bandwidth occupation is 9.8x and 4.6x lower than Istio and Ambient.
Alibaba Cloud achieved those numbers with an architecture that sees proxies moved out of the user cluster – albeit with a minimal on-node proxy retained to handle some security and observability chores.
eBPF-based kernel bypass and remote mTLS acceleration are also employed. The paper describes how Alibaba Cloud uses its hyperscale smarts to place proxies across its pools of resources.
The paper and presentation state that Canal Mesh has run at Alibaba Cloud for a year – without quite confirming it is in production. Both also omit a link or even mention of code for you to peruse or implement – but the presentation includes contacts at Alibaba Cloud for those who have questions.
[12]
If Alibaba Cloud intends to keep Canal Mesh to itself, it may operate more efficiently than its rivals. The Chinese cloud does compete with the likes of AWS, Google, and Azure in some markets, but The Register understands most of Alibaba's clients outside China have roots or connections in the Middle Kingdom that see them feel more comfortable with the outfit than is the case for buyers based in other countries. ®
Get our [13]Tech Resources
[1] https://www.youtube.com/watch?v=7SLvzQ697qE
[2] https://dl.acm.org/doi/pdf/10.1145/3651890.3672221
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_specialfeatures/cloudinfrastructuremonth&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2ZrndR3WlSz1sq7b5zok9AQAAAJQ&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_specialfeatures/cloudinfrastructuremonth&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZrndR3WlSz1sq7b5zok9AQAAAJQ&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_specialfeatures/cloudinfrastructuremonth&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZrndR3WlSz1sq7b5zok9AQAAAJQ&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[6] https://istio.io/latest/blog/2022/introducing-ambient-mesh/
[7] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_specialfeatures/cloudinfrastructuremonth&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZrndR3WlSz1sq7b5zok9AQAAAJQ&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[8] https://www.theregister.com/2024/06/27/alibaba_network_datacenter_designs_revealed/
[9] https://www.theregister.com/2024/07/01/alibaba_cloud_closes_india_australia/
[10] https://www.theregister.com/2024/08/07/riscv_business_thead_c910_vulnerable/
[11] https://www.theregister.com/2024/08/05/tencent_clouds_homegrown_traffictamer_halves/
[12] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_specialfeatures/cloudinfrastructuremonth&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZrndR3WlSz1sq7b5zok9AQAAAJQ&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[13] https://whitepapers.theregister.com/
Black box abstraction layers...
... are, as it happens, what Moxie Marlinspike was [1]reported here of bemoaning recently.
I get the distinct impression that in the search for "flexibility" we're simply piling Pelion on Ossa and risk reaching the point at which the minimum resources needed for some "scalable" applications aren't that much different from the peak resources required if they were (literally) left to their own devices. Alibaba claim their approach reduces CPU load on the application servers by up to 70% - that's a mind-boggling waste of resources in the status quo ante.
This is the second report here of a SIGCOMM paper detailing the use of eBPF filters to allegedly improve network performance and I can't help feeling it's a glaring symptom of a problem rather than a solution to it. Apart from the layers of complexity (and the need to manage them), I'm instinctively wary of applications protocols messing around with the routing layer they're using to communicate with each other: it's always been a challenge to keep even simple routing algorithms stable in the case of frequent topology changes and, even setting aside the added potential for bugs in the screeds of extra code, any failure to understand the subtle interaction between the various layers is a potential source of new instability.
Of course, part of the increased efficiency simply comes from moving the functionality outside of the compute nodes and into the network infrastructure - where it still has to be paid for, but the claim is that the network infrastructure is already doing most of the required work already and most of it was being duplicated unnecessarily.
The real question is, of course, how did we get into this situation? How, in the pursuit of resource efficiency, have we come to solutions that are apparently so resource-hungry and so impenetrable - but no-one really noticed? Marlinspike, perhaps rightly, points the finger at the "ballooned" ranks of the large engineering organizations , but this bloat is not just a red flag for performance, but for security too.
[1] https://www.theregister.com/2024/08/09/marlinspike/
latency 1.7 times lower
Something 0 times lower latency has the same latency. 0.5 times lower would be half the latency. 1 times lower must be 0 latency, a cool trick indeed. 1.7 times lower latency must mean traffic arrives before it was sent, which reaches Thiotimoline-like spookiness.
Chatting with some of the team that run our in-house hyper-converged platform and they tell me you have to allow 10-20% system capacity just to keep the lights on, so Alibaba's overhead figures don't seem out of the ordinary.