Google confirms in-house scheduler open-sourced into Linux
- Reference: 1597042520
- News link: https://www.theregister.co.uk/2020/08/10/google_scheduling_code_reaches_linux/
- Source link:
As described by Googler Peter Oskolkov on the Linux kernel mailing list, the tech is “an M:N userspace threading subsystem backed by Google-private SwitchTo Linux Kernel API. This subsystem provides latency-sensitive services at Google with fine-grained user-space control/scheduling over what is running when, and this subsystem is used widely internally (called schedulers or fibers).”
Scheduling matters because it queues jobs for a CPU and makes sure that cores don’t lie idle if there’s work they could usefully be doing. M:N scheduling is one approach to the task and can deliver impressive speed as it requires fewer operations than other scheduling techniques.
The Register asked Google to explain what it’s up to with this contribution to the kernel and was told: "Threading technology is widely used across Google."
Which makes a lot of sense given Google tries to wring the most out of its colossal server fleet.
We were told the company “generally wants to open source helpful production APIs" and is releasing this code in the spirit that saw it share the [1]Abseil scheduling code and [2]tcmalloc memory allocation code.
While the post to the kernel mailing list mentions targets version 5.9 of the Linux kernel, Google told us it has: "No firm timeframe to share at this time" regarding when Linux users might be able to play with its API.
Google first discussed this work in 2013 in a [3]talk , and the lengthy time elapsed since that event could mean that Google now runs more advanced scheduling code in-house and is therefore comfortable open-sourcing old work.
Android’s reliance on the Linux kernel could be another factor here, as with many smartphones now boasting multi-core CPUs better scheduling could be desirable.
A few posts to the mailing list discussed the API in recent weeks and Oskolkov [4]said he’d submitted the last of the code. With the Linux 5.9 merge window currently open, we’ll soon now if it will hit the kernel sooner or later. ®
Get our [5]Tech Resources
[1] https://github.com/abseil/abseil-cpp/blob/master/absl/base/internal/low_level_scheduling.h
[2] https://github.com/google/tcmalloc
[3] https://www.youtube.com/watch?v=KXuZi9aeGTw
[4] http://lkml.iu.edu/hypermail/linux/kernel/2008.0/01996.html
[5] https://whitepapers.theregister.com/
Re: hit the kernel sooner or later?
Why not wait until the code becomes available before you judge it? Google is actively involved in several open source projects and has a pretty good record for the quality of the code it contributes. If anything, critcism is levelled at it for keeping code to itself.
Applicabilitiy for Android
While I bet the code will be eagerly inspected by teams running server systems, I'm not sure if it has anything to with scheduling for Android, which doesn't have to be that fine-grained.
Re: Applicabilitiy for Android
Maybe they plan on finally making Android useful for audio apps.
This could be quite tasty.
Especially for server use, but even for personal. I've never really understood complaints about high load or memory usage etc. I paid good money for a gazillion cores and umpteen gigabytes of RAM - bloody well use them. Obviously not just for the sake of it, but if this gets more performance out of your kit, why not eh?
hit the kernel sooner or later?
TBH, I hope it is much much later. Schedulers are nasty things to code. All sorts of edge cases and race conditions keep popping up especially in General Purpose workloads. These are a million miles away from an Android phone especially on HPC installations. The last thing you want is a CPU hog that you can't kill.
As long as there is a way of disabling their scheduler (should it ever be included in the mainstream kernel) I won't mind. I'd treat it much like systemd. Avoid if at all possible.