News: 1663345811

  ARM Give a man a fire and he's warm for a day, but set fire to him and he's warm for the rest of his life (Terry Pratchett, Jingo)

Linux luminaries discuss efforts to bring Rust to the kernel

(2022/09/16)


Open Source Summit Both Linus Torvalds' Open Source Summit keynote and Jonathan Corbet's "Kernel Report" discussed efforts to allow Rust modules in Linux.

The next version of the kernel will be 6.0, but as ever, the change of major version number doesn't denote any major technical changes. Torvalds acknowledged that it might have been desirable to have a headline feature such as "now you can develop drivers in Rust," but this probably won't happen until kernel 6.1.

So far, there is a [1]working group , and a preliminary [2]patch has been submitted. There are also two preliminary drivers implemented in the new language, which are [3]discussed in depth over on LWN: a [4]driver for NVMe drives, and an in-kernel [5]server for the [6]9P network protocol from the Plan 9 operating system.

[7]

Some significant issues remain, however. Compilers are a big hurdle: currently, the kernel is usually compiled with GCC, the GNU Compiler Collection, but Rust is usually compiled with the rival LLVM compiler.

[8]

[9]

This isn't a complete deal-breaker. The kernel can be built with the LLVM C compiler, Clang , and the process is [10]documented . It's the way that Android, ChromeOS, and OpenMandriva compile their kernels. One significant snag with building the kernel with Clang, though, is CPU architecture support. Only x86 and Arm are well supported this way, and other architectures are not as solid yet.

The flip side of compiling the kernel with Clang is compiling Rust with GCC. There is an experimental [11]Rust-on-GCC compiler front end, although the project page notes:

the compiler is in a very early stage and not usable yet for compiling real Rust programs.

And…

this project was a community effort several years ago where Rust was still at version 0.9.

However, as our [12]sister site DevClass reports, this may land in GCC 13 . To track the development of Rust in GCC, follow the blog of lead developer [13]Phil Herron .

Developer [14]Miguel Ojeda has written about the [15]reasoning behind the effort, and delivered a status update [16]talk at the Linux Plumbers Conference that overlapped the Open Source Summit. His [17]slides [PDF] have some in-depth info, talking about the potential stability improvements from Rust's much improved memory safety compared to C.

[18]The next deep magic Linux program to change the world? Io_uring

[19]Fedora 37 beta: Hints of what's to come in Red Hat's free flagship

[20]Linux kernel's eBPF feature put to unexpected new uses

[21]Linux 6.0 debuts, missing some Rusty bits and a magic mushroom reference

There's already a working NVMe driver in the kernel, so the new driver is for research, as developer Andreas Hindborg described in his [22]talk , which followed Ojeda's presentation – the YouTube video is of the whole Rust micro-conference. Be prepared for it to get technical, as Hindborg's [23]slides [PDF] demonstrate.

The Reg FOSS desk asked our tame compiler boffin Mark Morgan Lloyd for his assessment of the significance of Rust. As he put it:

Rust obviously uses braces which lulls the incautious into a sense of security. However, its underlying assignment model is sufficiently novel that I'm inclined to call it a sibling of ALGOL rather than a derivative… and just about every general-purpose language (except Smalltalk, LISP, Forth and APL) is a derivative of ALGOL, so you see how big it is.

One risk of its adoption, though, was summarized in LWN executive editor Jonathan Corbet's Kernel Report. To quote "a longtime kernel develper" from his [24]slides [PDF], the problem is not just conservatism among developers, but the implication that Rust is desirable simply because of its memory safety:

There are possibly some well-designed and written parts which have not suffered a memory safety issue in many years. It's insulting to present this as an improvement over what was achieved by those doing all this hard work. ®

[25]Youtube Video

Get our [26]Tech Resources



[1] https://github.com/Rust-for-Linux

[2] https://lkml.org/lkml/2021/12/6/461

[3] https://lwn.net/SubscriberLink/907685/0290fbfe1ba855ea/

[4] https://github.com/metaspace/rust-linux/commit/3446d310d9082c00eecb4b396a167e3f19b3b2f1

[5] https://github.com/wedsonaf/linux/commits/9p

[6] https://www.techopedia.com/definition/26091/9p

[7] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/oses&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YyTx9ed0nRpOJYWTVgTWZAAAAFU&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0

[8] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/oses&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YyTx9ed0nRpOJYWTVgTWZAAAAFU&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0

[9] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/oses&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YyTx9ed0nRpOJYWTVgTWZAAAAFU&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0

[10] https://docs.kernel.org/kbuild/llvm.html

[11] https://rust-gcc.github.io/

[12] https://devclass.com/2022/07/12/rust-gcc-front-end-approved-by-steering-committee-beta-expected-in-gcc-13/

[13] https://thephilbert.io/

[14] https://ojeda.dev/

[15] https://www.memorysafety.org/blog/memory-safety-in-linux-kernel/

[16] https://lpc.events/event/16/contributions/1256/

[17] https://lpc.events/event/16/contributions/1256/attachments/1048/2003/Rust%20Status%20-%20LPC%202022.pdf

[18] https://www.theregister.com/2022/09/16/column/

[19] https://www.theregister.com/2022/09/14/fedora_37_beta/

[20] https://www.theregister.com/2022/09/14/linux_ebpf/

[21] https://www.theregister.com/2022/08/15/linux_60_debuts_missing_some/

[22] https://lpc.events/event/16/contributions/1180/

[23] https://lpc.events/event/16/contributions/1180/attachments/1017/1961/deck.pdf

[24] https://static.sched.com/hosted_files/osseu2022/04/osseu-kr.pdf

[25] https://www.youtube.com/watch?v=Xw9pKeJ-4Bw

[26] https://whitepapers.theregister.com/



All of that effort

bombastic bob

So, it appears that a LOT of effort is needed to get Rust into the LInux kernel. A _LOT_.

Question: what is the 'bang for the buck' expected at the end of the process?

In comparison, Microsoft has spent a GREAT DEAL of effort re-re-redesigning the user interface. Let's suppose they had LEFT IT as it was for Winxows 7, but then spent a BUNCH of time making improvements needed for touch screens without breaking the keyboard+mouse NOR the hotkeys NOR the UI that people were now very familiar with. They COULD have made the kernel SO robust that patches would no longer have to be deployed weekly. OK maybe not but still.

NOW we have a significant amount of Linux kernel dev time dedicated to folding RUST into it, and THEN having to MAINTAIN it.

In My Bombastic Opinion, there may be BETTER ways to spend this time, maybe more code reviews and performance tweeks? And maybe we could use some really good open source video drivers for various adapters (for XOrg, naturally, why waste time on Wayland).

I once again cite Arthur C. Clarke's "Superiority"

[1]http://www.mayofamily.com/RLM/txt_Clarke_Superiority.html

[1] http://www.mayofamily.com/RLM/txt_Clarke_Superiority.html

Re: All of that effort

Ken Hagan

Question: what is the 'bang for the buck' expected at the end of the process?

Three possibilities spring to mind:

1) it turns out that rusty code is no easier to write and/or slower to run. Over time, take-up is minimal and eventually the support is dropped. As a result, we now have a better idea of the properties that a language must have if you want to write a kernel.

2) The opposite. Over time, more and more of the kernel gets rusty and eventually the kernel is just some assembly language routines propping up a lot of rust. As a result, well, same as before except it is C that no-one ever uses again.

3) Something in between, with each language having a well-defined role for different kinds of kernel component. As a result, no-one ever tries to write a kernel without multi-lingual support designed in from the start.

Whatever the outcome, I'm pretty sure we're going to learn something fairly important about how to write an OS. I'm also fairly sure that lesson will carry over to a few other large software projects.

Re: All of that effort

jake

Possibility number the fourth:

The next language du jour will displace rust in the fanboi's fancy before rust works properly with GCC, making this yet another footnote in the annals of computing.

No, mainline kernel development is not moving to llvm/clang any time soon.

Re: All of that effort

thejoelr

I think the bang for the buck is kernel work that would be lost without rust support. It is yet to be seen if the cost of supporting rust will be worth it, but I am guessing it will. Linux isn't a single entity managed project, so you have to be flexible about working with others.

Re: All of that effort

jake

So what you are saying is that the Kernel will gain devs who refuse to code in anything but rust ... and this is a fine example of flexibly working with others?

Re: All of that effort

thejoelr

I mostly don't see how there's much argument about it. I'm not writing any kernel contributions, so I just watch from the outside. Clearly they think it is interesting... I'm curious about the possibilities. Linus tends to be conservative and sometimes counterintuitive on some restrictions, so I am mostly depending on him when he allows something vs denies it.

Re: All of that effort

teknopaul

Try it, writing rust is fast, real fast. Interop with C is great.

There is plenty of evidence to back up the "fearless" claim.

If you don't like it, go back to C, because its going to be a while before they let you write Perl in the kernel. ;)

Re: All of that effort

Munchausen's proxy

"If you don't like it, go back to C, because its going to be a while before they let you write Perl in the kernel. ;)"

systemd-cpan

Re: All of that effort

teknopaul

Interop between c and rust is great, far better that other languages

Re: All of that effort

karlkarl

That may be so, but a homogeneous codebase (i.e C everywhere) is still easier in this regard. No bindings needed at all.

C++ being a 99% "superset" of C kind of achieves this but there are a few other issues.

That said, much of the AMDGPU drivers is C++ and it seems to integrate OK with the rest of the kernel.

Re: All of that effort

Aitor 1

C interaction with C is better.

It's not an insult

sarusa

> 'There are possibly some well-designed and written parts which have not suffered a memory safety issue in many years. It's insulting to present this as an improvement over what was achieved by those doing all this hard work.'

This is just silly. Yes, most of the kernel has been painstakingly scrubbed of memory safety issues, but there have been tons of them and there are many more lurking. I used to write in assembly language - I don't any more (once a year or so) because I can use C#, Rust, or Python and get literally 100x times as much work done in the same amount of time without worrying if my JMPing is idempotent. It's not an 'insult' to my past self to admit that assembly language programming is a touchy, dangerous, pain in the ass. Good for me for having been able to do that, but thankfully it's rarely necessary any more.

Similarly, it will be great to know that future kernel drivers, and maybe more significant parts, written by other people who are not as awesome as me (tongue -> cheek) can be guaranteed free of memory safety issues. This is indeed a giant improvement. So kudos to the guys who did it in C (including me!) while keeping memory safe, but they're not going to be 'insulted' by making it possible to go from banging two rocks together to hammers.

Re: It's not an insult

Someone Else

Methinks you miss the point. The fact that the kernel has, as you say, been painstakingly scrubbed of memory safety issues, is indeed the result of necessary "hard work". That hard work included (and continues to include) extreme care, planning, and discipline to prevent propagating those same errors in future work. Now comes the next panacea-du-jour, "memory-safe" languages (air-quotes intentional), because the next generation of "coders" can't be arsed to develop the discipline, do the planning, and execute the extreme care because, well the "compiler will do all that for me". Except it won't in all cases. So the ongoing development will be dumbed-down with the "practitioners" counting on an imperfect tool to be perfect. We all know how that will end.... 1

Perhaps "insult" was too strong a term. But the impending deterioration of developer skills deserves some form of epithet to properly label it. My suggestion: s/insult/pathetic/

1 cf. garbage collection as the panacea for object and heap management; and of course, the Tesla "autopilot".

Re: It's not an insult

jake

I think the hubris of the rust devotes is both pathetic and insulting.

Re: It's not an insult

teknopaul

Things is when you write kernel C you don't have libc, it the syntax but not the language you are used to, so you have to write a lot of code you would otherwise use a library for yourself. Having a lot of help from the compiler is necessary.

Rust is like -Wall on steroids, it's not hubris, it's the exact opposite: it's acknowledging that you are not invincible and you need help from the compiler.

Rust is desirable simply because of its memory safety

Howard Sway

This much-hyped memory safety is a bit of a red herring when it comes to compiling Rust code. For sure, the language doesn't allow many risky C type operations. But ultimately it's got to be compiled into machine code, and both CLang and GCC break down source code written in any supported language into lower level representations before the machine code is generated. And then there's a subsequent optimisation phase if you want it. So whilst it might offer less risky operations at the source level, there's no way a language can actually guarantee memory safety after compilation.

Re: Rust is desirable simply because of its memory safety

johannesburgel

> there's no way a language can actually guarantee memory safety after compilation.

I don't think you understand how things work.

Re: Rust is desirable simply because of its memory safety

Howard Sway

Dont understand? Really? Can you guarantee that a Rust front end for the GCC would always generate memory-safe GIMPLE representations for every possible Rust expression? That's not a function of the Rust language (which I find likeable, with many good ideas in its design), but of the compiler implementation, so my point that memory safety can't be guaranteed by language design alone is pretty straightforward when talking about the GCC, which generates all its machine language from GIMPLE, not direct from source code.

Re: Rust is desirable simply because of its memory safety

teknopaul

Rust does not guarantee memory safety, it helps you write memory safe code. It gives you enough rope to hang yourself should you want to.

It's very easy to write safe rust, and for it to be obvious that it it is safe. Independent of the compiler or target arch.

It's safe even when multi threaded. That's pretty clever it's almost impossible write multithreaded c that is _obviously_ safe. Multithreaded rust is as easy as single threaded rust because single ownership works across threads.

Rust _is_ pretty cool. It's not just the memory safety, also the ability to write test code along side production code and documentation code that you know compiles and is correct too. It's number system is strangely convenient too.

<Zoid> I still think you guys are nuts merging Q and QW. :P
<knghtbrd> Of course we're nuts. Even John said so. =>
<taniwha> Zoid: we're nuts, but we're productive nuts:)