Xen Project officially ports its hypervisor to Raspberry Pi 4
(2020/09/29)
- Reference: 1601386271
- News link: https://www.theregister.co.uk/2020/09/29/xen_on_rpi_4/
- Source link:
The Xen project has officially ported its hypervisor to the Raspberry Pi – the 64-bit model 4, to be precise.
The idea to do an official port bubbled up from the Xen community and then reached the desk of George Dunlap, chairman of the Xen Project’s Advisory Board. Dunlap mentioned the idea to an acquaintance who works at the Raspberry Pi Foundation, and was told that around 40 percent of Pis are sold to business users rather than hobbyists.
With more than 30 million Arm-based Pis sold as of December 2019, and sales running at a brisk 600,000-plus a month in April 2020, [1]according to Pi guy Eben Upton, Dunlap saw an opportunity to continue Xen’s drive towards embedded and industrial applications.
Stefano Stabellini, who by day works at FPGA outfit Xilinx, and past Apache Foundation director Roman Shaposhnik took on the task of the port. The pair clocked that the RPi 4's system-on-chip used a regular GIC-400 interrupt controller, which Xen supports out of the box, and thought this was a sign this would, overall, be an easy enough job. That, the duo admitted, was dangerous optimism. Forget the IRQs, there was a whole world of physical and virtual memory addresses to navigate.
The pair were “utterly oblivious that we were about to embark on an adventure deep in the belly of the Xen memory allocator and Linux address translation layers,” we're told.
“The first hurdle was the availability of low memory addresses,” the pair wrote in an announcement heralding the Xen Pi port, seen by The Register ahead of its public release today. “RPi4 has devices that can only access the first 1GB of RAM. The amount of memory below 1GB in Dom0 was not enough. Julien Grall solved this problem with a simple one-line fix to increase the memory allocation below 1GB for Dom0 on RPi4. The patch is now present in Xen 4.14.”
With that problem sorted, the pair again hoped their port would just work, and boot a guest Arm Linux operating system.
It didn’t because: “The Xen subsystem in Linux uses virt_to_phys to convert virtual addresses to physical addresses, which works for most virtual addresses but not all. It turns out that the RPi4 Linux kernel would sometimes pass virtual addresses that cannot be translated to physical addresses using virt_to_phys , and doing so would result in serious errors.”
The two coders found a fix by using a different address translation function when appropriate. That work is now present in the Linux kernel's master branch.
Another hurdle – the biggest – still lay ahead.
RasPad 3.0 converts Raspberry Pi 4 to a tablet – be prepared for some quirks [2]READ MORE
“The Linux kernel has always had the concept of physical addresses and DMA addresses, where DMA addresses are used to program devices and could be different from physical addresses,” the pair explained. “In practice, none of the x86, ARM, and ARM64 platforms where Xen could run had DMA addresses different from physical addresses. The Xen subsystem in Linux is exploiting the DMA/physical address duality for its own address translations. It uses it to convert physical addresses, as seen by the guest, to physical addresses, as seen by Xen.”
But the Raspberry Pi 4 didn't play ball with this technique, which made the Xen subsystem in Linux break.
“It wasn't easy to narrow down the issue,” the pair wrote. “Once we understood the problem, a dozen patches later, we had full support for handling DMA/physical address conversions in Linux.”
“Solving the address translation issue was the end of our fun hacking adventure. With the Xen and Linux patches applied, Xen and Dom0 work flawlessly. Once Linux 5.9 is out, we will have Xen working on RPi4 out of the box.”
Speaking of Linux 5.9, Linus Torvalds on Monday [3]issued release candidate seven and suggested another cycle of testing will be needed before a full version release. So it will be a while before the official Xen-on-Pi is a doddle to deploy.
For those who can't wait, The Register understands [4]Project EVE , the Linux Foundation OS for the IoT-slash-network-edge, will soon publish details on its [5]Github page detailing how to build your own Xen-for-RPi. Hacks to get it up running should also appear on the [6]Xen project blog . And there are unofficial ports, [7]eg this one , though they have limitations, such as being restricted to 1GB of system RAM.
Stefano Stabellini told The Register that an official Xen-on-RPi port will make a difference in the Internet-of-Things community, because other Arm development boards are more costly than the Pi, and programmers will gravitate towards a cheaper alternative for prototyping. He also outlined scenarios, such as a single edge device running both a real-time operating system alongside another OS, each dedicated to different tasks but inhabiting the same hardware and enjoying the splendid isolation of a virtual machine rather than sharing an OS as containers.
George Dunlap also thinks that an official Xen-on-RPi port could also be of use to home lab builders, or perhaps just give developers a more suitable environment for their side projects than a virtual machine or container on their main machines. ®
Get our [8]Tech Resources
[1] https://twitter.com/EbenUpton/status/1248573138344050690
[2] https://www.theregister.com/2020/08/25/raspad_raspberry_pi/
[3] http://lkml.iu.edu/hypermail/linux/kernel/2009.3/03904.html
[4] https://www.lfedge.org/projects/eve/
[5] https://github.com/lf-edge/eve
[6] https://xenproject.org/blog/
[7] https://github.com/dornerworks/xen-rpi4-builder
[8] https://whitepapers.theregister.com/
The idea to do an official port bubbled up from the Xen community and then reached the desk of George Dunlap, chairman of the Xen Project’s Advisory Board. Dunlap mentioned the idea to an acquaintance who works at the Raspberry Pi Foundation, and was told that around 40 percent of Pis are sold to business users rather than hobbyists.
With more than 30 million Arm-based Pis sold as of December 2019, and sales running at a brisk 600,000-plus a month in April 2020, [1]according to Pi guy Eben Upton, Dunlap saw an opportunity to continue Xen’s drive towards embedded and industrial applications.
Stefano Stabellini, who by day works at FPGA outfit Xilinx, and past Apache Foundation director Roman Shaposhnik took on the task of the port. The pair clocked that the RPi 4's system-on-chip used a regular GIC-400 interrupt controller, which Xen supports out of the box, and thought this was a sign this would, overall, be an easy enough job. That, the duo admitted, was dangerous optimism. Forget the IRQs, there was a whole world of physical and virtual memory addresses to navigate.
The pair were “utterly oblivious that we were about to embark on an adventure deep in the belly of the Xen memory allocator and Linux address translation layers,” we're told.
“The first hurdle was the availability of low memory addresses,” the pair wrote in an announcement heralding the Xen Pi port, seen by The Register ahead of its public release today. “RPi4 has devices that can only access the first 1GB of RAM. The amount of memory below 1GB in Dom0 was not enough. Julien Grall solved this problem with a simple one-line fix to increase the memory allocation below 1GB for Dom0 on RPi4. The patch is now present in Xen 4.14.”
With that problem sorted, the pair again hoped their port would just work, and boot a guest Arm Linux operating system.
It didn’t because: “The Xen subsystem in Linux uses virt_to_phys to convert virtual addresses to physical addresses, which works for most virtual addresses but not all. It turns out that the RPi4 Linux kernel would sometimes pass virtual addresses that cannot be translated to physical addresses using virt_to_phys , and doing so would result in serious errors.”
The two coders found a fix by using a different address translation function when appropriate. That work is now present in the Linux kernel's master branch.
Another hurdle – the biggest – still lay ahead.
RasPad 3.0 converts Raspberry Pi 4 to a tablet – be prepared for some quirks [2]READ MORE
“The Linux kernel has always had the concept of physical addresses and DMA addresses, where DMA addresses are used to program devices and could be different from physical addresses,” the pair explained. “In practice, none of the x86, ARM, and ARM64 platforms where Xen could run had DMA addresses different from physical addresses. The Xen subsystem in Linux is exploiting the DMA/physical address duality for its own address translations. It uses it to convert physical addresses, as seen by the guest, to physical addresses, as seen by Xen.”
But the Raspberry Pi 4 didn't play ball with this technique, which made the Xen subsystem in Linux break.
“It wasn't easy to narrow down the issue,” the pair wrote. “Once we understood the problem, a dozen patches later, we had full support for handling DMA/physical address conversions in Linux.”
“Solving the address translation issue was the end of our fun hacking adventure. With the Xen and Linux patches applied, Xen and Dom0 work flawlessly. Once Linux 5.9 is out, we will have Xen working on RPi4 out of the box.”
Speaking of Linux 5.9, Linus Torvalds on Monday [3]issued release candidate seven and suggested another cycle of testing will be needed before a full version release. So it will be a while before the official Xen-on-Pi is a doddle to deploy.
For those who can't wait, The Register understands [4]Project EVE , the Linux Foundation OS for the IoT-slash-network-edge, will soon publish details on its [5]Github page detailing how to build your own Xen-for-RPi. Hacks to get it up running should also appear on the [6]Xen project blog . And there are unofficial ports, [7]eg this one , though they have limitations, such as being restricted to 1GB of system RAM.
Stefano Stabellini told The Register that an official Xen-on-RPi port will make a difference in the Internet-of-Things community, because other Arm development boards are more costly than the Pi, and programmers will gravitate towards a cheaper alternative for prototyping. He also outlined scenarios, such as a single edge device running both a real-time operating system alongside another OS, each dedicated to different tasks but inhabiting the same hardware and enjoying the splendid isolation of a virtual machine rather than sharing an OS as containers.
George Dunlap also thinks that an official Xen-on-RPi port could also be of use to home lab builders, or perhaps just give developers a more suitable environment for their side projects than a virtual machine or container on their main machines. ®
Get our [8]Tech Resources
[1] https://twitter.com/EbenUpton/status/1248573138344050690
[2] https://www.theregister.com/2020/08/25/raspad_raspberry_pi/
[3] http://lkml.iu.edu/hypermail/linux/kernel/2009.3/03904.html
[4] https://www.lfedge.org/projects/eve/
[5] https://github.com/lf-edge/eve
[6] https://xenproject.org/blog/
[7] https://github.com/dornerworks/xen-rpi4-builder
[8] https://whitepapers.theregister.com/
ARM or x86?
phuzz
So would you only be able to run ARM VMs, or could you run x86 ones on a Pi as well?
Apple and Microsoft have both cracked it I suppose.
RISC OS / Linux Chimera
druck
Hopefully with a hypervisor we can get RISC OS running on one core, Linux on the rest, and have the best of both worlds on one tiny British box.
Nice achievement
LeahroyNake
I personally think the best version is the Zero W followed by the 3b+
I have the 3b running pihole DNS, mysql and apache for home monitoring temps etc. The zero is in the garden solar powered with a 12v battery and controls the water feature (it's not a pond, I'm lot allowed one of them :) pumps and heaters to keep the fishies happy. It was fun setting it all up for under £100, kept me occupied for several days while on furlough:)
Nice
A good excuse reason for a 8Gb model-4 purchase.
Easy micro farming on a Pi.