Linux 6.20~7.0 To Bring Prep Changes For CXL Soft Reserve Recovery & Accelerator Memory
([Hardware] 6 Hours Ago
CXL Initialization Changes)
- Reference: 0001601470
- News link: https://www.phoronix.com/news/Linux-6.20-7.0-CXL-Init-Prep
- Source link:
The next kernel cycle that will be known as either Linux 6.20 or Linux 7.0 depending upon how Linus Torvalds handles the versioning for this next x.20 milestone. More than likely it will be Linux 7.0 given his historical versioning scheme, but whatever the case, ahead of this next kernel cycle some initialization changes for the CXL subsystem are building up.
Linux work around Compute Express Link ( [1]CXL ) continues with many patches continuing out of Intel but also more work in recent times from AMD and other hardware vendors too. There is now a new Git branch in CXL.git of [2]for-7.0/cxl-init . This contains a set of patches for reworking CXL initialization as will be needed to support Soft-Reserve Recovery and Accelerator Memory functionality.
There are patches by AMD engineers for working on the CXL Soft-Reserve Recovery and Accelerator Memory support while these queued patches by Intel engineer Dan Williams is some necessary prep work. Williams [3]explained when these CXL init changes were floated on the mailing list:
"The CXL subsystem is modular. That modularity is a benefit for separation of concerns and testing. It is generally appropriate for this class of devices that support hotplug and can dynamically add a CXL personality alongside their PCI personality. However, a cost of modules is ambiguity about when devices (cxl_memdevs, cxl_ports, cxl_regions) have had a chance to attach to their corresponding drivers on @cxl_bus_type.
This problem of not being able to reliably determine when a device has had a chance to attach to its driver vs still waiting for the module to load, is a common problem for the "Soft Reserve Recovery", and "Accelerator Memory" enabling efforts.
For "Soft Reserve Recovery" it wants to use wait_for_device_probe() as a sync point for when CXL devices present at boot have had a chance to attach to the cxl_pci driver (generic CXL memory expansion class driver). That breaks down if wait_for_device_probe() only flushes PCI device probe, but not the cxl_mem_probe() of the cxl_memdev that cxl_pci_probe() creates.
For "Accelerator Memory", the driver is not cxl_pci, but any potential PCI driver that wants to use the devm_cxl_add_memdev() ABI to attach to the CXL memory domain. Those drivers want to know if the CXL link is live end-to-end (from endpoint, through switches, to the host bridge) and CXL memory operations are enabled. If not, a CXL accelerator may be able to fall back to PCI-only operation. Similar to the "Soft Reserve Memory" it needs to know that the CXL subsystem had a chance to probe the ancestor topology of the device and let that driver make a synchronous decision about CXL operation."
We'll see what more work is queued into CXL.git in the coming weeks ahead of February's Linux 6.20~7.0 merge window and whether the Soft Reserve Recovery and Accelerator Memory features are all squared away in time.
[1] https://www.phoronix.com/search/CXL
[2] https://git.kernel.org/pub/scm/linux/kernel/git/cxl/cxl.git/log/?h=for-7.0/cxl-init
[3] https://lore.kernel.org/all/20251216005616.3090129-1-dan.j.williams@intel.com/
Linux work around Compute Express Link ( [1]CXL ) continues with many patches continuing out of Intel but also more work in recent times from AMD and other hardware vendors too. There is now a new Git branch in CXL.git of [2]for-7.0/cxl-init . This contains a set of patches for reworking CXL initialization as will be needed to support Soft-Reserve Recovery and Accelerator Memory functionality.
There are patches by AMD engineers for working on the CXL Soft-Reserve Recovery and Accelerator Memory support while these queued patches by Intel engineer Dan Williams is some necessary prep work. Williams [3]explained when these CXL init changes were floated on the mailing list:
"The CXL subsystem is modular. That modularity is a benefit for separation of concerns and testing. It is generally appropriate for this class of devices that support hotplug and can dynamically add a CXL personality alongside their PCI personality. However, a cost of modules is ambiguity about when devices (cxl_memdevs, cxl_ports, cxl_regions) have had a chance to attach to their corresponding drivers on @cxl_bus_type.
This problem of not being able to reliably determine when a device has had a chance to attach to its driver vs still waiting for the module to load, is a common problem for the "Soft Reserve Recovery", and "Accelerator Memory" enabling efforts.
For "Soft Reserve Recovery" it wants to use wait_for_device_probe() as a sync point for when CXL devices present at boot have had a chance to attach to the cxl_pci driver (generic CXL memory expansion class driver). That breaks down if wait_for_device_probe() only flushes PCI device probe, but not the cxl_mem_probe() of the cxl_memdev that cxl_pci_probe() creates.
For "Accelerator Memory", the driver is not cxl_pci, but any potential PCI driver that wants to use the devm_cxl_add_memdev() ABI to attach to the CXL memory domain. Those drivers want to know if the CXL link is live end-to-end (from endpoint, through switches, to the host bridge) and CXL memory operations are enabled. If not, a CXL accelerator may be able to fall back to PCI-only operation. Similar to the "Soft Reserve Memory" it needs to know that the CXL subsystem had a chance to probe the ancestor topology of the device and let that driver make a synchronous decision about CXL operation."
We'll see what more work is queued into CXL.git in the coming weeks ahead of February's Linux 6.20~7.0 merge window and whether the Soft Reserve Recovery and Accelerator Memory features are all squared away in time.
[1] https://www.phoronix.com/search/CXL
[2] https://git.kernel.org/pub/scm/linux/kernel/git/cxl/cxl.git/log/?h=for-7.0/cxl-init
[3] https://lore.kernel.org/all/20251216005616.3090129-1-dan.j.williams@intel.com/