AMD Proposes New LLVM iTHP Option For Squeezing More Performance
([LLVM] 71 Minutes Ago
-fenable-readonly-thp)
- Reference: 0001654457
- News link: https://www.phoronix.com/news/LLVM-fenable-readonly-thp
- Source link:
AMD's compiler team is proposing a new LLVM Clang compiler option, -fenable-readonly-thp for implementing instruction Transparent Huge Pages "iTHP" at compile and link-time. With -fenable-readonly-thp, they are finding more performance out of SPEC CPU, Geekbench, and other workloads from the arranging of 2MB segment alignment at link time and allowing the executable text range can be promoted to huge pages.
Bhuvanendra Kumar N of AMD's compiler technologies team announced the -fenable-readonly-thp proposal on Tuesday as their latest step for squeezing more performance out of the LLVM toolchain. He explained in the request for comments (RFC) posting:
"We propose an opt-in Clang feature, -fenable-readonly-thp, that implements iTHP (instruction Transparent Huge Pages) at compile and link time.
The driver arranges 2 MiB (or larger) segment alignment at link time, links a small startup object, and at process start calls madvise(MADV_COLLAPSE) on the executable text range so the kernel can promote default 4 KiB page mappings to huge pages.
A prototype patch is available; we welcome feedback on this RFC before posting the patch series to Phabricator.
...
Large binaries with big instruction footprints can be frontend-bound due to iTLB pressure from 4 KiB text mappings. Linux can promote eligible executable .text regions to 2 MiB pages when alignment requirements are met and the process opts in via madvise().
Today this requires manual linker scripts and custom startup code. A compiler-managed, opt-in path makes the optimization practical for production builds."
A prototype patch is already created and tested. There were not any firm performance numbers shared as part of the RFC announcement, but it was noted broadly of performance gains from this option:
"We measured performance uplift with SPEC CPU 2026, Geekbench, and other benchmarks (CPython, database/server workloads, micro-architectural tests). Gains are workload-dependent—strongest on large .text workloads such as interpreters, simulators, and compiler-like binaries; some workloads are unchanged. We did not see broad regressions in our testing."
This does come at the cost though of potentially larger binaries and small overhead costs at start-up. With this option for iTHP usage, it would work fully with Linux 7.2 and newer due to older kernel versions of the kernel only working with iTHP from TMPFS-based storage.
While the BOLT binary layout optimizer can provide some optimizations to enhance iTLB performance, unlike BOLT the "-fenable-readonly-thp" doesn't require any workload profiling or similar.
A Google compiler engineer noted in response they have developed a library to carry out a similar approach to what AMD is seeking. The hope with AMD is to get this into the upstream LLVM (and potentially GNU) toolchains rather than requiring another external library on the system for this support.
We'll see where this discussion leads and if it manages to be upstreamed but for those interested in the initial proposal can find it on [1]LLVM Discourse .
[1] https://discourse.llvm.org/t/rfc-enable-read-only-transparent-huge-pages-thp-for-executable-text-segment/91646
Bhuvanendra Kumar N of AMD's compiler technologies team announced the -fenable-readonly-thp proposal on Tuesday as their latest step for squeezing more performance out of the LLVM toolchain. He explained in the request for comments (RFC) posting:
"We propose an opt-in Clang feature, -fenable-readonly-thp, that implements iTHP (instruction Transparent Huge Pages) at compile and link time.
The driver arranges 2 MiB (or larger) segment alignment at link time, links a small startup object, and at process start calls madvise(MADV_COLLAPSE) on the executable text range so the kernel can promote default 4 KiB page mappings to huge pages.
A prototype patch is available; we welcome feedback on this RFC before posting the patch series to Phabricator.
...
Large binaries with big instruction footprints can be frontend-bound due to iTLB pressure from 4 KiB text mappings. Linux can promote eligible executable .text regions to 2 MiB pages when alignment requirements are met and the process opts in via madvise().
Today this requires manual linker scripts and custom startup code. A compiler-managed, opt-in path makes the optimization practical for production builds."
A prototype patch is already created and tested. There were not any firm performance numbers shared as part of the RFC announcement, but it was noted broadly of performance gains from this option:
"We measured performance uplift with SPEC CPU 2026, Geekbench, and other benchmarks (CPython, database/server workloads, micro-architectural tests). Gains are workload-dependent—strongest on large .text workloads such as interpreters, simulators, and compiler-like binaries; some workloads are unchanged. We did not see broad regressions in our testing."
This does come at the cost though of potentially larger binaries and small overhead costs at start-up. With this option for iTHP usage, it would work fully with Linux 7.2 and newer due to older kernel versions of the kernel only working with iTHP from TMPFS-based storage.
While the BOLT binary layout optimizer can provide some optimizations to enhance iTLB performance, unlike BOLT the "-fenable-readonly-thp" doesn't require any workload profiling or similar.
A Google compiler engineer noted in response they have developed a library to carry out a similar approach to what AMD is seeking. The hope with AMD is to get this into the upstream LLVM (and potentially GNU) toolchains rather than requiring another external library on the system for this support.
We'll see where this discussion leads and if it manages to be upstreamed but for those interested in the initial proposal can find it on [1]LLVM Discourse .
[1] https://discourse.llvm.org/t/rfc-enable-read-only-transparent-huge-pages-thp-for-executable-text-segment/91646