A 10x Workaround & Less Network Egress Downtime Change Submitted For Linux 6.17
([Linux Kernel] 5 Hours Ago
Linux 6.17 Locking)
- Reference: 0001564915
- News link: https://www.phoronix.com/news/Linux-6.17-Locking
- Source link:
The kernel locking changes submitted today for Linux 6.17 contain a temporary change worth discussion for yielding a 10x speed-up of a particular function call and as part of that yielding less network egress downtime until a better solution is developed.
The [1]locking changes sent out today for the Linux 6.17 merge window include a number of low-level code changes while catching my eye was beginning to use synchronize_rcu_expedited() within lockdep_unregister_key() for better performance.
Meta engineer and Debian developer Breno Leitao authored the one line of code change in switching from synchronize_rcu to synchronize_rcu_expedited. He explained on the patch:
"lockdep: Speed up lockdep_unregister_key() with expedited RCU synchronization
lockdep_unregister_key() is called from critical code paths, including sections where rtnl_lock() is held. For example, when replacing a qdisc in a network device, network egress traffic is disabled while __qdisc_destroy() is called for every network queue.
If lockdep is enabled, __qdisc_destroy() calls lockdep_unregister_key(), which gets blocked waiting for synchronize_rcu() to complete.
For example, a simple tc command to replace a qdisc could take 13 seconds:
# time /usr/sbin/tc qdisc replace dev eth0 root handle 0x1: mq
real 0m13.195s
user 0m0.001s
sys 0m2.746s
During this time, network egress is completely frozen while waiting for RCU synchronization.
Use synchronize_rcu_expedited() instead to minimize the impact on critical operations like network connectivity changes.
This improves 10x the function call to tc, when replacing the qdisc for a network card.
# time /usr/sbin/tc qdisc replace dev eth0 root handle 0x1: mq
real 0m1.789s
user 0m0.000s
sys 0m1.613s"
The original issue stems from a problem discovered at Meta. This principally will help with the performance of the debug kernel builds.
The synchronize RCU expedited call has been [2]known for years to be much faster albeit despised by developers and at the cost of system IPIs. This workaround for faster performance in lockdep_unregister_key is expected to be temporary with ultimately working to remove RCU usage and relying on hazptr in its place. See [3]this LKML thread for more details on that discussion. But for Linux 6.17 this one-liner change to use the expedited call is part of the locking pull request.
[1] https://lore.kernel.org/lkml/aIhvlNUoN5KyPzg6@gmail.com/
[2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=be3fc413da9eb17cce0991f214ab019d16c88c41
[3] https://lore.kernel.org/all/20250321-olivine-harrier-of-cubism-f2ad98@leitao/
The [1]locking changes sent out today for the Linux 6.17 merge window include a number of low-level code changes while catching my eye was beginning to use synchronize_rcu_expedited() within lockdep_unregister_key() for better performance.
Meta engineer and Debian developer Breno Leitao authored the one line of code change in switching from synchronize_rcu to synchronize_rcu_expedited. He explained on the patch:
"lockdep: Speed up lockdep_unregister_key() with expedited RCU synchronization
lockdep_unregister_key() is called from critical code paths, including sections where rtnl_lock() is held. For example, when replacing a qdisc in a network device, network egress traffic is disabled while __qdisc_destroy() is called for every network queue.
If lockdep is enabled, __qdisc_destroy() calls lockdep_unregister_key(), which gets blocked waiting for synchronize_rcu() to complete.
For example, a simple tc command to replace a qdisc could take 13 seconds:
# time /usr/sbin/tc qdisc replace dev eth0 root handle 0x1: mq
real 0m13.195s
user 0m0.001s
sys 0m2.746s
During this time, network egress is completely frozen while waiting for RCU synchronization.
Use synchronize_rcu_expedited() instead to minimize the impact on critical operations like network connectivity changes.
This improves 10x the function call to tc, when replacing the qdisc for a network card.
# time /usr/sbin/tc qdisc replace dev eth0 root handle 0x1: mq
real 0m1.789s
user 0m0.000s
sys 0m1.613s"
The original issue stems from a problem discovered at Meta. This principally will help with the performance of the debug kernel builds.
The synchronize RCU expedited call has been [2]known for years to be much faster albeit despised by developers and at the cost of system IPIs. This workaround for faster performance in lockdep_unregister_key is expected to be temporary with ultimately working to remove RCU usage and relying on hazptr in its place. See [3]this LKML thread for more details on that discussion. But for Linux 6.17 this one-liner change to use the expedited call is part of the locking pull request.
[1] https://lore.kernel.org/lkml/aIhvlNUoN5KyPzg6@gmail.com/
[2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=be3fc413da9eb17cce0991f214ab019d16c88c41
[3] https://lore.kernel.org/all/20250321-olivine-harrier-of-cubism-f2ad98@leitao/
Kjell