How Long Does It Take to Fix Linux Kernel Bugs? (itsfoss.com)
- Reference: 0180565836
- News link: https://it.slashdot.org/story/26/01/12/0443201/how-long-does-it-take-to-fix-linux-kernel-bugs
- Source link: https://itsfoss.com/news/linux-kernel-bugs-arent-found-for-years/
> Jenny Guanni Qu, a researcher at [VC fund] Pebblebed, [2]analyzed 125,183 bugs from 20 years of Linux kernel development history (on Git). The findings show that the average bug takes 2.1 years to find. [Though the median is 0.7 years, with the average possibly skewed by "outliers" discovered after years of hiding.] The longest-lived bug, a buffer overflow in networking code, went unnoticed for 20.7 years! [But 86.5% of bugs are found within five years.]
>
> The research was carried out by relying on the Fixes: tag that is used in kernel development. Basically, when a commit fixes a bug, it includes a tag pointing to the commit that introduced the bug. Jenny wrote a tool that extracted these tags from the kernel's git history going back to 2005. The tool finds all fixing commits, extracts the referenced commit hash, pulls dates from both commits, and calculates the time frame. As for the dataset, it includes over 125k records from [3]Linux 6.19-rc3 , covering bugs from April 2005 to January 2026. Out of these, 119,449 were unique fixing commits from 9,159 different authors, and only 158 bugs had CVE IDs assigned.
It took six hours to assemble the dataset, [4]according to the blog post , which concludes that the percentage of bugs found within one year has improved dramatically, from 0% in 2010 to 69% by 2022. The blog post says this can likely be attributed to:
The Syzkaller fuzzer (released in 2015)
Dynamic memory error detectors like KASAN, KMSAN, KCSAN sanitizers
Better static analysis
More contributors reviewing code
But "We're simultaneously catching new bugs faster AND slowly working through ~5,400 ancient bugs that have been hiding for over 5 years."
They've also developed an AI model called VulnBERT that predicts whether a commit introduces a vulnerability, claiming that of all actual bug-introducing commits, it catches 92.2%. "The goal isn't to replace human reviewers but to point them at the 10% of commits most likely to be problematic, so they can focus attention where it matters..."
[1] https://itsfoss.com/news/linux-kernel-bugs-arent-found-for-years/
[2] https://pebblebed.com/blog/kernel-bugs
[3] https://www.phoronix.com/news/Linux-6.19-rc3-Released
[4] https://pebblebed.com/blog/kernel-bugs
earlone (Score:1)
I thought there are no bugs on Linux !
Re: (Score:2)
It's because you have to look inside, not "on" it. /s
Re: (Score:1)
What about the bugs under linux ? did you have a look ?
BS (Score:3)
If you don't know it exists you cannot fix it. So all that time until you find it should be subtracted. How long it takes to fix it should start at the time it was found.
& how long does it take for patches to get mer (Score:2)
started to upstream a patch on average a day, but man has that an overhead, ... [1]https://www.youtube.com/watch?... [youtube.com]
[1] https://www.youtube.com/watch?v=57fbAa6gzs4
More naunced (Score:3)
I think that some bug scenarios are more complex than described in this article.
I recall working on a Bluetooth kernel crash more than 10 years ago. The crash was a race condition between 2 Bluetooth threads when the RFCOMM link was being disconnected. To trigger the crash, the execution order of the 2 threads had to be reversed from "normal" so that 1 thread freed the memory containing the reference counter that the other thread relied on to not free the memory! This was a case of inappropriate use of reference counters when an abrupt disconnect had to be handled.
The crash was observed under very heavy processor loading that caused CPU starvation to the Bluetooth threads resulting in an "abnormal" execution order. Static analysis suggested that the reference counter would provide lock protection because reference counters are used through-out the kernel's networking protocol implementations. However, the flaw is that the reference counter can momentarily go to zero to trigger memory freeing but a second thread can come along and add 1 to the reference counter despite the memory now being in a freed state.
Looking through the commit history revealed about 4 attempts to fix this issue by 4 individuals over a period of about 5 years. I had to remove these ineffective and misguided workarounds to implement a proper fix that completely removed the use of the reference counter and immediately delete the state memory as soon as the disconnect was detected. NULL pointer checks prevented any use-after-free scenarios.
Therefore, kernel bugs may go through several workaround fixes before the root cause is fully understood and fixed. So saying that a bug was resident for 5 years may not mean that no attempts had been made to resolve it during that period.
Re: More naunced (Score:2)
Yup, and other areas of nuance, too.
The story says half of the issues are fixed quickly. The actual research post (not the short writeup /. links to) adds that 30% are what they called "self fixes" and incomplete features being implemented rather than stable features with defects. Most developers consider this just normal development processes, not the dangerous types of bugs.
It doesn't break down issues by severity and frequency. Bugs caught in experimental features not used by anyone VS bugs in features
Tootsie Roll Pop Owl says... (Score:2)
One.. Two... Three... decades. That's how long I've been trying to get working video drivers. Every time I retire a Windows or Mac system, I try to extend its life with Linux. I fail every time. All the way back to the ET4000.
While interesting... (Score:2)
Without context, I don't know what to do with this. .7 years median seems quite okay to me.
Re: (Score:3, Funny)
> Without context, I don't know what to do with this. .7 years median seems quite okay to me.
You want context? OK.
How about that one time at bug camp when the OS became irrelevant after we discovered speculative execution bugs that “only” affected damn near every x86 CPU that rolled off an assembly line for the last twenty-five fucking years.
Re: (Score:3)
> How about that one time at bug camp...
You couldn't find a way to work the flute into that?
Re: (Score:1)
>> How about that one time at bug camp...
> You couldn't find a way to work the flute into that?
Not until “Flute” makes it to the tippy top of the TIOBE index. I only fuck the best.
Re: (Score:2)
No, he didn't have any lubricant handy and the damn thing just wouldn't go in.