Security warning deluge from 'npm audit' is driving developers to distraction
- Reference: 1625812332
- News link: https://www.theregister.co.uk/2021/07/09/npm_audit_security/
- Source link:
"As of today, npm audit is a stain on the entire npm ecosystem," Abramov declared in a [1]blog post . "The best time to fix it was before rolling it out as a default. The next best time to fix it is now."
According to Abramov, 99 per cent of the vulnerabilities flagged by the command are false alarms in common usage scenarios. And this appears to be a [2]fairly [3]widespread [4]sentiment among npm users.
[5]
More than a decade ago, Isaac Schlueter [6]created the npm package manager and co-founded a company under the same name that would later be absorbed by Microsoft's GitHub.
[7]
[8]
In April 2018, npm version 6 was released, bringing with it the audit command, because security in the npm ecosystem had become [9]something that could no longer be ignored . JavaScript developers using npm could thereafter type npm audit and they'd receive a security analysis of their projects' dependency tree – the various intertwined libraries imported into the project to avoid having to rewrite common functions from scratch.
The problem is npm audit overcorrected. Where a few years ago, JavaScript developers could look forward to being blindsided by security problems, npm runs its audit automatically after every npm install command and often produces a flood of vulnerability advisories that may not be easily fixable and may not really be applicable.
[10]Sitting comfortably? Then it's probably time to patch, as critical flaw uncovered in npm's netmask package
[11]Google pushes bug databases to get on the same page for open-source security
[12]Malicious backdoored NPM package masqueraded as Twilio library for three days until it was turfed out
[13]Python Package Index nukes 3,653 malicious libraries uploaded soon after security shortcoming highlighted
To some extent, the situation is unavoidable given the attack surface in the Node.js ecosystem, where the installation of an average npm package means trusting around 80 other packages due to transitive dependencies
[14]PDF
. But for Abramov, npm audit produces security warnings in contexts where the risks are not a realistic concern and the alert overload doesn't help anyone involved."The root of the issue is that npm added a default behavior that, in many situations, leads to a 99+ per cent false positive rate, creates an incredibly confusing first programming experience, makes people fight with security departments, makes maintainers never want to deal with Node.js ecosystem ever again, and at some point will lead to actually bad vulnerabilities slipping in unnoticed," he wrote.
Original npm crew agree
Kat Marchán, who helped create npm audit fix and is now a senior software engineer at Microsoft, responded [15]via Twitter , "This isn't wrong," while going on to explore some of the tradeoffs involved in security alerts and the decisions that led to the current state of affairs, some of which had to do with [16]NPM's management and labor challenges in 2018 and 2019.
"The feature overall, for the company, was kind of a marketing (scare-ish) tactic to promote its private registry service," Marchán [17]explained . "This isn't to say it was malicious: there was, and still is, a loud clamour for this kind of security visibility/improvement of the ecosystem."
[18]
Rebecca Turner, also involved in the creation of npm's auditing feature and now a principal engineer at Microsoft, also responded to Abramov's broadside, acknowledging that NPM's need for revenue shaped some design decisions.
"The start of the theatre was the report of the number of vulnerabilities found," Turner wrote in [19]a Twitter thread . "It didn't report the number of vulnerable modules, but the number of things ever depending on the vulnerable module, producing numbers often larger than the number of modules in the tree."
Turner said she'd have pushed back on that at the time if she'd realized the consequences, but testing didn't show an excessive number of advisories.
[20]
"No further development of this feature happened in the main-line life of npm," Turned said. "Priorities and resources were shifted elsewhere in the push for profitability. Discussion around how to make results more manageable was starting to happen the next year…"
"... but between the business pushing to develop those as premium features and the firing of half the CLI team for union organizing (followed by the other half, [21]myself included, resigning ) they never went anywhere."
It's not quite so simple as union busting broke npm audit. Crafting security alerts that provide just the right amount of information at just the right time in the appropriate context is a challenge. As Marchan [22]put it , "I personally spent a long time workshopping the CLI messages around audit to make them as unobtrusive as possible. But sometimes noise is noise, no matter how much you try to reduce it."
Further [23]code adjustments being considered could improve the situation by providing a manual way to resolve audit warnings, as could Abramov's [24]call for a way to exclude certain transitive dependencies from generating security warnings.
But calibrating the level of security concern to be appropriate for every individual and situation is a thankless task – dial back too much on the frantic hand waving and suppressed vulnerability mitigation advice might just lead to the next SolarWinds or Kaseya compromise. ®
Get our [25]Tech Resources
[1] https://overreacted.io/npm-audit-broken-by-design/
[2] https://twitter.com/tvanantwerp/status/1412804704103510019?s=20
[3] https://twitter.com/watware/status/1412858768933675020?s=20
[4] https://twitter.com/bascule/status/1412825923938836484?s=20
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/devops&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YOgeRqiCP@OCCgPd2@DYBQAAAEM&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[6] https://github.com/npm/cli/commit/4626dfa73b7847e9c42c1f799935f8242794d020
[7] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/devops&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YOgeRqiCP@OCCgPd2@DYBQAAAEM&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[8] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/devops&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YOgeRqiCP@OCCgPd2@DYBQAAAEM&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[9] https://www.theregister.com/2018/08/22/npm_vulnerability_scanner/
[10] https://www.theregister.com/2021/03/29/netmask_cve/
[11] https://www.theregister.com/2021/06/24/google_security_fix/
[12] https://www.theregister.com/2020/11/03/malicious_npm_package_masquerading_as/
[13] https://www.theregister.com/2021/03/02/python_pypi_purges/
[14] https://software-lab.org/publications/usenixSec2019-npm.pdf
[15] https://mobile.twitter.com/zkat__/status/1412914083020480515
[16] https://www.theregister.com/2019/04/22/npm_fired_staff_union_complaints/
[17] https://twitter.com/zkat__/status/1412916480438542340?s=20
[18] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/devops&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YOgeRqiCP@OCCgPd2@DYBQAAAEM&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[19] https://twitter.com/i_a_r_n_a/status/1413188069126139906?s=20
[20] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/devops&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YOgeRqiCP@OCCgPd2@DYBQAAAEM&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[21] https://www.theregister.com/2019/04/03/npm_product_manager_quits/
[22] https://twitter.com/zkat__/status/1412916482028179459?s=20
[23] https://github.com/npm/rfcs/pull/18
[24] https://twitter.com/dan_abramov/status/1412380714012594178
[25] https://whitepapers.theregister.com/
Job done ...
Sounds to me like the warnings worked then. Security depends on context and mitigation. So the tool flagged up warnings, you reviewed those warnings and decided they did not apply in your context. For a few warnings you decided they did apply. Working as intended.
So now you must document all that so that in future if the context changes you have a reference for further review. This is called being professional.
99% false positives is worse than nothing
You need below 5% false positives at the very worst, 1% is better (0% is impossible). Remember when Windows implemented UAC and programs triggered it every 5 minutes (because they weren't UAC-aware at the time) so everyone just turned it off because it was worse than useless? This is like that. You're not even going to notice a legitimate security issue in all the spam.
Eventually, if npm is actually more interested in making it useful rather than revenue enhancement, I think this can be largely mitigated. It's just a retread of what's been done before with compiler errors/warnings and lint errors/warnings though most of the JS people may have no idea what that even means or that this is hardly a new problem. The compiler (checker) gets better at deciding what's a real error and what's just a warning, the package authors get better about adapting their code to the compiler (checker) - which usually always results in better code - and then you get the option to manually disable specific things for specific packages. Just getting rid of the cascading errors (where a single thing generates 20 errors) would be a big start.
The difference between good and bad security product
Is the bad one are pushing tons of errors without checking dependencies/applicability in the context.
This is not only true for products for SW, but also for infra/cloud products.
For infra/cloud, there are one or 2 that do elaborated links models between security issues, and rate them according to applicability.
Basically, red means you're already pwned already.
Rest should be analyzed carefully, minus the info that can be ignored.
This is strangely redolent..
... of the present England approach to coronavirus: the data looks bad so let's ignore it.
Frankly, if installation of an average npm package means trusting around 80 other packages there is something very wrong with the packaging structure and the alert overload is a clear warning of that. Given the amount of code you'd have to carefully analyse to claim that there is a 99+ per cent false positive rate , I'm not sure how credible that figure really is.
And this is [1]not the only measure of the apparent fragility of this ecosystem.
[1] https://www.theregister.com/2016/03/23/npm_left_pad_chaos/
In one of the last projects I worked on, we used a library to generate random numbers, and the audit tool complained that the random numbers generated by the library were not good enough to be used for cryptography. Fine - but we were using them for simulations, not for cryptography!
We used another library that had many features, including some that went online and depended on a vulnerable SSL implementation. Thanks - but we were not using any online features at all!
Then there was another that had a SQL injection problem. Okay - but in our project we weren't even using a SQL database!
Then we had a whole gaggle of vulnerabilities in a test project. I get it - but it's a test project; it's not going to be Internet-facing at any time!
I understand that any of those issues might suddenly become real issues... but only if some specific sets of other much bigger mistakes happen. It's not worth ripping and replacing an entire library just because some insane person in the future might accidentally decide to remove the localhost-only restriction on the test suite and then put it online. Especially considering that the new library you just spent twenty hours replacing will most likely get flagged for something else in a couple months' time anyway, and it still won't matter because we are still not using it in a way where the vuln is relevant.
And yet, with all of that said, there were a few cases where the tool pointed out stuff that actually needed addressing.
This is definitely a problem, although I'm not sure how it could be solved cleanly and efficiently.