PyTorch dependency poisoned with malicious code
(2023/01/04)
- Reference: 1672840813
- News link: https://www.theregister.co.uk/2023/01/04/pypi_pytorch_dependency_attack/
- Source link:
An unknown attacker used the PyPI code repository to get developers to download a compromised PyTorch dependency that included malicious code designed to steal system data.
Developers who last week downloaded the nightly builds of the open source PyTorch framework also unknowingly installed a malicious version of the torchtriton dependency found in the Python Package Index, according to PyTorch's maintainers.
In a [1]blog post this week, PyTorch recommended those who installed the PyTorch nightly on Linux through pip between December 25 and December 30 to uninstall it and use the latest nightly binaries that were released after December 30.
[2]
They said devs using the PyTorch stable packages were not affected by the malicious binary.
'Security research gone bad'
However, the extent of the attack is unclear. A person taking responsibility for the incident said it was part of a security research project that went awry. They apologized, saying they erred in not making this clear and that they've deleted all data that was exfiltrated. A copy of the note can be found at the bottom of this [3]Checkmarx blog .
The dependency confusion attack included uploading a copy of torchtriton – a legitimate dependency – that was laced with malware to PyPI, an online repository of packages for Python developers. The compromised torchtriton package came with the same name as the one PyTorch maintainers ship on the PyTorch nightly package index.
[4]
[5]
"Since the PyPI index takes precedence, this malicious package was being installed instead of the version from our official repository," they wrote. "This design enables somebody to register a package by the same name as one that exists in a third party index, and pip will install their version by default. This malicious package has the same name torchtriton but added in code that uploads sensitive data from the machine."
The sensitive data includes nameservers, hostnames, the current username and working directory name. In addition, it accessed a range of files, including /etc/hosts, /etc/password/, the first 1,000 files in $HOME/*, $HOME/.gitconfig, and $HOME/.ssh*.
[6]
The malicious binary would upload files ranging in size up to 99,999 bytes and send the contents to a specified domain.
PyTorch – like Keras, TensorFlow and Jax – is a framework developers can use for machine learning applications like natural language processing and computer vision. It's based on the Torch library and was developed by Meta AI, though it now is under the auspices of the Linux Foundation.
The PyTorch maintainers have taken several steps to fix the issue, including removing torchtriton as a dependency for the nightly packages and replacing it with pytorch-triton. In addition, they registered a dummy package on PyPI to prevent similar attacks.
[7]Crooks copy source code from Okta's GitHub repository
[8]Malicious PyPI package found posing as a SentinelOne SDK
[9]US military goes zero-trust on software and government gets busy
[10]Still using a discontinued Boa web server? Microsoft warns of supply chain attacks
All nightly packages that depend on torchtriton were removed from the package indices and they are asking the PyPI security team for greater ownership of the torchtriton package and to delete the malicious version.
PyPI and other open source code repositories have become a target in supply chain attacks. A malicious package on PyPI was found last month [11]masquerading as a legitimate SDK from SentinelOne and Phylum in November identified a campaign distributing the [12]W4SP malware via PyPI packages.
[13]
During the summer, PyPI talked about a [14]phishing attack against developers using the index and said it was offering security keys for two-factor authentication for projects with the most downloads over the previous six months.
In April, the Open Source Security Foundation said it [15]created a community-based working group to address the issue of securing software repositories.
"This attack is the first known significant dependency confusion attack on the PyPi ecosystem," Zack Tzachi, head of software supply chain for Checkmarx, wrote in the blog post. "Dependency confusion attacks were first revealed by Alex Birsan in 2021. Since then, the technique has been used countless times in both PyPI and NPM package registries."
While attacks on software library dependencies aren't new, they've become more frequent, said Mike Parkin, senior technical engineer at Vulcan Cyber, adding that the greater visibility should lead to more mitigations.
"The question is whether the primary fix will fall to the repositories to make sure libraries they distribute aren't compromised or to the developers who use those libraries to make sure the expected library is loaded," Parkin told The Register .
John Bambenek, principal threat hunter at Netenrich, told The Register that while there are benefits to open source software, there is little institutional protection beyond an almost entirely voluntary effort to address inherent supply chain risks. Until more money is directed to the issue, the problems will continue, Bambenek said. ®
Get our [16]Tech Resources
[1] https://pytorch.org/blog/compromised-nightly-dependency/#how-to-check-if-your-python-environment-is-affected
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2Y7WwrpoZdBHvCy1utPt21QAAAEw&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://medium.com/checkmarx-security/py-torch-a-leading-ml-framework-was-poisoned-with-malicious-dependency-e30f88242964
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Y7WwrpoZdBHvCy1utPt21QAAAEw&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Y7WwrpoZdBHvCy1utPt21QAAAEw&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Y7WwrpoZdBHvCy1utPt21QAAAEw&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[7] https://www.theregister.com/2022/12/23/okta_code_copy_hack/
[8] https://www.theregister.com/2022/12/21/pypi_malware_sentinelone/
[9] https://www.theregister.com/2022/11/23/dod_cisa_omb_cybersecurity/
[10] https://www.theregister.com/2022/11/23/microsoft_boa_web_server/
[11] https://www.theregister.com/2022/12/21/pypi_malware_sentinelone/
[12] https://www.theregister.com/2022/11/16/wasp_python_malware_checkmarx/
[13] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Y7WwrpoZdBHvCy1utPt21QAAAEw&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[14] https://www.theregister.com/2022/08/26/pypi_warns_of_firstever_phishing/
[15] https://openssf.org/blog/2022/04/19/your-favorite-software-repositories-now-working-together/
[16] https://whitepapers.theregister.com/
Developers who last week downloaded the nightly builds of the open source PyTorch framework also unknowingly installed a malicious version of the torchtriton dependency found in the Python Package Index, according to PyTorch's maintainers.
In a [1]blog post this week, PyTorch recommended those who installed the PyTorch nightly on Linux through pip between December 25 and December 30 to uninstall it and use the latest nightly binaries that were released after December 30.
[2]
They said devs using the PyTorch stable packages were not affected by the malicious binary.
'Security research gone bad'
However, the extent of the attack is unclear. A person taking responsibility for the incident said it was part of a security research project that went awry. They apologized, saying they erred in not making this clear and that they've deleted all data that was exfiltrated. A copy of the note can be found at the bottom of this [3]Checkmarx blog .
The dependency confusion attack included uploading a copy of torchtriton – a legitimate dependency – that was laced with malware to PyPI, an online repository of packages for Python developers. The compromised torchtriton package came with the same name as the one PyTorch maintainers ship on the PyTorch nightly package index.
[4]
[5]
"Since the PyPI index takes precedence, this malicious package was being installed instead of the version from our official repository," they wrote. "This design enables somebody to register a package by the same name as one that exists in a third party index, and pip will install their version by default. This malicious package has the same name torchtriton but added in code that uploads sensitive data from the machine."
The sensitive data includes nameservers, hostnames, the current username and working directory name. In addition, it accessed a range of files, including /etc/hosts, /etc/password/, the first 1,000 files in $HOME/*, $HOME/.gitconfig, and $HOME/.ssh*.
[6]
The malicious binary would upload files ranging in size up to 99,999 bytes and send the contents to a specified domain.
PyTorch – like Keras, TensorFlow and Jax – is a framework developers can use for machine learning applications like natural language processing and computer vision. It's based on the Torch library and was developed by Meta AI, though it now is under the auspices of the Linux Foundation.
The PyTorch maintainers have taken several steps to fix the issue, including removing torchtriton as a dependency for the nightly packages and replacing it with pytorch-triton. In addition, they registered a dummy package on PyPI to prevent similar attacks.
[7]Crooks copy source code from Okta's GitHub repository
[8]Malicious PyPI package found posing as a SentinelOne SDK
[9]US military goes zero-trust on software and government gets busy
[10]Still using a discontinued Boa web server? Microsoft warns of supply chain attacks
All nightly packages that depend on torchtriton were removed from the package indices and they are asking the PyPI security team for greater ownership of the torchtriton package and to delete the malicious version.
PyPI and other open source code repositories have become a target in supply chain attacks. A malicious package on PyPI was found last month [11]masquerading as a legitimate SDK from SentinelOne and Phylum in November identified a campaign distributing the [12]W4SP malware via PyPI packages.
[13]
During the summer, PyPI talked about a [14]phishing attack against developers using the index and said it was offering security keys for two-factor authentication for projects with the most downloads over the previous six months.
In April, the Open Source Security Foundation said it [15]created a community-based working group to address the issue of securing software repositories.
"This attack is the first known significant dependency confusion attack on the PyPi ecosystem," Zack Tzachi, head of software supply chain for Checkmarx, wrote in the blog post. "Dependency confusion attacks were first revealed by Alex Birsan in 2021. Since then, the technique has been used countless times in both PyPI and NPM package registries."
While attacks on software library dependencies aren't new, they've become more frequent, said Mike Parkin, senior technical engineer at Vulcan Cyber, adding that the greater visibility should lead to more mitigations.
"The question is whether the primary fix will fall to the repositories to make sure libraries they distribute aren't compromised or to the developers who use those libraries to make sure the expected library is loaded," Parkin told The Register .
John Bambenek, principal threat hunter at Netenrich, told The Register that while there are benefits to open source software, there is little institutional protection beyond an almost entirely voluntary effort to address inherent supply chain risks. Until more money is directed to the issue, the problems will continue, Bambenek said. ®
Get our [16]Tech Resources
[1] https://pytorch.org/blog/compromised-nightly-dependency/#how-to-check-if-your-python-environment-is-affected
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2Y7WwrpoZdBHvCy1utPt21QAAAEw&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://medium.com/checkmarx-security/py-torch-a-leading-ml-framework-was-poisoned-with-malicious-dependency-e30f88242964
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Y7WwrpoZdBHvCy1utPt21QAAAEw&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Y7WwrpoZdBHvCy1utPt21QAAAEw&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Y7WwrpoZdBHvCy1utPt21QAAAEw&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[7] https://www.theregister.com/2022/12/23/okta_code_copy_hack/
[8] https://www.theregister.com/2022/12/21/pypi_malware_sentinelone/
[9] https://www.theregister.com/2022/11/23/dod_cisa_omb_cybersecurity/
[10] https://www.theregister.com/2022/11/23/microsoft_boa_web_server/
[11] https://www.theregister.com/2022/12/21/pypi_malware_sentinelone/
[12] https://www.theregister.com/2022/11/16/wasp_python_malware_checkmarx/
[13] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Y7WwrpoZdBHvCy1utPt21QAAAEw&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[14] https://www.theregister.com/2022/08/26/pypi_warns_of_firstever_phishing/
[15] https://openssf.org/blog/2022/04/19/your-favorite-software-repositories-now-working-together/
[16] https://whitepapers.theregister.com/
Lil Endian
Bambenek is referring to the supply chain (eg. PyPI), not an individual package (eg. PyTorch).
Regardless, his statement says nothing. Risks apply to closed source supply chains as to OSS. However, exposure/visibility are not the same, nor is funding - and the two are tied. Institutionally, a proprietary for-profit dev house isn't going to advertise its fuck ups if it can help it, because: ow! profit!
Logical extension of a ML package
Elongated Muskrat
As far as I can tell, ML (such as those "AI" art algorithms) is largely based on slurping up as much "training data" as possible and then denying all knowledge when it produces things that are suspiciously similar to copyrighted works. This is just the logical next step - cut out the middle man and just move directly to stealing valuable data.
In all seriousness, though - why is this trusting a third-party package repository over an official one, and if they want any credibility at all, why does that third-party repository just allowing people to upload anything they like, including packages with identical names to ones in the official repository?
I expect to be reading more about how their security model is woefully broken in Bruce Schneier's next blog post...
Epeen > Research
Lil Endian
@Anonymous Security Researcher: giving you the benefit of the doubt, and believing your claims of research, then clearly your epeen is more important to you than following genuine research principles.
This is how a significant number viruses came into being during the 80s & 90s: a disenfranchised hacker all alone in their bedroom, wanting to prove their mettle.
Re: Epeen > Research
Anonymous Coward
She only gets the benefit of the doubt if he publishes her approved thesis documents so we know who to blame for approving them in the first place, with no notification of the package maintainers. Seems some other university did that within the last 2 years and were banned from kernel changes.
"John Bambenek, principal threat hunter at Netenrich, told The Register that while there are benefits to open source software, there is little institutional protection beyond an almost entirely voluntary effort to address inherent supply chain risks. Until more money is directed to the issue, the problems will continue, Bambenek said."
The problem with PyTorch is... that there isn't enough money behind it? Bold claim.