Rust projects open to denial of service thanks to Hyper mistakes
- Reference: 1672986608
- News link: https://www.theregister.co.uk/2023/01/06/flaws_rust_projects_ddos/
- Source link:
Security firm JFrog found that an undisclosed number of projects incorporating [1]Hyper , like [2]Axum , [3]Salvo and [4]conduit-hyper , were susceptible to denial of service (DoS) attacks arising from HTTP requests crafted to take advantage of the vulnerabilities.
Those three have fixed their code, but an undisclosed number of other vulnerable projects have not yet responded, according to JFrog. Currently, 2,579 projects listed in Rust's package repository crates.io depend on Hyper, which has been downloaded more than 67 million times.
[5]
"These multiple vulnerabilities in various Rust packages stem from the same root cause – forgetting to set proper limits on HTTP requests when using the Hyper library," said Shachar Menashe, senior director of JFrog Security Research, in [6]a blog post . "The lack of size limitations while using Hyper is a very serious issue that can be easily exploited by attackers to crash both HTTP clients and servers."
RTFM
The problem identified by the JFrog researchers resides in [7]to_bytes , a function for copying a request or response body to a single Bytes buffer. It can be invoked in an inadvisable manner (for both [8]unsafe and safe Rust) by failing to set header size limits – something explicitly warned against in the [9]function's documentation .
The function reads chunks of data and can create a Vector with enough space for the expected length of the request body. But the size of the Vector comes from the "Content-Length" header which gets passed directly to the Rust memory allocator, explain Menashe and JFrog researcher Ori Hollander. So if that's too large, the allocator will panic and crash the process.
[10]
[11]
Ensuring that the "Content-Length" header value is suitably sized has been left to developers. [12]RFC 7230 for HTTP/1.1 Message Syntax and Routing (June 2014) notes, "HTTP does not place a predefined limit on the length of each header field or on the length of the header section as a whole" and advises developers that servers receiving a larger-than-anticipated request header field or set of fields should respond with a 4xx client error status code.
[13]If GNU please: Rust support merged for the forthcoming GCC 13
[14]Google says Android runs better when covered in Rust
[15]Linux kernel 6.1: Rusty release could be a game-changer
[16]Sting op takes down 50 DDoS-for-hire domains, seven people collared
This issue has come up [17]before . [18]In 2014 and [19]in 2015 , Hyper's developers fixed DoS vulnerabilities arising from receipt of excessively large request headers. Last year, in a related [20]issues post , Rust developer Michal Varner suggested the incorporation of a warning mechanism.
He wrote, "[I]f the client is sending with chunked encoding, the header is optional. That makes the function
[21]aggregate
(and its sibling to_bytes ) potential DoS hazard."We know how to write secure code. We're just not very good at doing so. ®
Get our [22]Tech Resources
[1] https://github.com/hyperium/hyper
[2] https://github.com/tokio-rs/axum
[3] https://crates.io/crates/salvo
[4] https://crates.io/crates/conduit-hyper
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2Y7f-S86WvxYdpUzeFX9TsgAAABQ&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[6] https://jfrog.com/blog/watch-out-for-dos-when-using-rusts-popular-hyper-package/
[7] https://docs.rs/hyper/latest/hyper/body/fn.to_bytes.html
[8] https://doc.rust-lang.org/book/ch19-01-unsafe-rust.html
[9] https://docs.rs/hyper/latest/hyper/body/fn.to_bytes.html
[10] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Y7f-S86WvxYdpUzeFX9TsgAAABQ&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[11] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Y7f-S86WvxYdpUzeFX9TsgAAABQ&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[12] https://www.rfc-editor.org/rfc/rfc7230#section-3.2.5
[13] https://www.theregister.com/2022/12/15/gcc_13_rust_support_merged/
[14] https://www.theregister.com/2022/12/02/android_google_rust/
[15] https://www.theregister.com/2022/12/09/linux_kernel_61_column/
[16] https://www.theregister.com/2022/12/15/ddos_sites_takedown_fbi_europol/
[17] https://github.com/hyperium/hyper/issues?q=is%3Aissue+dos+header+is%3Aclosed+
[18] https://github.com/hyperium/hyper/issues/187
[19] https://github.com/hyperium/hyper/issues/256
[20] https://github.com/hyperium/hyper/issues/2414
[21] https://docs.rs/hyper/0.14.*/hyper/body/fn.aggregate.html
[22] https://whitepapers.theregister.com/
Re: RTFM
" on your MSDN CD's,"
CD's
Right - so what belongs to this CD?
Maybe, just maaaaaaaaybe, the library should fail safe, instead of failing open. Make a limit the default and require anyone needing more to explicitly state that, instead of starting with no limit and an obvious resource exhaustion problem, but then burying the proscription against that in the individual function docs.
Saying "don't do this" but then making it the default behavior is one of the most asinine security practices ever. What exactly did they expect to happen?
The warning was hardly buried - it's right there on the documentation page (https://docs.rs/hyper/latest/hyper/body/fn.to_bytes.html) between the function's signature and the example of how to use it. It's not unreasonable to expect users to think about what they're doing when calling a function that clearly could allocate a load of memory.
The article's mention of the content-length header is a little misleading: the to_bytes function doesn’t implement any length checks so it doesn't matter what's in the content-length header if the user's code doesn't check that itself.
I am glad people always read manuals and terms& conditions first and definitely will not cut any corners.
IMHO these checks should be enforced on both by Hyper and people using it in their code. That is the only way to make sure these things would happen in a scale of once in a million.
So Rust is not memory safe then
When you tell people they don't have to think about something, they won't.
This is a bug in Hyper, not the developers using it.
Everything has a maximum. "Unlimited" is never true.
Either handle the case of "insufficient resources", set a default maximum or require library users to set a maximum.
And then handle "insufficient resources" anyway, because it's going to happen sooner or later.
Does Rust really just terminate the process because a memory allocation failed?
Re: So Rust is not memory safe then
I concur and my jaw dropped reading the article. All the hyper around Rust is that it supposedly protects the developer from mistakes like this. I'm sorry, writing "don't do this" in TFM is not what I would associate with the future of the brave new world of magically secure systems programming.
For those who can't be bothered looking up the function in question it says
"Care needs to be taken if the remote is untrusted. The function doesn’t implement any length checks and an malicious peer might make it consume arbitrary amounts of memory."
To which I respond "then what did we gain from Ye Olde C Days?"
Re: So Rust is not memory safe then
"then what did we gain from Ye Olde C Days?"
We have gained a new programming language with some advantages and some disadvantages. Programmers still need to be programmers and fools will still be fools. We will have a new body of code, which needs to be handled. This body of code will be just as vulnerable to the programmer's logic errors, omissions and short-cuts than other programming languages.
Basically, everything changed and everything stayed the same.
Re: everything changed and everything stayed the same.
... and everyone's guilty, but no-one's to blame?
Re: So Rust is not memory safe then
As the French would say:
Plus ça change, plus c'est la même chose
Re: So Rust is not memory safe then
This is not an issue of memory safety. In the event that the length in the header is too long then the process will panic and stop which is defined behaviour. It is a problem because it can lead to a DOS, but it does not invoke undefined behaviour (for example, in 'Ye Olde C Days', a failure to check an input length and reading into a fixed length buffer could cause a buffer over-run - this is not possible in safe Rust).
Re: So Rust is not memory safe then
The looking-over-their-shoulders C programmers will jump on anything like this to comfort themselves.
Rust Not Different Here From C, C++ or Java
See http://sappeur.ddnss.de/discussion.html, section D9
Re: So Rust is not memory safe then
Everything has a maximum. "Unlimited" is never true.
I refer you to Einstein's quote about the universe and human stupidity.
Re: So Rust is not memory safe then
Panic in rust is not a old school interrupt 'panic and shit the bed while the disk is still spinning', it will unwind the stack etc so still memory safe. The library developers have called panic, which is essentially a process exit. So no 'danger' (except your server goes down, oops).
e.g. an if branch:
panic!("this will never happen")
Still a crap design, I totally agree with safe defaults... why allow a trivial DOS, just set to OWASP recommendation but allow up to platform max
Thanks
Your reasoning is the proper one. A deterministic crash is much better than Silent Subversion. See http://sappeur.ddnss.de/discussion.html section D9
Rust, An FSD Language?
When you tell people they don't have to think about something, they won't.
See: Tesla FSD :)
I'm not sure I'd call it a Hyper bug, as 'they' are presumably sticking to the [1]RFC linked in the article. But I do think it's a little odd that the door wasn't closed, beyond the RFC, to support the Rust "FSD programmers" - which is to say: I defo think it is programmer deficiency/neglect/ignorance. To me, this means that the devs on the large dependent projects (eg. Axum, Salvo and conduit-hyper) that didn't handle this would not be receiving any cookies, but rather directions to remedial programming courses.
In all else, and not staying true to the RFC, I agree with you.
[1] https://www.rfc-editor.org/rfc/rfc7230#section-3.2.5
Out Of Memory in C, C++, Java, Rust
In all the above languages, you will get a deterministic crash if heap allocation fails. You either get a NULL pointer from malloc() or new or some sort of OutOfMemoryException. Accessing a NULL pointer typically creates (some sort of) SIGSEV and stops the program. OutOfMemoryException typcially stops the thread.
This is exactly what you want. A deterministic, debuggable crash from a programming error/cybernetic attack. Much better than Silent Subversion from e.g. a buffer overflow.
How else could an out of memory condition be handled ?
(this applies to Windows, Linux, BSD, HPUX, Solaris, AIX, but maybe not to embedded systems)
FALSE
Please look at http://sappeur.ddnss.de/discussion.html , section D9 for why you are wrong.
Resource Limit Management != Memory Safety
The management of RAM allocation, database connection numbers, file handles, number of threads etc must be managed by the application programmer. There is no sensible way an automatic runtime mechanism can do this for the app programmer. Except, of course, stopping the thread or program upon resource exhaustion.
So - the application programmer must think about all the resources he allocates in his program. For example, an http server must reject too many parallel requests(Code 429 Resource Exhausted). An application using database handles must limit the number of database connections by some sort of pooling and semaphores. No automatic mechanism on the runtime/language level can replace programmer reasoning here(except maybe some sort of database pool which blocks until a connection becomes free).
Memory Safety is not the paradise of programming, it "just" eliminates an ugly kind of cancer.
Software Engineering is a highly complex craft+science with lots of aspects. If it were simple, we would not earn good money on it.
RTFM
https://bulgier.net/Q209354.htm