Won't duke, duke, duke the URLs: AWS backtracks on plans to block old-style S3 paths
- Reference: 1600958073
- News link: https://www.theregister.co.uk/2020/09/24/aws_backtracks_block_s3_paths/
- Source link:
S3, introduced in 2006, is one of the oldest AWS services. It is cloud storage where files can be accessed programmatically or published to the internet, this latter feature being a common cause of unintended data leaks.
In its original design, all S3 URLs began with an S3 domain such as s3.amazonaws.com or s3-eu-west-1.amazonaws.com . AWS calls these path-style addresses. This was later changed to support addresses where each "bucket", or container for files, is a subdomain. This type of URL begins yourdomain.s3.amazonaws.com and AWS calls it virtual hosted addressing. For example:
Old path style: https://s3.amazonaws.com/TestBucket/MyFile.txt
New virtual hosted style: https://TestBucket.s3-eu-west-1.amazonaws.com/MyFile.txt
Currently, buckets support both styles so the above examples could point to the same file.
AWS would like to remove support for path-style URLs. S3 has "many trillions of objects and processes millions of requests per second for them," according to its [1]post on the subject .
The path-style URLs means that all these requests go to a limited number of endpoints, whereas with the virtual hosted style, each bucket has its own entry in DNS, the distributed database through which the internet maps names to locations.
AWS said the virtual hosted style makes scaling easier as well as helping it to defend against DDoS (Distributed Denial of Service) attacks. Some new security features depend on using virtual hosted addresses.
There are complications, though. Domains and subdomains are not case-sensitive, but paths (everything after the double slash) potentially are. There are also issues with SSL (secure sockets layer) and wildcard certificates if there are dots in the bucket name. A wildcard certificate may match foo.somedomain.com but not foo.bar.somedomain.com.
These dotted bucket names are fine in the path-style URL, but AWS currently does not support them in the virtual hosted style. Another issue is that some characters are valid in the path but not in the domain name.
AWS did plan to end support for [2]path-style URLs completely on 30 September 30. Then it said, after protests from customers with applications that depend on them, that it would continue to support path style for existing buckets, but not for new ones created after the deadline.
Now it is has backtracked further, saying that it heard from "many customers who have asked us to extend the deprecation date". AWS said that it will postpone the deprecation to "ensure that customers have the time they need to transition to virtual hosted-style URLs."
No time scale is given, but there is a clue based on the dot-in-bucket issue. AWS said it is working on providing support for dotted bucket names, and that once it delivers that support, there will be at least a year before path-style URLs are no longer supported.
Ending path-style URLs for new buckets is not so bad, though developers may have to revise their code. Ending path-style support for existing buckets is more serious, because it breaks URLs for existing files, adding to the multitude of broken links that already afflict the internet as well as potentially breaking applications. ®
Get our [3]Tech Resources
[1] https://aws.amazon.com/blogs/aws/amazon-s3-path-deprecation-plan-the-rest-of-the-story/
[2] https://www.theregister.com/2019/05/10/amazon_backtracks_on_s3/
[3] https://whitepapers.theregister.com/
Ambidextrous
I am liking the new site feature for supporting both left and right hand sides of the pond, as per your fine example 30 September 30 (though possibly a th would help?)
Could I propse non-Reg units go through a similar thing? Maybe 68 Farencius 20? 20 feetres 6.1?
I didn't have anything of any value to add, no. Why do you ask?
Rosie
Other possible side effects:
(1) DNS tables entries could potentially become far more numerous, increasing the burden on DNS servers to update and maintain status.
At least the DNS servers will have to check whether the subdomain exists in the DNS table or the root should be used.
(2) Increased security risk as described here:
"There is no known security vulnerability in Let’s Encrypt that can be exploited. What is usually meant by hacker threat in this context is connected with the type of certificate validation. Let’s Encrypt and many other paid SSLs are domain-validated only (DV). This means that in order to issue the certificate, the CA (certificate authority) only checks if the certificate requester owns the domain. If a hacker manages to acquire access (usually through phishing) to your domain account at your domain registrar, they can create subdomains of your domain and issue security certificates for the subdomains as if they were the owner. This is called domain shadowing and can result in misleading people that they are visiting your website while in fact, it is a subdomain not related to your site at all."
Although (2) shouldn't be an issue with AWS, it is a problem with increased used of subdomains in general, especially as other cloud providers are pressed into following AWS's example, especially if they want to pass through great firewalls.
More time...
otherwise a load of unsupported IoT may stop working.
Oh dear.
Cool URIs don't change
> Ending path-style support for existing buckets is more serious, because it breaks URLs for existing files
Amazon should take a look at this [1]1998 article . Written by some guy called Tim Berners-Lee. Is he still known in the web biz?
[snarking aside, Amazon's decision has also implications on privacy and censorship. It is quite easy to block access to e.g. unwantedpoliticalopinion.s3.amazonaws.com. But dropping all traffic to s3.amazonaws.com? The back lash will be rather huge.]
[1] https://www.w3.org/Provider/Style/URI