Tape is so dead, 152.9 exabytes worth of LTO media shipped last year
- Reference: 1716784332
- News link: https://www.theregister.co.uk/2024/05/27/tape_shipments_2023/
- Source link:
HPE, IBM, and Quantum are the only three LTO Program Technology Providers, and last week jointly released [1]The Annual LTO Program Media Shipment Report [PDF] which revealed that 152.9-exabyte figure along with the tidbit that it represents 3.14 percent shipment growth compared to 2022.
The three attributed some of that growth to "rapid data generation and the increased infrastructure requirements of hyperscalers and enterprises." Which is good news for tape – if hyperscalers are using it that likely means demand will continue for the foreseeable future.
[2]
Because it's 2024 they also attributed some demand for tape to AI, which the trio described as just the sort of workload that spawns unstructured data and can "cause increases in storage requirements and costs."
[3]
[4]
Tape has huge capacity, and can easily be taken offline. Purveyors therefore commend it as an ideal medium for bulk data that isn't often accessed – a role in which it can often be cheaper than disk – and to protect data by literally putting it on the shelf and therefore out of reach of ransomware infections.
Just don't mention access or restore times, which are not swift. Or the reason for that asterisk * up there, which we included because the LTO trio's 1529EB figure refers to compressed tape capacity.
[5]
That matters, because each generation of LTO tape has a native capacity and a compressed capacity. In the case of the latest LTO generation – the ninth – compressed capacity is 45TB and native capacity is 18TB. LTO-8's numbers are 30TB and 12TB.
[6]Is there anything tape can’t fix? This techie used it to defeat the Sun
[7]If your storage admin is a bit excitable today, be kind: 45TB LTO-9 tape media and drives just debuted
[8]If your storage admin is a bit excitable today, be kind: 45TB LTO-9 tape media and drives just debuted
[9]Sony, Fujifilm storage patent lawsuit is all taped up: Better LTO-8 than never, right?
So suffice to say the 152.9EB figure is therefore a little less impressive that it appears at first blush. And again, remember that read and write times – and therefore recovery efforts – take even longer when there's compression or decompression to be done.
Know, also, that Seagate alone shipped 99 exabytes worth of hard disk drives in Q3 2024, as revealed in its late April [10]results presentation [PDF]. Other hard disk makers will also have shipped many exabytes, as will vendors of solid state storage.
That said, 152.9 exabytes is a decidedly non-trivial sum, even if the native capacity of tapes shipped last year was probably around a third of the headline figure.
Even at 50EB – 50,000 petabytes – that's a lot of cat videos, scraped-for-free LLM corpus data, log files, and whatever else it is that people put on tape so it's always around. Just like tape itself. ®
Get our [11]Tech Resources
[1] https://www.lto.org/wp-content/uploads/2024/05/LTOUltrium_2023_MediaShipmentReport.pdf
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/storage&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2ZlRZwhH2SfrEkBf-Ssh2FAAAAE4&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/storage&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZlRZwhH2SfrEkBf-Ssh2FAAAAE4&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/storage&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZlRZwhH2SfrEkBf-Ssh2FAAAAE4&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/storage&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZlRZwhH2SfrEkBf-Ssh2FAAAAE4&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[6] https://www.theregister.com/2023/05/12/on_call/
[7] https://www.theregister.com/2021/09/08/lto_9_debuts/
[8] https://www.theregister.com/2021/09/08/lto_9_debuts/
[9] https://www.theregister.com/2019/08/06/sony_fujifilm_storage_patent_lawsuit_settled/
[10] https://s24.q4cdn.com/101481333/files/doc_financials/2024/q3/STX-FQ3-24-Supplemental.pdf
[11] https://whitepapers.theregister.com/
Re: So The Next Time Your Service Provider Assures You That...........
Google archives or used to archive data on media that could not be overwritten or deleted. The way they handle the fact that it must be possible to delete data for legal reasons is by encrypting the data, and storing the decryption key separately. Instead of deleting the data, they delete the decryption key, so the archive becomes unreadable for all practical purposes.
Re: So The Next Time Your Service Provider Assures You That...........
Why not just destroy the media?
Re: So The Next Time Your Service Provider Assures You That...........
> Why not just destroy the media?
Because one lump of media can hold a lot more data than just the stuff that is to be "deleted"?
So destroying the whole lump would mean copying everything but the bad stuff to a new lump. Repeat when another small amount has to be destroyed, which gets costly quite quickly.
Instead, consider filling the lump with, say, n zip files, each one with is own password. Delete two passwords and (n-2)*100/n percent of the lump's value is retained.
Penalty
Dear Simon,
You have been mocking Oh Great Vendor products. The LTO revolution is now upon you and are sentenced to fill one tape manually and compressed with the text (*) : "I Will Not Mock LTO." .
The tape will be checked and you will be informed of any failures to restore.
Thank you for your article.
Signed HPE, IBM and Quantum.
(*) In alternating UTF-8 and EBCDIC, of course
"take even longer when there's compression or decompression to be done."
Does it?
I would have thought the rate limiting step would be getting the bits (compressed or not) on or off the magnetic media? Unless the cpu and memory inside the tape drives are seriously underpowered or slow, I would think a modern cpu could more than keep up.
Even then, if the drive can operate with hardware compression turned off, host based compression (and encryption) is an option. If you were going to use host based encryption you need to compress first anyway. (Would you really trust encryption in tape hardware?)
If you were to use asymmetric encryption (public key) to encrypt your archives and kept each decrypting key in a "tamper proof" device then destroying the device effectively deletes the archives so encrypted. The encrypting key doesn't need to be secret.
In practise I imagine you would use ephemeral symmetric encryption with those keys protected by the asymmetric cryptography.
Its ironic that in more than 50 years that archive storage is still magnetic tape based. "Breakthroughs" in optical/photographic/holographic technologies haven't got much past headlines. I vaguely recall decades ago a proposal to store such data on silicon(?) wafers using the same photolithographic(?) technology used in IC fabrication yet in 2024 we still trust our data to rust on great lengths of plastic. :)
Re: "take even longer when there's compression or decompression to be done."
I'm not an expert, but as far as i'm aware you're correct about the compression.
The limiting factor is the storage medium, and most vendors actually indicate the read and write speeds scale (almost) 1:1 with compression ratios (2:1 compression ratio would theoretically double the read and write speeds). [1] Source (IBM)
As far as encryption goes, [2] LTO says "Native hardware encryption typically affects less than 1% of tape drive performance.". This page also goes into the specifics of the encryption used, for how they manage the keys you'd need to look up the vendor documentation, IBM should have those publicly available.
[1] https://www.ibm.com/docs/en/ts4500-tape-library?topic=performance-lto-specifications
[2] https://www.lto.org/encryption/
Re: "take even longer when there's compression or decompression to be done."
yet in 2024 we still trust our data to rust on great lengths of plastic.
"Because it works" is a powerful technological argument although price is probably a factor as well.
152 ExaBytes? Pah. I bought a USB stick of that capacity on fleabay last week for just 29.99
Restore time on that USB: "Maybe some of it"
In tape we trust
I sleep better knowing our data is being carved into sophisticated linear rust* and taken offsite**
It scares me when I hear of companies that think replication is a sufficient backup, and long-term archive? Whassat?
It depresses me that requesting a tape library in our datacentre raises eyebrows and sarcastic comments.
* Yes, we do regularly test restore too.
** Yes, it is encrypted.
Re: In tape we trust
If they were that concerned about duration of data, they'd print it on vellum
Re: In tape we trust
That would be many flocken of sheep you'd have to peel to get the vellum.
Squish
> the LTO trio's 1529EB figure refers to compressed tape capacity.
I accidentally "compressed" a tape once (á la Apple's ad ). It didn't work very well
> which revealed that 152.9-exabyte figure along with the tidbit that it represents 3.14 percent shipment growth
So tape is getting a bigger slice of the Pi?
I can't get my head round that joke
You've got to get the right angle on it.
Have you got a Radius server to authenticate against?
Unmentioned, but...
Probably it's partly an anti-ransomware measure in part - it's hard for the ransomware gangs to mess with that offline storage even if they're in the system in some way while the backup is made.
So The Next Time Your Service Provider Assures You That...........
.......your stuff has REALLY been "deleted".........
.......you will know for sure that the claim is a bare-faced lie!!!!