“Looks like he’ll never need to buy more storage space again,” my friend idly remarked as we saw the row of five 6TB hard drives installed in the open case of our mutual friend’s computer.
“I bet that’s what the guy who bought the first 1GB hard drive said,” I replied.
30TB of hard disk space may seem like a lot today, but how long will it be until 30TB becomes the norm? By 2020, will our friend still be sitting comfortably on his 30TB, or will he need more space? Just how big will hard drives be by then? My friend’s comment sparked my curiosity, and so I set out to perform a back-of-the-envelope calculation to find some answers.
In science, we use what we call “back-of-the-envelope” calculations to arrive at rough estimates. They’re more than just educated guesses, but they use math simple enough to be resolved in your head or on a piece of scrap paper. For instance, the force of acceleration due to gravity is 9.81 meters per second squared, but for a back-of-the-envelope calculation, we would round that up to 10. We’re only looking for an answer that is accurate within an order of magnitude (i.e. factors of ten). But don’t worry – the math we’ll be doing today will be even simpler than that.
Now, one way of answering my questions would be to conduct extended research into the field of computer hardware and look for expert opinions on the progression of data storage technology. But a more entertaining approach would be to have some fun with basic math – let’s treat this problem like it were on a high school math exam. (Because what’s more fun than that, am I right?)
First, we need to make some simplifying assumptions. The questions we asked above are complex, so in order to arrive at an answer that is more than “we can’t know for sure,” we need to make things easier on ourselves.
Assumption 1: Hard drive size will continue to progress at the rate it has been progressing for the past decade.
This will form the basis for our estimate: looking for trends in past data and projecting into the future. If we take a look at the history of hard disks, we can see that the past 10 years have seen consistent progress at a steady rate. In 2005, Hitachi shipped the first 500GB drive. In 2007, Hitachi released the first 1TB drive. In 2009, Western Digital released the first 2TB drive. 2011 saw Seagate release the first 4TB drive. And the first 8TB drive shipped in 2014, courtesy once again of Seagate.
Barring the slight delay in the jump from 4GB to 8GB, what we’re observing is a doubling of disk size every other year. If we wanted to be pedantic, we would fit a trend line through these data to arrive at a rate of increase that is slightly less than doubling every two years, but this is a back-of-the-envelope calculation, remember?
Assumption 2: There will be no major breakthroughs that will revolutionize the industry.
Breakthroughs are inherently unpredictable, making them what statisticians call “outliers.” Outliers are data points that deviate significantly from the rest of the data, and while the practice of ignoring them when looking for trends is frowned upon by many, we can do so for the purposes of our simple calculation. You know, unless you can calculate polynomial functions in your head.
Sudden and dramatic jumps in disk space may occur, but unless we see them occur in predictable patterns, they don’t help us establish a trend. While we’re almost guaranteed a breakthrough at some point in the future, by keeping our estimates to the near future, it’s not entirely unreasonable to assume there will be no major breakthroughs, as there haven’t been any in in at least the past decade.
Assumption 3: Our storage needs will continue to increase over time.
The thought of ever filling 30TB of storage space today may seem like a fantasy, but that may not be the case in the future. Basically, this comes down to the computing corollaries of Parkinson’s law: “Data expands to fill the space available for storage,” and “Storage requirements will increase to meet storage capacity.” In other words, the more space we’re given, the more space we’ll fill up.
Until the year 2010, I was still (somehow) using a 60GB hard drive that I had for ages. The only reason I ended up swapping to a new drive was because the disk crashed. I can’t imagine how I managed on a 60GB drive, but I did. I swapped to a 1.5TB drive thereafter, and within a year, I had just about maxed out my space on it. My storage capacity increased by a factor of 25, and yet within one year I had adjusted my habits to accommodate the new amount of free space.
This effect isn’t just observed on the consumer’s side of the equation. Game developers adopt similar practices – as we moved from floppy to CD to DVD to digital download, games have grown larger and larger. With the advent of 4K resolution, we can expect this trend to continue unabated. The same can be said for movies and TV shows – the jump in resolution translates into tremendous file size increases. Compression technology just isn’t advancing as fast as graphics, because it simply doesn’t have to. Necessity is the mother of invention, and we don’t need better compression when we can easily get more storage space.
So where does that leave us?
If we continue our doubling trend, by 2016, we’ll have 16TB drives, 32TB drives by 2018, and by 2020, 64TB behemoths. By then, 30TB will still be respectable, but it certainly will no longer be impressive.
Of course, we’re talking about the largest drives commercially available to consumers. How would 30TB compare to the norm? A quick look at Amazon’s best-selling hard drives tells us that the average consumer currently has a 1TB disk. That’s one-eighth the size of the current largest commercially available drive. If we assume the same scaling, then 8TB drives will be the norm by 2020.
But when would 30TB become the norm? In 2022, the largest drives will be 128TB, and in 2024, 256TB – at which point 32TB drives will be average (one-eighth of 256).
As for whether or not my friend will need more storage space by 2020… Well, by Parkinson’s Law, he probably will have filled out most of it by then. Five years worth of games, movies, TV shows… it’ll add up, especially if he has no reason to delete anything. But just like I made due with a 60GB drive, that doesn’t necessarily mean he’ll need more space – he can simply adapt to what he has.
Now, how accurate are our answers? I suppose time will tell. What’s important is that we didn’t just hazard a random guess; we thought critically, based ourselves on data and facts, and put our precious little skull-dwelling jelly-monster to work. Perhaps you tackled this problem from a different angle and arrived at a different answer – and if you did, please do share your methods and results. As long as we’re thinking, we’re living.
