How to Pretend That Flash is Cheaper Than Disk

Recently, Dell EMC published a white paper claiming that the long-awaited price inflection point between Solid State Drives (SSDs) and Hard Disk Drives (HDDs) – not “Hark” disk drives as they incorrectly call them – has finally arrived. Unfortunately, their reasoning is self-serving and disingenuous and simply proves the classic George Costanza posit that something is “not a lie if you believe it.”

Let’s face it. Storage vendors aren’t nonprofits. They need to make a return for their investors just like any other company. They cannot provide that return if their revenue streams don’t grow. And they can’t make their revenue streams grow unless they deliver what their customers demand (or more).

The storage layer of the data center stack is fundamental to application performance and hence can be a significant bottleneck in the form of I/O latency and throughput constriction. To deliver higher performance storage systems, the overwhelming strategy of (nearly) all storage vendors in the past has been to incorporate the newest, fastest, most expensive persistent media technology in their product lines.

Historically, that media has been spinning disks. Architectures varied by size, capacity, feature set, etc., but persistent media was stratified into various speed HDDs – from low cost, high capacity 7.2K RPM to high cost, low capacity 15K RPM HDDs. As applications demanded better performance, the consumer electronics market enabled production of a lower latency, lower capacity, persistent storage technology in the form of very expensive Solid State Drives (Flash). Rather than do the hard work of developing an architecture that sensibly used SSDs to deliver higher performance while reducing the overall cost of storage, vendors followed their old playbook. Within a very short time, SSDs replaced 15K HDDs for 100% of the persistent storage layer in their products – driving production costs up significantly and passing that cost through to their customers. The more things change, the more they stay the same.

Customers, however, are experiencing new performance demands and eroding IT budgets, coupled with increased capacity requirements. In the Dell EMC whitepaper referenced above, the claim is made that SSDs have reached the cost inflection point where their capacity is the same or lower cost than HDDs. No supporting evidence or detail is given for this assertion.

The fact is, while the declining price per GB of SSDs vs. HDDs has accelerated, both media types continue to develop denser drives. Add to this that the starting price gap was initially on the order of 20X or more and HDDs continue to be much less expensive than their solid-state counterparts and will be into the foreseeable future. The following two charts illustrate this gap.

Source: Gartner Forecast: Hard-Disk Drives, Worldwide, 2014-2021, 1Q17 Update

 

Source: Cloud Hyperscale and Enterprise Storage Service CQ4 ’15 Quarterly Update and Long-Term Forecast TrendFocus [February 2016]

How then can the author of the Dell EMC white paper assert the following? –
“However, the cost of flash has fallen faster, and sometime in 2016, flash reached dollar per gigabyte parity with HDDs.“

Since the introduction of SSDs as enterprise storage media, storage vendors and
industry pundits have made the simplistic assumption that they can only be compared to the highest priced, lowest density HDD class – 15K RPM drives (which they all stopped using when switching to all flash). Their reasoning has been that no vendor could compete performance-wise with SSD latency while primarily using High-Density Slow Rotation (HDSR,7.2K RPM near-line SAS) drives in their arrays. And so, while the gap between SSD and near-line SAS drives is still huge and is expected to be huge into the future, they view the comparison as irrelevant.

Well, at INFINIDAT, we see this huge price gap as an opportunity to turn the
conventional thinking upside down. By using intelligent software engineering, we save our customers huge amounts of cash – all while delivering array densities like the Hyperscalers (Google, Amazon, Facebook), performance that exceeds that of All-Flash Arrays, and 7-9s of reliability.

How do we do it? Our engineers are the holders of a hundred or so software patents that serve to de-couple performance and reliability from the underlying persistent media. While utilizing only 3% SSD and 97% extremely low cost, near-line NL-SAS drives, our flagship InfiniBox arrays routinely smack around expensive, All-Flash Arrays in both performance and reliability.

So what does the future look like for SSDs? Capacities will continue to increase,
however, they will do so at the expense of media endurance. By moving from eMLC (4 states per cell) to TLC (8 states per cell), PE cycles decreased by a factor of 10 (30,000 to 3,000). And predictions for next-gen QLC chips suggest that trend will continue. That translates to higher failure rates and more cost to the storage vendor to fulfill warranties to their customer base, further eroding their profits.

In a recent article on the Data Machine website, the author made an important observation that profitability of companies selling All-Flash Arrays is exceedingly rare. Perhaps instead of spinning the facts and investing lots of marketing dollars to make a case for the extremely expensive, all-flash data center, companies should put their money to better use by developing products that deliver what their customers truly want – high value, low-cost storage that enables their businesses to thrive.

Why would a company pushing All-Flash Arrays as the one architecture for the
datacenter ignore these facts? I think we all know the answer to that.

About Chip Elmblad
Chip Elmblad is an Enterprise Architect at INFINIDAT. He is a 20 year veteran of
the IT industry, having focused on enterprise data storage for most of his career.
He has an M.S. in Computer Science from Azusa Pacific University in Southern
California and makes his home above the clouds in Divide, Colorado.

Comments

  1. I’m pretty sure EMC was using 15K drives as their comparison point, I had a bet internally at NetApp that Flash capacity would be less that 10K drives sometime this year .. I’m pretty sure that I’m going to lose that bet, and owe someone a nice bottle of single malt.

    Even so, the overall economics of an all flash solution outweigh that of a hybrid configuration in most cases, but effectively combing (a) performance tier(s) made of various kinds of solid state media with (a) capacity tier(s) made of near-line magnetic media will continue to make sense for a while. Having said that with the performance density of magnetic media getting exponentially lower, traditional caching and tiering approaches designed even a few years ago probably won’t scale. Even today a large capacity SSD with capacity efficiency technologies applied to it have the same performance density as a 500TB SATA drive .. in a year or two when we get 50 and 100TB SSD’s the only way magnetic drives will be able to maintain their price advantage at a reasonable level of blended performance/capacity point will be by introducing shingling and other techniques which will significantly decrease drive overwrites. In short you won’t be able to treat those drives in the same way as you treat drives today, and that transition is going to be very difficult for architectures that aren’t able to adapt quickly.

  2. You use only 3% SSD and the rest is 97% NL-SAS. Suppose 1: you have 1 PB of total storage, 2: the service time of the NL-SAS is 8.2 ms total, 3: suppose the SSD gives 0.2 ms service time. Then you run a benchmark of small random reads across a working set of the 1 PB. The observed response time will be dominated by the NL-SAS 8.2 ms service times. Assuming the request rate does not build queues the response can be expected to be 0.03*0.2ms+0.97*8.2ms = 7.96ms.

    Any AFA running the exact same thing will provide SSD response times across the entire working set. This will be *many* times faster than the NL-SAS showing.

    No caching algorithm can predict a random pattern. You can not decouple the performance of a system from its underlying components. This is a consequence of Amdahl’s law on system speedups. Under enough load a computing system exhibits the speed of its slowest component.

  3. INFINIDAT has been awarded over 80 patents (with many more pending award), many of which address the issue of randomness in workloads. Since DRAM is 1000 times faster that SSD, we optimize our platform to achieve 93% or more of IO as cache hits. In addition, we convert all random workloads (which in real life only seem random) to essentially sequential workloads by collecting access statistics over the life of the data in cache and using this information to layout the data for hyper-efficient pre-fetching (up to 4MB per fetch). Access locality is a much more efficient method for laying out data than temporal locality. Combine this with the fact that data resides in DRAM much longer than competitive platforms (5 minutes) and is then de-staged to SSD (warm data) and disk (cold data) and our performance eclipses All-Flash Arrays all day long. It’s the reason that customers who try INFINIDAT overwhelmingly buy more and more each quarter. Check out IDC’s recent report on our technology, or contact INFINIDAT for a demonstration. http://idcdocserv.com/US43193317.

Leave a Reply

Your email address will not be published. Required fields are marked *

×

Request a Demo