From File to Object and Back Again
If you deal with storage, you’re certainly noticing an increase in talk around object storage. Many analysts position it as a next big thing, something that may address the ever-increasing growth of unstructured data.
From the recent Gartner Critical Capabilities for Object Storage report recommendations:
- Choose object storage products as alternatives to block and file storage when you need huge scalable capacity, reduced management overhead and lower cost of ownership.
- Build on-premises object storage repositories with the hybrid cloud in mind, and evaluate their API support and level of compatibility.
- Select object storage vendors that offer a wide choice of deployment (software-only versus packaged appliances versus managed hosting) and licensing models (perpetual versus subscription).
- Train developers on best practices related to application design and the operational considerations relevant to an object storage system.1
There are certainly some interesting capabilities offered by object storage solutions (metadata search, global accessibility, etc). However, the most common reasons for customers’ interest in object storage are large storage capacity, with huge amount of documents, for (perceived) lower cost.
Today, the majority of customers’ on-premises unstructured data is stored on NAS systems. The data is being accessed by numerous applications – all using well-known file protocols, such as NFS and SMB. Object storage uses different protocols – Amazon-driven S3, OpenStack-driven Swift, SNIA-defined CDMI and numerous other proprietary protocols coming from over a dozen different object storage vendors.
Transitioning existing applications from a file to object interface is extremely hard. As a result, some of the object storage vendors offer a “file-protocol gateway,” something that allows NFS or SMB access to the underlying object storage, hiding object protocols from the application. I was recently talking to a customer, and he had asked about INFINIDAT’s object storage solution. My question to the customer was “would you prefer S3 or Swift protocol?” The answer was “no, we need it to support NFS, we just think the cost of object storage should be lower.”
This was not the first time I met a potential customer who had been driven to something new, which he potentially didn’t really need, because of excessive marketing. As I wrote earlier, INFINIDAT offers a highly-scalable NAS solution as part of our unified storage, which allows customers to have highly-reliable, PB-scale NAS to store billions of files with attractive economics and familiar file access. This solution might be more than sufficient for that customer.
In addition, INFINIDAT NAS offers multiple additional capabilities, such as high throughput and low latency, efficient snapshots, compression, thin provisioning, data replication, etc.
INFINIDAT has numerous customers who run large scale on-premises storage environments leveraging our InfiniBox systems. Many of them are thinking about the way to bridge between their on-prem environment and a public cloud. In my next post, I’ll describe how InfiniBox file and block storage can be used in public clouds, such as AWS and Azure. This may provide the easiest path for transition from private cloud into a hybrid deployment, without the need to invest in adoption of new protocols.
Object storage can certainly be attractive for those willing to invest in the new generation of applications, which may leverage new capabilities such as rich metadata and search capabilities. We’re working on several offerings in this space as well, leveraging our reliable storage as a basis for object stores for those who might really need one.
Let’s review how a reliable object storage offering can be created using INFINIDAT as a backend storage:
Traditionally, object storage is deployed using local disks. Using highly reliable, shared INFINIDAT storage instead provides the following benefits:
- Reduce demand for multiple replicas or CPU-hungry erasure code calculations common with traditional object deployments.
- Lower amount of replicas also yields more efficient object cluster network utilization
- Efficient snapshots and consistency groups provide consistent backup and rollback functionality for large scale object environments
- Much higher capacity (hundreds of TB) can be allocated per object node, reducing overall amount of object nodes
- Depending on the type of data, InfiniBox compression may provide significant capacity savings
- Data-at-rest encryption
- SSD caching
Object storage is a great technology, and many customers can benefit from it. However, it is not a replacement for modern, scalable NAS systems. At INFINIDAT, we see growth in all domains – block, file and object, and we’re working with our customers to support their private, hybrid and public-cloud based needs.
1 Gartner “Critical Capabilities for Object Storage” by Arun Chandrasekaran, Raj Bala, Garth Landers, March 31, 2016.