If you haven’t heard of Quobyte before, it was founded in 2013 by some ex-Googlers and HPC experts. The folks at Quobyte were heavily influenced by Google’s scale-out software model and wanted to bring that to the enterprise. Quobyte has had software in production since 2016 and has customers across a range of industry verticals, including financial services and media streaming. It’s not really object storage, more a parallel file system or, at a stretch, scale-out NAS.
Kolbeck describes Quobyte as “storage for Generation Scale-Out” and is focussed on “getting storage out of the ugly corner of specialised appliances”.
- Linear scaling delivers unlimited performance
- No bottlenecks – scale from small to 1000s of servers
- No more NFS – it’s part of the problem
- True software storage runs anywhere – bare metal, containers, cloud
- Almost any x86t server – no appliances
- Anyone can do storage, it’s just another Linux application
- All in user space, installs in minutes
The first part of the announcement is that there’s a free edition (previously there was a 45 day trial on offer). It’s limited in terms of capacity, support, and file system clients, but could be useful in labs and smaller environments.
[image courtesy of Quobyte]
The 3.0 release is also a big part of Quobyte’s news, with the new version delivering a bunch of new features, most of which are outlined below.
- Holistic data protection
- End to end AES encryption (in transit / at rest / untrusted storage nodes)
- Selective TLS support
- Access keys for the file system
- X.509 certificates
- Event stream (metadata, file access)
Powerful Policy Engine
- For: Tenant, volume, file, client
- Control: Layout, tiering, QoS, recoding, caching
- Dynamic: Runtime re-configurable
- Auto file layout: replication + EC and Flash + HDD
- Auto selection of replication factor, EC schema
Quobyte is looking to deliver a “cloud-like experience” with its self-service capabilities.
Login for users
- Manage access keys
- Check resource consumption
Authenticate using access keys
- File system driver
- K8s / CSI
- User-space drivers: HDFS, TF, MPI-IO
- Bi-directional sync (evental consistency)
- Policy-based data tiering between clusters
TLS between clusters
More Native Drivers
Benefit of kernel bypass
- Lower latency
- Less memory bandwidth
Thoughts and Further Reading
One of the challenges with software-defined storage is invariably the constraint that poor hardware choices can put on performance. Kolbeck acknowledged that Quobyte is “as fast as your hardware”. I asked him whether Quobyte provided guidance on hardware choices that worked well with the platform. There is a bunch of recommended (and tested) hardware listed on this page. He did mention that whichever way you decided to go, it was recommended to stick with either Mellanox or Broadcom NICs due to issues observed with other vendors’ Linux drivers. There’re also recommendations on the site for public cloud instance sizing covering AWS, GCP, and Oracle.
Quobyte is being deployed to support scale-out workloads in the enterprise across a number of sectors including financial services, life sciences, media and entertainment, and manufacturing in Europe and Asia. Kolbeck noted that one of the interesting things about the advent of smart everything is that “car manufacturers are suddenly in the machine learning field” and looking for new ways to support their businesses.
There are a lot of reasons to like software-defined storage offerings. You can generally run them on anything, and performance enhancements can frequently be had via code upgrades. That’s not to say that you don’t get that with the big box slingers, but the flexibility of hardware choice has tremendous appeal, particularly in the enterprise market where it can feel like the margin on commodity hardware can be exorbitant. Quobyte hasn’t been around forever, but the folks over there seem to have a pretty solid heritage in software-defined and scale-out storage solutions – a good sign if you’re in the market for a software-defined, scale-out storage solution. Some folks are going to rue the lack of NFS support, but I’m sure Kolbeck and the team would be happy to sit down and discuss with them why that’s no great loss. There’s some pretty cool stuff in this release, and the free edition is definitely worth taking for a spin. I’m looking forward to hearing more from Quobyte over the next little while.