Disclaimer: I recently attended Storage Field Day 10. My flights, accommodation and other expenses were paid for by Tech Field Day. There is no requirement for me to blog about any of the content presented and I am not compensated in any way for my time at the event. Some materials presented were discussed under NDA and don’t form part of my blog posts, but could influence future discussions.
Smart and at Scale
Cloudian took us through some of their driving design principles, and I thought it was worth covering these off again. You’ll notice the word “scale” gets used a lot, and this has been a particularly important capability for Cloudian. They did a blog post on it too.
One of the key features of the HyperStore solution is that it needed to support what Cloudian term “Smart Operations at Scale”. This requires the tech to:
- Be simple and intuitive;
- Be fully automated from an operations perspective (e.g. add/remove drives/nodes, upgrades);
- Provide visual storage analytics to automatically see hot spots; and
- Offer self service consumption (via a policy based approach).
Cloudian have also worked hard to ensure they can provide “Extreme Durability at Scale”, with the HyperStore solution offering the ability to:
- Be always repaired, always verified;
- Offer automated failure avoidance (through the use of Dynamic Object Routing); and
- Be “enterprise” grade.
One of the keys to being able deliver a scaleable solution has been the ability to provide the end user with “Smart support at Scale”, primarily through the use of:
- Proactive (not reactive) support;
- Continuous monitoring; and
- Global analytics.
The analytics piece is a big part of the Cloudian puzzle, and something they’ve been working hard on recently. With their visual analytics you can analyse your data across globe and plan for future based on your demand. Cloudian not only performs analytics at scale, but also designed to facilitate operations at scale, with:
- One screen for hundreds of nodes (in a kind of “beehive” layout);
- Instant view of a node’s health;
- The ability to add nodes with one click; and
- The ability to dynamically rebalance the cluster.
When it comes to software defined storage platforms, the simple things matter, particularly as it relates to your interactions with the hardware platform. To that end, with HyperStore you’ve got the ability to do some basic stuff, like:
- Identifying node types;
- Blinking suspect servers; and
- Blinking suspect drives.
When you’re running a metric s**t-tonne of these devices in a very big data centre, this kind of capability is really important, especially when it comes to maintenance. As is the ability to perform rolling upgrades of the platform with no downtime and in an automated fashion. When it comes to rebuilds, Cloudian provides insight into both data rebuild information and cluster rebalance information – both handy things to know when something’s gone sideways.
The Cloudian platform also does “Smart Disk Balancing”. If there’s a disk imbalance it will change the tokens pointing from “highly used disk to low used disk”. If there’s a disk failure, new data automatically routes to newly assigned resources. Makes sense, and nice to see they’ve thought it through.
Further Reading and Conclusion
Cloudian make quite a big deal of their S3 compatibility. They even give me a sticker that says it’s guaranteed. It looks a lot like this:
Chris Evans also did a series of posts on S3 and Cloudian that you can read here, here and here. He also did a great preview post prior to SFD10 which is also worth a look. He’s a good lad, he is. Particularly when I need to point you, my loyal reader, to well written articles on topics I’m a little sketchy on.
S3 compatibility is a big thing for a lot of people looking at deploying object storage, primarily because AWS are leaps and bounds ahead of the pack in terms of object storage functionality, deployed instances, and general mindshare. Cloudian haven’t just hitched their wagon to S3 compatibility though. In my opinion they’ve improved on the S3 experience through clever design and a solid approach to some fundamental issues that arise when you’re deploying a whole bunch of devices in data centres that don’t often have staff members present.