Disclaimer: I recently attended Dell EMC World 2017. My flights, accommodation and conference pass were paid for by Dell EMC via the Dell EMC Elect program. There is no requirement for me to blog about any of the content presented and I am not compensated in any way for my time at the event. Some materials presented were discussed under NDA and don’t form part of my blog posts, but could influence future discussions.
Dell EMC today “pre-announced” ScaleIO.Next (3.0). It will be available by end of year, and it looks like there’s going to be a tonne of new features included. I’ll cover off the main ones below, but obviously things might change too. I call this one “death by dot point”.
Compression and Space Efficient Layout
Space Efficient Layout
Created on a storage pool level
- SSDs can use new layout optimized for SSD, or they can still leverage existing layout if they care about performance more than efficiency
- Takes full advantage of NVDIMM technology
- Designed to work well in hyperconverged deployments – Minimised RAM and CPU consumption
Great efficiency and high performance
- Snapshot and Thin volumes now work on 4K granularity
- Everything is thin provisioned with no thin performance penalty (no need to worry about zero padding)
- Snapshots will be highly efficient (space and perf) – great for test/dev
- Excellent write response times, commits being done in NVDIMMs
Compatible with all existing ScaleIO feature/functionality – still the same scalability and resiliency
- SSD (including NVMe)
- Designed from the ground up for SSD and leveraging NVDIMMs
- Expect best in class performance and space saving
- Can be turned on / off at storage pool or even volume level
- Writes are staged with NVDIMM, great latency
- Variable block size compression based on LZ4.
- Great for hyperconverged deployments – designed to minimise RAM and CPU requirement.
- Easy button: Dynamic compression engine – customers no longer have to worry about balancing compression overhead vs. actual compression savings.
- ScaleIO will automatically not compress data, even if compression is turned on, if the data is not very compressible.
Requirement: Space efficient layout
Full Support for Dell EMC PowerEdge 14G and NVMe Drives
ScaleIO 3.0 is built to leverage PowerEdge 14G with NVDIMM and NVMe Drives:
- PowerEdge 14G based ScaleIO Ready Nodes
- End-to-end support of PowerEdge 14G with ScaleIO AMS (Automated Management Service)
- Single support for both 14G PowerEdge and ScaleIO software
Enterprise Grade Virtual Volumes Support
VMware Virtual Volumes enable customers to tie VMs more directly to storage volumes (requires ESX 6.0 or newer).
Today, many VMs run on the same volume. This provides:
- Simplified management
- Less resource contention
- Ability to leverage storage side features like snapshot
Exposes speed of storage within a Storage Pool – It ties the pool to a storage profile (Gold, Silver, or Bronze). When VM is started, the requirements are evaluated and a volume is created in the right class of storage.
- Snapshot count increased by at least 4x
- From 31 to 127 for current “medium granular” layout
- “Fine granular layout” (aka space efficiency layout) snapshot count is increased 8x – 255 snapshots
- Volumes can be reverted to any snapshot within the vTree
- Users can revert a volume, on the fly, to different, older or new versions of the volume
- Volumes can be refreshed in place
- No need for host to remount a different volume, data can be refreshed under the covers
- Snapshot can be deleted anywhere in the vTree without affecting others snapshot in vTree
- Automated snapshot management
- Customers can set snapshot creation and retention policy and ScaleIO will automatically manage the snapshots without use intervention
- Snapshot management works w/ CG as well – i.e define policy for a set of snapshots
- No impact snapshot overhead for “fine granular layout”.
System Analysis Tool
Tool to identify potential issues with your ScaleIO system – performance and stability
- Used to analyze the ScaleIO system immediately after deployment, before provisioning volumes, and before using the system in production. You can also use it to check the health of a system that is already operational.
First added in 2.0.1, supporting RHEL storage-only (2-layer) nodes. Expanded to include support for VMware – includes both hyperconverged and 2-layer VMware deployments.
Enhanced to better handle networks like:
- NIC Teaming
Hardware Awareness Enhancement
ScaleIO.Next also includes improved “Hardware Awareness”
- Extends this feature added in 2.0.1 to display extended data in the GUI
- Hardware Awareness greatly simplifies drive replacement in the DC by blinking LEDs
- Better troubleshooting by providing deeper insight into the RAID controller and the devices attached
- SMART data, S/N, temperature, etc.
- Support for environments with:
- Linux SDS servers;
- LSI3108-based cards;
- Dell PERC H730 & HBA330;
- HPE Smart Array P440ar &P410i; and
- Requires hardware vendor specific management tools to be installed.
More cards are compatible, but not yet verified
I’ve been intrigued by the potential of ScaleIO for some time. It’s by all accounts a great performing SDS solution offering some seriously cool features. If you’re interested in trying it for yourself – go here to download it. Chad also did a great write-up on ScaleIO when the ScaleIO node was announced in 2015. Whilst it’s a little dated now, I think it’s nonetheless an invaluable insight into ScaleIO.
SDS means a lot of things to a lot of people. Abstract, pool, automate. Doing the same thing for storage that virtualisation did for compute. And everyone fancies themselves as a web-scale outfit. Especially the ones who aren’t. The presence of ScaleIO and VMware vSAN in the same portfolio seems to confuse people at times. I’m not going to write about the differences between the two, but they’re arguably pitching at different markets and ultimately do different things. One of the cool things about the Dell acquisition of EMC has been that the products you know and like / use haven’t automatically been thrown in the bin. The downside is that newcomers to the portfolio can find it all a bit overwhelming. All I can say is that if you’re SDS-curious it’s worth having a conversation with Dell EMC as they’ll invariably have a solution that will align with your requirements.
I like ScaleIO – but the resiliency is VERY BAD. Only one HDD can fail at one time. If another one fails at the same time the complete data pool is defect.
This is why we decided to use MSFT Storage Spaces with 3way mirror. Other good solutions are some Linux appliances like Open-E Jovian DSS; Nexenta or FreeNAS. All of them are ZFS based and you will have great resiliency with a good usable capacity.
Do you know if ScaleIO also provides 3way mirror with version 3?
I know about parallel rebuild and that it takes only minutes to be in sync after a HDD fails. But for our workload this is to dangerous.
Thanks for your comment. I tried asking about the resiliency issue and was referred to a best practices guide. You’ve probably already read it (https://www.emc.com/collateral/white-papers/h15148-emc-scaleio-deployment-guide.pdf) but the guidance around fault sets may be useful.
I agree the ZFS-based stuff is great, although it strikes me that FreeNAS are going through a tough time realising their vision.
I couldn’t tell you about 3-way mirror – but as more information about the release comes to hand (later this year I assume), I’ll be posting about it.
Pingback: ScaleIO Is Not Your Father’s SDS | penguinpunk.net