ScaleIO Is Not Your Father’s SDS

Disclaimer: I recently attended Storage Field Day 13.  My flights, accommodation and other expenses were paid for by Tech Field Day and Pure Storage. There is no requirement for me to blog about any of the content presented and I am not compensated in any way for my time at the event.  Some materials presented were discussed under NDA and don’t form part of my blog posts, but could influence future discussions.

 

I’ve written about ScaleIO before (here and here), but thought it might be useful to deliver a basic overview of what ScaleIO actually is and what it can do. You can see Dell EMC’s Storage Field Day presentation video here and you can grab a copy of my rough notes from here.

 

ScaleIO Overview

What is it?

In a nutshell, it’s a software-defined storage product that leverages captive server storage at scale.

 

Benefits

According to Dell EMC, the useful life of ScaleIO is perpetual.

  • Deploy once
  • Grow incrementally
  • No data migration
  • Rolling upgrades
  • Perpetual software licenses

 

ScaleIO Vision and Architecture

Core, Fundamental Features of ScaleIO

Configuration Flexibility

  • Hyperconverged and/or 2-layers

Highly scalable

  • 100s / 1000s of nodes

High performance / low footprint

  • Performance scales linearly
  • High I/O parallelism
  • Gets the maximum from flash media
  • Various caching options (RAM, flash)

Platform agnostic

  • Bare-metal: Linux / Windows
  • Virtual: ESX, XEN, KVM, Hyper-V

Any network

  • Slow, fast, shared, dedicated, IPv6

Flash and Magnetic

  • SSD, NVMe, PCI or HDD
  • Manual and automatic multi-tiering

Elastic / flexible / multi-tenancy

  • Add, move, remove nodes or disks “on the fly”
  • Auto-balance

Various partitioning schemes:

  • Protection-domains
  • Storage pools
  • Fault sets
  • Seamlessly move assets from one partition to another
  • QoS – bandwidth/IOPS limiter

Resilient

  • Distributed mirroring
  • Fast auto many-to-many rebuild
  • Extensive failure handling / HA
  • Background disk scanner

Secure

  • AD/LDAP, RBAC integration
  • Secure cluster formation and component authentication
  • Secure connectivity with components, secure external client communication
  • [email protected] (SW, followed by SED*)

Ease of management & operation

  • GUI, CLI, REST, OpenStack Cinder, vSphere plugin and more
  • Instant maintenance mode
  • NDU

Competent Snapshots

  • Writeable, no hierarchy limits
  • Large consistency groups
  • Automatic policies*

Thin-provisioning

Space-efficient layout*

  • Fine-grain snapshots and thin-provisioning*
  • Compression*

*Soon

 

Two-ways

You can use ScaleIO in a hyperconverged configuration and a “two-layer” configuration. With hyperconverged, you can run:

  • Application and storage in the same node, where  ScaleIO is yet another application running alongside other applications
  • Asymmetric nodes, where nodes may have a different # of spindles, etc

You can also run ScaleIO in a two-layer configuration

  • app-only nodes can access ScaleIO volumes
  • app+storage – hyperconverged nodes

 

Components

ScaleIO Components

  • ScaleIO Data Client (SDC) exposes shared block volumes to the application (block device driver)
  • ScaleIO Data Server (SDS) owns local storage that contributes to the ScaleIO storage pool (daemon/service)

SDS and SDC in the same host

  • Can live together
  • SDC serves the I/O requests of the resident host applications
  • SDS serves the I/O requests of various SDCs

 

Volume Layout, Redundancy and Elasticity

A volume appears as a single object to the application.

Volume Layout (No Redundancy)

  • Chunks (1MB) are spread across the cluster in a balanced manner
  • No hot spots, no I/O splitting

2-Copy Mirror Scheme

Free and Spare Capacity

  • Free and reserved space scattered across the cluster

 

Fast, balanced and smart rebuild

Forwards Rebuild

  • Once disk/node fails – the rebuild load is balanced across all the cluster partition disks/nodes -> faster and smoother rebuild

Backwards Rebuild

  • Smart and selective transition to “backwards” rebuild (re-silvering), once a failed node is back alive
  • Short outage = small penalty

 

Elasticity, Auto-rebalance

Add: one may add nodes or disks dynamically -> the system automatically rebalances the storage

  • Old volumes can use the wider striping
  • No extra exposure
  • Most minimal data transferred

Remove: One may remove nodes / disks dynamically -> the system automatically rebalances the storage

  • Minimal data transferred in a many to many fashion

Combination: The same rebalance plan could handle additions and removals simultaneously

 

Conclusion and Further Reading

I’ve spoken to a range of people in the industry, from customers to Dell EMC folks to competitive vendors, and one thing that gets raised constantly is that Dell EMC are offering both ScaleIO and VMware vSAN. If you’ve been following along at home, you’ll know that this isn’t the first time Dell EMC have offered up products that could be seen as competing for the same market share. But I think they’re doing different things and are aimed at different use cases. My second favourite Canadian Chad Sakac explains this better than I would here.

Put this software on the right hardware (the key to any successful software defined storage product) and you’ve got something that can deliver very good block storage performance across a range of use cases. If you want to know more, Dell EMC have a pretty handy architecture overview document you can get here, and you can download a best practice white paper from here. You can also access a brief introduction to ScaleIO here (registration required). But the best bit is you can download ScaleIO for free from here.

ScaleIO.Next promises to deliver a range of new features, including space efficient storage and NVMe support. I’m curious to see the market uptake has been given the accessibility of the software. I’d also like to see what the uptake has been given the availability of ScaleIO-ready nodes based on Dell PowerEdge hardware. In any case, if you’ve got some spare tin, I recommend taking ScaleIO for a spin.

Dell EMC (Pre-)Announces ScaleIO 3.0

Disclaimer: I recently attended Dell EMC World 2017.  My flights, accommodation and conference pass were paid for by Dell EMC via the Dell EMC Elect program. There is no requirement for me to blog about any of the content presented and I am not compensated in any way for my time at the event.  Some materials presented were discussed under NDA and don’t form part of my blog posts, but could influence future discussions.

ScaleIO.Next

Dell EMC today “pre-announced” ScaleIO.Next (3.0). It will be available by end of year, and it looks like there’s going to be a tonne of new features included. I’ll cover off the main ones below, but obviously things might change too. I call this one “death by dot point”.

 

Compression and Space Efficient Layout

Space Efficient Layout

Created on a storage pool level

  • SSDs can use new layout optimized for SSD, or they can still leverage existing layout if they care about performance more than efficiency
  • Takes full advantage of NVDIMM technology
  • Designed to work well in hyperconverged deployments – Minimised RAM and CPU consumption

Great efficiency and high performance

  • Snapshot and Thin volumes now work on 4K granularity
  • Everything is thin provisioned with no thin performance penalty (no need to worry about zero padding)
  • Snapshots will be highly efficient (space and perf) – great for test/dev
  • Excellent write response times, commits being done in NVDIMMs

Compatible with all existing ScaleIO feature/functionality – still the same scalability and resiliency

Requirement:

  • SSD (including NVMe)
  • NVDIMM

 

Compression

  • Designed from the ground up for SSD and leveraging NVDIMMs
    • Expect best in class performance and space saving
    • Can be turned on / off at storage pool or even volume level
    • Writes are staged with NVDIMM, great latency
  • Variable block size compression based on LZ4.
  • Great for hyperconverged deployments – designed to minimise RAM and CPU requirement.
  • Easy button: Dynamic compression engine – customers no longer have to worry about balancing compression overhead vs. actual compression savings.
  • ScaleIO will automatically not compress data, even if compression is turned on, if the data is not very compressible.

Requirement: Space efficient layout

 

Full Support for Dell EMC PowerEdge 14G and NVMe Drives

ScaleIO 3.0 is built to leverage PowerEdge 14G with NVDIMM and NVMe Drives:

  • PowerEdge 14G based ScaleIO Ready Nodes
  • End-to-end support of PowerEdge 14G with ScaleIO AMS (Automated Management Service)
  • Single support for both 14G PowerEdge and ScaleIO software

 

Enterprise Grade Virtual Volumes Support

VMware Virtual Volumes enable customers to tie VMs more directly to storage volumes (requires ESX 6.0 or newer).

Today, many VMs run on the same volume. This provides:

  • Simplified management
  • Less resource contention
  • Ability to leverage storage side features like snapshot

Exposes speed of storage within a Storage Pool – It ties the pool to a storage profile (Gold, Silver, or Bronze). When VM is started, the requirements are evaluated and a volume is created in the right class of storage.

 

Snapshot Enhancements

  • Snapshot count increased by at least 4x
    • From 31 to 127 for current “medium granular” layout
    • “Fine granular layout” (aka space efficiency layout) snapshot count is increased 8x – 255 snapshots
  • Volumes can be reverted to any snapshot within the vTree
    • Users can revert a volume, on the fly, to different, older or new versions of the volume
  • Volumes can be refreshed in place
    • No need for host to remount a different volume, data can be refreshed under the covers
  • Snapshot can be deleted anywhere in the vTree without affecting others snapshot in vTree
  • Automated snapshot management
    • Customers can set snapshot creation and retention policy and ScaleIO will automatically manage the snapshots without use intervention
    • Snapshot management works w/ CG as well – i.e define policy for a set of snapshots
  • No impact snapshot overhead for “fine granular layout”.

 

System Analysis Tool

Tool to identify potential issues with your ScaleIO system – performance and stability

  • Used to analyze the ScaleIO system immediately after deployment, before provisioning volumes, and before using the system in production. You can also use it to check the health of a system that is already operational.

First added in 2.0.1, supporting RHEL storage-only (2-layer) nodes. Expanded to include support for VMware – includes both hyperconverged and 2-layer VMware deployments.

Enhanced to better handle networks like:

  • IPv6
  • LACP
  • NIC Teaming

 

Hardware Awareness Enhancement

ScaleIO.Next also includes improved “Hardware Awareness”

  • Extends this feature added in 2.0.1 to display extended data in the GUI
  • Hardware Awareness greatly simplifies drive replacement in the DC by blinking LEDs
  • Better troubleshooting by providing deeper insight into the RAID controller and the devices attached
    • SMART data, S/N, temperature, etc.
  • Support for environments with:
    • Linux SDS servers;
    • LSI3108-based cards;
    • Dell PERC H730 & HBA330;
    • HPE Smart Array P440ar &P410i; and
    • Requires hardware vendor specific management tools to be installed.

More cards are compatible, but not yet verified

 

Conclusion

I’ve been intrigued by the potential of ScaleIO for some time. It’s by all accounts a great performing SDS solution offering some seriously cool features. If you’re interested in trying it for yourself – go here to download it.  Chad also did a great write-up on ScaleIO when the ScaleIO node was announced in 2015. Whilst it’s a little dated now, I think it’s nonetheless an invaluable insight into ScaleIO.

SDS means a lot of things to a lot of people. Abstract, pool, automate. Doing the same thing for storage that virtualisation did for compute. And everyone fancies themselves as a web-scale outfit. Especially the ones who aren’t. The presence of ScaleIO and VMware vSAN in the same portfolio seems to confuse people at times. I’m not going to write about the differences between the two, but they’re arguably pitching at different markets and ultimately do different things. One of the cool things about the Dell acquisition of EMC has been that the products you know and like / use haven’t automatically been thrown in the bin. The downside is that newcomers to the portfolio can find it all a bit overwhelming. All I can say is that if you’re SDS-curious it’s worth having a conversation with Dell EMC as they’ll invariably have a solution that will align with your requirements.

Dell EMC Announces ScaleIO Ready Nodes

Fine. It was announced a month ago. But I thought it would be nonetheless useful to cover off the Dell EMC announcement regarding ScaleIO Ready Nodes. The landing page for ScaleIO Ready Nodes is here and you can check out Dell EMC’s press release here. The box shot isn’t super exciting, but it’s a Dell 13G server, isn’t it?

dell-pic

[image courtesy of Dell EMC]

Dell EMC tell me that the ScaleIO Ready Node “combin[es] powerful hardware with agile, scalable and performant software”. In a nutshell, you get:

  • Dell 13G PowerEdge Servers – if you’re already a Dell customer, this is great.
  • Pre-validated, tested and configured – enterprise folk crave this stuff.
  • Locked down parts list, firmware, bios – everyone loves the “simplicity” of appliances.
  • Simplified planning and building – you don’t need to think too much about the hardware.
  • Single vendor support of HW & SW – sleep better at night.
  • Market based pricing – my experience is Dell are traditionally competitive with pricing.

 

Specifications

You can grab the datasheet from here. There are high density (1RU) nodes and high capacity (2RU) nodes available in all flash and hybrid configurations.

Different ScaleIO Ready Node types can be part of the same system and this is a fully supported deployment model, as long as networking and storage pools are not mixed across node types.

The Hybrid configurations offer SSD/HDD and HDD only options. These can also be configured for hyper-converged or storage only deployments.

All models include dual 750w Power Supplies. RAID controllers used are the Dell PERC H730 for hybrid or disk only nodes, and the PERC H330 for All Flash nodes.

Hybrid nodes leverage SanDisk DAS Cache software for caching both reads and writes. Using standard SSDs as cache devices significantly reduces latency for most workloads, without the need for PCI Flash cards. Setup is an easy 3 step process, just install the software, identify the cache devices (SSDs), and assign the HDDs to the cache devices. And you’re good to go.

 

Software-defined as hardware appliance?

Dell EMC say the key advantages of the ScaleIO Ready Node are as follows:

  • Pre-architected to utilize all drives for data
  • Pre-configured with 64GB SLC Serial ATA Disk on Module (SATADOM) flash memory module embedded
  • Pre-validated configurations with complete combination of bios, driver and firmware tested and locked down
  • Fully supported with pre-selected and fully tested server parts and drives for new units and FRU replacements

This got me thinking (just a little bit – don’t be scared) about the whole software-defined X marketplace and the different approaches I’ve been seeing from vendors regarding the interoperability of their software offerings and their hardware offerings. Traditionally, I’ve thought of EMC as the storage company (with a heavy hardware tilt despite claims of being a software company) and Dell as a server company (although I’ve always had pretty good experiences with Dell storage, and started working with EMC via the original Dell | EMC partnership). It was only a matter of time before we started seeing EMC SDS getting coupled with Dell “appliances”, and there’s a lot to be said for the strategy. I’ve never had a big problem with Dell hardware (any more so than any other vendor) and so, potentially, the opportunity to hitch my wagon to an SDS offering via validated hardware and a “one throat to choke” offering has a lot of appeal. The nice thing about this, at least for the moment, is that you don’t need to run ScaleIO on Dell-flavoured hardware if you don’t want to.

I don’t envisage Dell forcing customers to adopt their hardware in order to leverage these types of offerings. There’s still a massive market for ScaleIO on non-Dell hardware, and the success of the product leads me to think that Dell is really just doing this for those customers who feel better using products that come from one vendor.

 

Further Reading and Final Thoughts

Whether you’re into Dell hardware or not, the concept behind ScaleIO Ready Nodes seems reasonable, and certainly has appeal for conservative enterprise types who want to be cool and use SDS but still want someone else to take care of all the boring support and interoperability stuff.

And ScaleIO in and of itself is a pretty cool product and worth checking out. You can also try ScaleIO for free, (along with IsilonSD Edge and ECS). Details can be found here. For a slightly more cynical take on the ScaleIO Ready node announcement, check out Simon’s article at El Reg here.

I’ve no doubt this isn’t the last time you’ll hear about “synergies” between the two companies as they start to merge (and with Dell EMC World just around the corner). I’m interested to see which ones work and which ones don’t, but it’s going to be a bit of time before we know which one is which.