Scality’s RING has a lot going on

Disclaimer: I recently attended VMworld 2016 – US.  My flights were paid for by myself, VMware provided me with a free pass to the conference and various bits of swag, and Tech Field Day picked up my hotel costs. There is no requirement for me to blog about any of the content presented and I am not compensated in any way for my time at the event.  Some materials presented were discussed under NDA and don’t form part of my blog posts, but could influence future discussions.

TFD-Extra-VMworld-300

 

scality

Here are my notes from Scality’s presentation at Tech Field Day Extra VMworld US 2016 Edition. You can get a rough copy here. You can also view videos of the Scality presentation here.

 

The Ring?

Like the movie? No. The RING. The Scality RING is object-based software-defined storage for the cloud. It runs on standard x86 servers to create a giant pool of storage.

scality_architecture_ring

[image via Scality website]

It can also protect the data and provides 100% reliable, high performance access for any capacity-driven applications. While it can run on any x86 hardware, it was pointed out that “[s]ome servers are better than others”.

Customers are telling Scality that:

  • The “cloudification” of enterprise IT is accelerating
  • Enterprise wants “multiple clouds”
  • Object is the best for large capacity storage, and S3 is the standard API
  • Files are integral part of enterprise IT
  • DevOps influences infrastructure choices

Scality have 116 customers so far, spread across the globe (50% North America, 35% EMEA, 15% APAC). Scality are big on hardware alliances (being a software play, this makes sense), and have agreements in place with HPE, Dell, and Cisco.

 

(The) RING 6.0 – A better sequel than we’d hoped for

Paul Speciale,  VP of Products at Scality,  took us through some of the features of RING 6.0.

tfdx-scality-ring6-0

The focus for Scality with 6.0 has been on

  • “Enterprization” – I’m not sure it’s a real word, but I do like the connotation
  • S3 Connector – Enterprise Deployments
  • Easy deployment model
  • Secure multi-tenancy and data at rest
  • Directory services federation
  • Utilisation reporting and management

 

Easy Deployment Model

  • All services deployed uniformly as Docker containers
  • Full scale-out: Any S3 request can be handled by any S3 Connector (“any-to-any”), standard IP load balancing and failover

Vault Service

  • Implements IAM Multi-tenancy with Accounts, Users, Groups, Roles, Access Key/Secret Key pairs
  • IAM REST compatible managed via AWS CLI
  • Can be federated with Active Directory over ADFS/sAML 2.0

Metadata Service

  • S3 optimised service: fast, available, scale-out
  • Integral in RING layer – leveraged for Bucket & Vault metadata

 

Comprehensive IAM multi-tenancy and encryption

AWS Identity and Access Management (IAM)

  • S3 Connector implements all IAM multi-tenancy concepts: Accounts, Keys, Users, Groups, Roles
  • IAM policies for highly granular access control
  • AWS compatible: Management of IAM entities (Users, Groups) via standard AWS CLI and JSON policy language
  • Secure authentication via AWS Signature v4 and v2 HMAC schemes

Bucket-level Encryption

  • Pre-bucket encryption-at-rest of object data (specified through header on Bucket PUT)
  • Encryption via AES-256bit OpenSSL libraries
  • Integrates with customer-provided Key Management Service (KMS) via KMIP 1.1 API
  • KMS is invoked on PUT and GET operations

tfdx-scality-comprehensiveencryption

 

Federated Access SSO to S3

  • Requires a SAML 2.0 Compatible ldP
  • ldP provides mapping from Enterprise Direcoty Server (AD)
  • Vault enables SSO via SAML assertion

 

S3 Utilization Reporting and Management

Stats and management framework

  • Real-time and historical statistics and metrics collected in scalable repository

Published RESTful APIs for monitoring and management

  • S3 Connector publishes key utilisation metrics (capacity, bandwidth and operations) at four levels of granularity
  • REST APIs for custom tool integrations

Management tools

  • User and Group management via standard AWS commands (CLI) and REST API
  • Integrated tools for graphing, metrics, log visualisation and search: Elastic Search and Kibana, Grafana, Redis.

 

S3 Metadata – the scale-out engine of the connector

Metadata Service

  • Purpose-built for availability, resiliency, scale-out and fast performance for requirements of S3 operations
  • Key/value store replicated on SSDs (one per server)
  • Additional copy maintained as diff backup in RING for DR

The hard part: Distributed Consensus Algorithm

  • Leader with dynamic election and management of consistency (modified Raft protocol)
  • Can be distributed across DCs to enable multi-geo operations
  • By default, strict consistency rules enforced

High-availability and Performance

  • The cluster consists of multiple servers – odd number to provide majority quorum (5, 7 or 9)
  • As long as the majority (quorum) of servers is available, the service and Bucket remain available
  • Restarts failed servers with automated resynchronization

 

S3 Connector Scale-out at all levels

tfdx-scality-s3_connector_scale-out_at_all_levels

S3 as the best On-ramp to Object Storage

tfdx-scality-s3_on-ramp

  • Developers can install and develop S3-based apps locally
  • Enterprises can host a small, local object storage systems in production
  • Enterprise can host a local test/dev environment to learn about object storage

 

Scality Open Source S3 Server

S3 API Compatible with the S3 Connector

  • Single Docker Container for simplified deployment
  • Stores data in local Docker Volume (local storage)
  • Metadata managed in single key/value database
  • S3 compatible Bucket and Object operations, error and response codes

Downloadable on Docker Hub

  • Can be pulled via UI or Docker pull command as per instructions on s3.scality.com
  • Can be hosted on laptops and single servers
  • Seamless transition to scale-out solution on RING

ISV Certified with multiple solutions

  • Backup, archive, sync-n-share, surveillance, migration

 

Summary

So what do you get with Scality?

  • S3 Server & S3 Connector
  • Provides a seamless transition from “free” test/dev single-server trial to full scale-out deployments (note that the trial is not available to robots).
  • Small to large deployments from local storage to full RING
  • Simple to deploy via Docker containers
  • Comprehensive Enterprise Deployment Features
  • Multi-tenancy
  • Active Directory SSO/federation

 

Further Reading and Thoughts

Justin did a comprehensive write-up on Scality here. Sure, I could have saved you a lot of time and sent you there in the first place, but that’s not how I roll. I admit I’m not super familiar with Scality and have yet to get cracking with the RING trial. That said, with version 6.0 they seem to included a lot of features that enterprises are interested in when looking at object storage with cloudy tendencies. There’s decent support for file protocols such as NFS and SMB, just no block. I covered some of the other enterprise features above, and they’ve been around for a little while now. But that’s not what the kids are into these days in any case. If you’re looking into rolling your own object solution, I recommend giving Scality a spin.