Elastic Cloud Storage (ECS to its friends) has been around a little while. Today Dell EMC announced the release of version 3.0. I thought I’d cover off some of the reasons why ECS might be something you’d be interested in. I’ll then go through the new features with ECS 3.0. If you make it that far you’ll be treated to some light opinionalysis to finish off.
Dell EMC provided me with a list of reasons why you might want to consider ECS.
Highly Efficient Data Protection
One of the problems we have is protecting unstructured data at scale. To this end, ECS uses a hybrid protection scheme comprised of triple mirroring, erasure coding and XOR algorithms. The key benefits of this approach are:
- Lower storage overhead option for cold data scenarios
- Enhanced data durability without the overhead of storing multiple copies
Efficient Large and Small File Storage
- Small files stored in cache and written to a single disk through box-carting
- Large files over 128MB in size are erasure coded immediately vs triple mirroring and erasure coded later
- Provides up to 20% higher throughput for larger files
Fully Geo-distributed High Availability & Protection
- A geographically distributed environment that acts as single logical resource
- Active/Active platform with access to content through a single global namespace
- Provides geo-caching to improve operational performance and reduces latency
- Read/write access from any location globally
Comprehensive Data Access
Simultaneous access to underlying data through multiple interfaces
- Object, File, HDFS
- Support for S3, Swift, Atmos, Centera CAS, and NFS v3
- HDFS compatible with Cloudera, Pivotal, Hortonworks, etc.
What does this mean?
- Native Upgrade path for Centera/Atmos
- Enables S3 like offering in-house
- Eliminates storage gateways
- Breaks down storage silos
Native Multi-tenant Architecture
- Shared storage resources amongst multiple applications and tenants
- System securely and automatically separates Namespaces, object buckets and users
- Integration with LDAP and AD environments
- Ensures the integrity of customers’ stored data
Built-in Metadata search
Integrated Metadata storage – store metadata using the same constructs as objects eliminating the need for a separate database and infrastructure to run it.
Metadata search via SQL construct
- Enables applications and users to query metadata using SQL constructs. Supports several attribute and sort functions.
- Global metadata search
- Enables applications and users to search across the global namespace.
So what’s really new in 3.0?
So this ECS stuff is great, but what’s exciting about 3.0?
Advanced Retention Management
- Event Based Retention – Enables application to specify retention period that will start when a specified event occurs
- Litigation Hold – Enables application to temporarily prevent deletion of an object that is subject to an investigation or legal action
- Min/Max Governor – Enables system administrator to specify a min and max value for the default retention period
This unblocks Centera customers using ARM from migration to ECS. I’m actually really excited about this, mainly because I was a big Centera fanboy and have found it difficult to put forward other EMC solutions to replace it for customers heavily leveraging ARM.
SNMP Traps Support
- ECS 3.0 will support for SNMP Traps for ECS critical events
- SNMP Traps is an optional feature, based on whether system admin configures SNMP information via UI/API
- When configured, ECS sends a SNMP Trap to the configured server for any event that causes an alert on the management API
- ECS supports the ability to configure up to 10 SNMP Trap Destination targets
- SNMPv2 and SNMPv3 (USM mode) support
- SNMP Query Service support (CPU & Memory)
Remote Syslog Support
- Shipping ECS Monitoring & Diagnostics logs to a remote syslog server
- Ability to forward all ECS Audit Logs and ECS Alerts to a centralized Syslog server
- Forward OS syslog messages
- Support for UDP and TCP based communication with syslog servers
- Support for multiple redundant syslog servers , all active
- Distributed service, resilient to node failures
- Only System Admin can perform syslog management operations
- Specify a severity threshold of logs to be forwarded
- Ability to Add, Edit and Delete Syslog server configuration from the portal and REST API
- Logs can be seen on ECS nodes in /var/log/<node IP>/syslog.log
ECS will support a ability to do the following via the ECS RESTful management API
- Lockdown an entire cluster
- Lockdown a specific node
- Unlock a locked node
A new management user role, the lock admin user, for locking is defined that will have the privilege of locking/unlocking the cluster.
In ECS 3.0 this will be a pre-provisioned local user ‘emcsecurity’.
The lock admin user i.e. ‘emcsecurity’ will have the ability to
- Modify their password (forced during first login)
- Lock the cluster
- Lock a node
- Unlock a node
System admin/monitor has the privilege to view the lock status of a node but NOT modify it
Thoughts and Further Reading
I’ve been talking to a lot of scale-out object storage folks lately. Given the amount of EMC stuff I’ve covered here previously, it’s a little surprising that this is the first time I’ve posted about ECS. That doesn’t mean it’s the first time I’ve looked, and I’ve had customers looking at it fairly seriously. In my opinion, the advanced retention management in 3.0 is really going to put a few customers over the line and finally give them the confidence to throw their Centera grids in the river (figuratively speaking).
The cool thing about ECS, like a lot of these types of solutions, is that you can consume it on your terms, via
- Software defined;
- Dedicated cloud; and
- Multi-tenant Storage Cloud.