NVMesh 2 – A Compelling Sequel From Excelero

The Announcement

Excelero recently announced NVMesh 2 – the next iteration of their NVMesh product. NVMesh is a software-only solution designed to pool NVMe-based PCIe SSDs.

[image courtesy of Excelero]

Key Features

There are three key features that have been added to NVMesh.

  • MeshConnect – adding support for traditional network technologies TCP/IP and Fibre Channel, giving NVMesh the widest selection of supported protocols and fabrics of software-defined storage platforms along with already supported InfiniBand, RoCE v2, RDMA and NVMe-oF.
  • MeshProtect – offering flexible protection levels for differing application needs, including mirrored and parity-based redundancy.
  • MeshInspect – with performance analytics for pinpointing anomalies quickly and at scale.

Performance

Excelero have said that NVMesh delivers “shared NVMe at local performance and 90+% storage efficiency that helps further drive down the cost per GB”.

Protection

There’s also a range of protection options available now. Excelero tell me that you can start at level 0 (no protection, lowest latency) all the way to “MeshProtect 10+2 (distributed dual parity)”. This allows customers to “choose their preferred level of performance and protection. [While] Distributing data redundancy services eliminates the storage controller bottleneck.”

Visibility

One of my favourite things about NVMesh 2 is the MeshInspect feature, with a “built-in statistical collection and display, stored in a scalable NoSQL database”.

[image courtesy of Excelero]

 

Thoughts And Further Reading

Excelero emerged form stealth mode at Storage Field Day 12. I was impressed with their offering back then, and they continue to add features while focussing on delivering top notch performance via a software-only solution. It feels like there’s a lot of attention on NVMe-based storage solutions, and with good reason. These things can go really, really fast. There are a bunch of startups with an NVMe story, and the bigger players are all delivering variations on these solutions as well.

Excelero seem well placed to capitalise on this market interest, and their decision to focus on a software-only play seems wise, particularly given that some of the standards, such as NVMe over TCP, haven’t been fully ratified yet. This approach will also appeal to the aspirational hyperscalers, because they can build their own storage solution, source their own devices, and still benefit from a fast software stack that can deliver performance in spades. Excelero also supports a wide range of transports now, with the addition of NVMe over FC and TCP support.

NVMesh 2 looks to be smoothing some of the rougher edges that were present with version 1, and I’m pumped to see the focus on enhanced visibility via MeshInspect. In my opinion these kinds of tools are critical to the uptake of solutions such as NVMesh in both the enterprise and cloud markets. The broadening of the connectivity story, as well as the enhanced resiliency options, make this something worth investigating. If you’d like to read more, you can access a white paper here (registration required).

Excelero are doing what? For how much?

Disclaimer: I recently attended Storage Field Day 12.  My flights, accommodation and other expenses were paid for by Tech Field Day. There is no requirement for me to blog about any of the content presented and I am not compensated in any way for my time at the event.  Some materials presented were discussed under NDA and don’t form part of my blog posts, but could influence future discussions.

Here are some notes from Excelero’s presentation at Storage Field Day 12. You can view the video here and download my rough notes here.

 

So what do they do?

Something called NVMesh Server SAN. And it’s pretty wild from what I saw during the demo. But first, what is NVMesh Server SAN? It’s basically magical software-defined block storage designed for NVMe.

[image courtesy of Excelero]

 

Benefits

So what’s so special about this? Well, it provides:

  • A virtual SAN solution that is optimised for NVMe, and 100% software-based (using the proper hardware, of course);
  • “Unified NVMe” by pooling NVMe across the network at what appear to the host as local speeds and latencies;
  • Really, really low CPU usage on the storage target;
  • Flexibility, and can be deployed as a virtual, distributed non-volatile array and supporting both converged and disaggregated architectures; and
  • Efficiency, with performance scaling linearly at close to 100% efficiency.

According to Excelero, some of the key NVMesh benefits include:

  • maximum utilisation of NVMe flash devices by creating a single pool of high performance block storage (they really flog these devices);
  • no data localisation for scale-out applications;
  • predictable application performance – no noisy neighbours; and
  • making storage as efficient as the optimised hardware platform (such as Open19).

What does this mean for your enterprise applications? You get access to:

  • Higher performance for random IO intensive enterprise applications;
  • A flexible architecture to support multiple workloads;
  • lower operating costs through deployment efficiency and easy serviceability; and
  • all your data is “local” with no application changes. These is a mighty fine trick.

Excelero’s solution also helps with high-performance computing (HPC) environments, offering:

  • massive performance: high IOPS and bandwidth, low latency;
  • unlimited scalability, supports analytics for massive data sets; and
  • lowest cost/IOP.

 

Excelero Software Components

[image courtesy of Excelero]

 

The Centralized Management component:

  • runs as a Node.js application on top of MongoDB;
  • pools drives, provisions volumes and monitors stuff; and
  • transforms drives from raw storage into a pool.

It’s built as a scale-out service to support huge deployments and offers some standard integration, including RESTful API access for seamless provisioning.  There’s also a client block driver with the kernel module presenting logical volumes via the block driver API.

From a performance perspective, it interacts directly with drives via RDDA or NVMf offering single hop access to the data, minimising latency overhead,  and maximising throughput and IOPS. As a result of this you get consistent access to share volumes spread across remote drives anywhere in the DC. The solution offers “RAIN” data protection (cross-node / rack) for standard servers and from a scalability perspective there’s point to point communication with management and targets, simple discovery, and no broadcasts.

 

Topology Manager:

  • Performs cluster management to ensure high availability;
  • Manages volume life cycle and failure recovery operations; and
  • Uses Raft protocol to ensure data consistency – avoiding “split brain” scenarios

 

Key Takeaways

  • Excelero is a “Software-defined block storage solution” – using standard servers with state of the art flash components and leveraging an intuitive management portal;
  • Excelero offers virtual SAN for NVMe – pooling NVMe over the network at local speeds to maximise utilisation. By making all data local you move the compute, not the data;
  • Scale-Out Server SAN – scales performance & capacity linearly, across DCs without limits – enabling just-in-time storage orchestration; and
  • Converged and disaggregated architectures – no noisy neighbours through full logical disaggregation of storage and compute – grow storage or compute independently

 

Feelings and Further Reading

Excelero came out of stealth mode during Storage Field Day 12. Ray was mighty impressed with what he saw, as was I. I was also mightily impressed with the relatively inexpensive nature of the hardware that they used to demonstrate the solution. Every SDS solution has a reasonably strict hardware compatibility list. In this case, it makes a lot of sense, as Excelero’s patented RDDA technology contributes a lot to the performance and success of the solution. It’s also NVMe over Fabrics ready too, so as this gains traction the requirement for RDDA will potentially fade away.

Super-fast storage solutions based on NVMe are a lot like big data and bad reality TV shows. They’re front and centre in a lot of conversations around the water cooler but a lot of people aren’t exactly sure what they are or what they should make of them. While Dell recently put the bullet through DSSD,  it doesn’t mean that the technology or the requirement for this kind of solution doesn’t exist. What it does demonstrate is that these kind of solutions can be had in the data centre for a reasonably inexpensive investment in hardware coupled with some really smart software. Version 1.1 is still raw in places, and it will be a while before we see widespread adoption of these types of solutions in the enterprise data centre (people like to wrap data services around these kinds of solutions). That said, if you have the need for speed right now, it might be a good idea to reach out to the Excelero folks and have a conversation.

You’ll notice my title was a bit misleading, as I don’t have pricing information in this post. Ray did some rough calculations in his article, and you should talk to Excelero to find out more. As an aside, it’s also worth checking out the Storage Field Day presentation for Yuval Bachar‘s presentation on Open19 as well – it was riveting stuff.

 

Storage Field Day – I’ll Be At Storage Field Day 12

In what can only be considered excellent news, I’ll be heading to the US in early March for another Storage Field Day event. If you haven’t heard of the very excellent Tech Field Day events, you should check them out. I’m looking forward to time travel and spending time with some really smart people for a few days. It’s also worth checking back on the Storage Field Day 12 website during the event (March 8 – 10) as there’ll be video streaming and updated links to additional content. You can also see the list of delegates and event-related articles that have been published.

I think it’s a great line-up of presenting companies this time around. There are a few I’m very familiar with and some I’ve not seen in action before.

 

It’s not quite a total greybeard convention this time around, but I think that’s only because of Jon‘s relentless focus on personal grooming. I won’t do the delegate rundown, but having met a number of these people I can assure the videos will be worth watching.

Here’s the rough schedule (all times are ‘Merican Pacific and may change).

Wednesday, March 8 10:00 – 12:00 StarWind Presents at Storage Field Day 12
Wednesday, March 8 13:00 – 15:00 Elastifile Presents at Storage Field Day 12
Wednesday, March 8 16:00 – 18:00 Excelero Presents at Storage Field Day 12
Thursday, March 9 08:00 – 10:00 Nimble Storage Presents at Storage Field Day 12
Thursday, March 9 11:00 – 13:00 NetApp Presents at Storage Field Day 12
Thursday, March 9 14:00 – 16:00 Datera Presents at Storage Field Day 12
Friday, March 10 09:00 – 10:00 SNIA Presents at Storage Field Day 12
Friday, March 10 10:30 – 12:30 Intel Presents at Storage Field Day 12

I’d like to publicly thank in advance the nice folks from Tech Field Day who’ve seen fit to have me back, as well as my employer for giving me time to attend these events. Also big thanks to the companies presenting. It’s going to be a lot of fun. Seriously.