VMware Cloud on AWS – Supplemental Storage – A Few Notes …

At VMware Explore 2022 in the US, VMware announced a number of new offerings for VMware Cloud on AWS, including something we’re calling “Supplemental Storage”. There are some great (official) posts that have already been published, so I won’t go through everything here. I thought it would be useful to provide some high-level details and cover some of the caveats that punters should be aware of.

 

The Problem

VMware Cloud on AWS has been around for just over 5 years now, and in that time it’s proven to be a popular platform for a variety of workloads, industry verticals, and organisations of all different sizes. However, one of the challenges that a hyper-converged architecture presents is that resource growth is generally linear (depending on the types of nodes you have available). In the case of VMware Cloud on AWS, we (now) have 3 nodes available for use: the I3, I3en, and I4i. Each of these instances provides a fixed amount of CPU, RAM, and vSAN storage for use within your VMC cluster. So when your storage grows past a certain threshold (80%), you need to add an additional node. This is a longwinded way of saying that, even if you don’t need the additional CPU and RAM, you need to add it anyway. To address this challenge, VMware now offers what’s called “Supplemental Storage” for VMware Cloud on AWS. This is ostensibly external dat stores presented to the VMC hosts over NFS. This comes in two flavours: FSx for NetApp ONTAP and VMware Cloud Flex Storage. I’ll cover this in a little more detail below.

[image courtesy of VMware]

 

Amazon FSx for NetApp ONTAP

The first cab off the rank is Amazon FSx for NetApp ONTAP (or FSxN to its friends). This one is ONTAP-like storage made available to your VMC environment as a native service. It’s fully customer managed, and VMware managed from a networking perspective.

[image courtesy of VMware]

There’s a 99.99% Availability SLA attached to the service. It’s based on NetApp ONTAP, and offers support for:

  • Multi-Tenancy
  • SnapMirror
  • FlexClone
​Note that it currently requires VMware Managed Transit Gateway (vTGW) for Multi-AZ deployment (the only deployment architecture currently supported), and can connect to multiple clusters and SDDCs for scale. You’ll need to be on SDDC version 1.20 (or greater) to leverage this service in your SDDC, and there is currently no support for attachment to stretched clusters. While you can only connect datastores to VMC hosts using NFSv3, there is support for connecting directly to guest via other protocols. More information can be found in the FAQ here. There’s also a simulator you can access here that runs you through the onboarding process.

 

VMware Cloud Flex Storage

The other option for supplemental storage is VMware Cloud Flex Storage (sometimes referred to as VMC-FS). This is a datastore presented to your hosts over NFSv3.

Overview

VMware Cloud Flex Storage is:

  • A natively integrated cloud storage service for VMware Cloud on AWS that is fully managed by VMware;
  • Cost effective multi-cloud Cloud storage solution built on SCFS;
  • Delivered via a two-tier architecture for elasticity and performance (AWS S3 and local NVMe cache); and
  • Provides integrated Data-Management.

In short, VMware has taken a lot of the technology used in VMware Cloud Disaster Recovery (the result of the Datrium acquisition in 2020) and used it to deliver up to 400 TiB of storage per SDDC.

[image courtesy of VMware]
The intent of the solution, at this stage at least, is that it is only offered as a datastore for hosts via NFSv3, rather than other protocols directly to guests. There are some limitations around the supported topologies too, with stretched clusters not currently supported. From a disaster recovery perspective, it’s important to note that VMware Cloud Flex Storage is currently only offered on a single-AZ basis (although the supporting components are spread across multiple Availability Zones), and there is currently no support for VMware Cloud Disaster Recovery co-existence with this solution.

 

Thoughts
I’ve only been at VMware for a short period of time, but I’ve had numerous conversations with existing and potential VMware Cloud on AWS customers looking to solve their storage problems without necessarily putting everything on vSAN. There are plenty of reasons why you wouldn’t want to use vSAN for high capacity storage workloads, and I believe these two initial solutions go some ways to solving that issue. Many of the caveats that are wrapped around these two products at General Availability will be removed over time, and the traditional objections relating to VMware Cloud on AWS being not great at high-capacity, cost-effective storage will also have been removed.
Finally, if you’re an existing NetApp ONTAP customer, and were thinking about what you were going to do with that Petabyte of unstructured data you had lying about when you moved to VMware Cloud on AWS, or wanting to take advantage of the sweat equity you’ve poured into managing your ONTAP environment over the years, I think we’ve got you covered as well.

Random Short Take #75

Welcome to Random Short Take #75. Half the year has passed us by already. Let’s get random.

  • I talk about GiB all the time when sizing up VMware Cloud on AWS for customers, but I should take the time to check in with folks if they know what I’m blithering on about. If you don’t know, this explainer from my friend Vincent is easy to follow along with – A little bit about Gigabyte (GB) and Gibibyte (GiB) in computer storage.
  • MinIO has been in the news a bit recently, but this article from my friend Chin-Fah is much more interesting than all of that drama – Beyond the WORM with MinIO object storage.
  • Jeff Geerling seems to do a lot of projects that I either can’t afford to do, or don’t have the time to do. Either way, thanks Jeff. This latest one – Building a fast all-SSD NAS (on a budget) – looked like fun.
  • You like ransomware? What if I told you you can have it cross-platform? Excited yet? Read Melissa’s article on Multiplatform Ransomware for a more thorough view of what’s going on out there.
  • Speaking of storage and clouds, Chris M. Evans recently published a series of videos over at Architecting IT where he talks to NetApp’s Matt Watt about the company’s hybrid cloud strategy. You can see it here.
  • Speaking of traditional infrastructure companies doing things with hyperscalers, here’s the July 2022 edition of What’s New in VMware Cloud on AWS.
  • In press release news, Aparavi and Backblaze have joined forces. You can read more about that here.
  • I’ve spent a lot of money over the years trying to find the perfect media streaming device for home. I currently favour the Apple TV 4K, but only because my Boxee Box can’t keep up with more modern codecs. This article on the Best Device for Streaming for Any User – 2022 seems to line up well with my experiences to date, although I admit I haven’t tried the NVIDIA device yet. I do miss playing ISOs over the network with the HD Mediabox 100, but those were simpler times I guess.

StorONE Announces Per-Drive Licensing Model

StorONE recently announced details of its Per-Drive Licensing Model. I had the opportunity to talk about the announcement with Gal Naor and George Crump about the news and thought I’d share some brief thoughts here.

 

Scale For Free?

Yes, at least from a licensing perspective. If you’ve bought storage from many of the traditional array vendors over the years, you would have likely paid for capacity-based licensing. Every time you upgraded the capacity of your array, there was usually a charge associated with that upgrade, beyond the hardware uplift costs. The folks at StorONE think it’s probably time that they stopped punishing customers for using higher capacity drives, so they’re shifting everything to a per-drive model.

How it Works

As I mentioned at the start, StorONE Scale-For-Free pricing is on a per-drive basis, so you can use the highest capacity, highest density drives without penalty, rather than metering capacity. The pricing is broken down thusly:

  • Price per HDD $/month
  • Price per SSD $/month
  • Minimum $/month
  • Cloud Use Case – $ per month by VM instance required

The idea is that this ultimately lowers the storage price per TB and brings some level of predictability to storage pricing.

How?

The key to this model is the availability of some key features in the StorONE solution, namely:

  • A rewritten and collapsed I/O stack (meaning do more with a whole lot less)
  • Auto-tiering improvements (leading to more consistent and predictable performance across HDD and SDD)
  • High performance erasure coding (meaning super fast recovery from drive failure)

 

But That’s Not All

Virtual Storage Containers

With Virtual Storage Containers (VSC), you can apply different data services and performance profiles to different workloads (hosted on the same media) in a granular and flexible fashion. For example, if you need 4 drives and 50,000 IOPS for your File Services, you can do that. In the same environment you might also need to use a few drives for Object storage with different replication. You can do that too.

[image courtesy of StorONE]

Ransomware Detection (and Protection)

StorONE has been pretty keen on its ransomware protection capabilities, with the option to run immutable snapshots on volumes every 30 seconds and store over 500,000+ snaps per volume. But it has added in some improved telemetry to enable earlier detection of potential ransomware events on volumes, as well as introducing dual-key deletion of snapshots and improved two-factor authentication.

 

Thoughts

There are many things that are certain in life, including the fact that no matter how much capacity you buy for your storage array on day one, by month 18 you’re looking at ways to replace some of that capacity with higher capacity. In my former life as a diskslinger I helped many customers upgrade their arrays with increased capacity drives, and most, if not all of them, had to pay a licensing bump as well as a hardware cost for the privilege. The storage vendors would argue that that’s just the model, and for as long as you can get away with it, it is. Particularly when hardware is getting cheaper and cheaper, you need something to drive revenue. So it’s nice to see a company like StorONE looking to shake things up a little in an industry that’s probably had its way with customers for a while now. Not every storage vendor is looking to punish customers for expanding their environments, but it’s nice that those customers that were struggling with this have the option to look at other ways of using the capacity they need in a cost-effective and predictable. manner.

This doesn’t really work without the other enhancements that have gone in to StorONE, such as the improved erasure coding and automated tiering. Having a cool business model isn’t usually enough to deliver a great solution. I’m looking forward to hearing from the StorONE team in the near future about how this has been received by both existing and new customers, and what other innovations they come out with in the next 12 months.

Random Short Take #74

Welcome to Random Short Take #74. Let’s get random.

QNAP – Expand Volume With Larger Drives

This is one of those articles I’ve been meaning to post for a while, simply because I forget every time I do it how to do it properly. One way to expand the capacity of your QNAP NAS (non-disruptively) is to replace the drives one at a time with larger capacity drives. It’s recommended that you follow this process, rather than just ripping the drives out one by one and waiting for the RAID Group to expand. It’s a simple enough process to follow, although the QNAP UI has always struck me as a little on the confusing side to navigate, so I took some pictures. Note that this was done on QNAP firmware 5.0.0.1986.

Firstly, go to Storage/Snapshots under Storage in the ControlPanel. Click on Manage.

Select the Storage Pool you want to expand, and click on Manage again.

This will give you a drop-down menu. Select Replace Disks One by One.

Now select the disk you want to replace and click on Change.

Once you’ve done this for all of the disks (and it will take some time to rebuild depending on a variety of factors), click on Expand Capacity. It will ask you if you’re sure and hopefully you’ll click OK.

It’ll take a while for the RAID Group to synchronise.

You’ll notice then that, while the Storage Pool has expanded, the Volume is still the original size. Select the Volume and click on Manage.

Now you can click on Resize Volume.

The Wizard will give you information on the Storage Pool capacity and give you the option to set the new capacity of the volume. I usually click on Set to Max.

It will warn you about that. Click on OK because you like to live on the edge.

It will take a little while, but eventually your Volume will have expanded to fill the space.

 

Random Short Take #71

Welcome to Random Short Take #71. A bit of home IT in this one. Let’s get random.

Datadobi Announces StorageMAP

Datadobi recently announced StorageMAP – a “solution that provides a single pane of glass for organizations to manage unstructured data across their complete data storage estate”. I recently had the opportunity to speak with Carl D’Halluin about the announcement, and thought I’d share some thoughts here.

 

The Problem

So what’s the problem enterprises are trying to solve? They have data all over the place, and it’s no longer a simple activity to work out what’s useful and what isn’t. Consider the data on a typical file / object server inside BigCompanyX.

[image courtesy of Datadobi]

As you can see, there’re all kinds of data lurking about the place, including data you don’t want to have on your server (e.g. Barry’s slightly shonky home videos), and data you don’t need any more (the stuff you can move down to a cheaper tier, or even archive for good).

What’s The Fix?

So how do you fix this problem? Traditionally, you’ll try and scan the data to understand things like capacity, categories of data, age, and so forth. You’ll then make some decisions about the data based on that information and take actions such as relocating, deleting, or migrating it. Sounds great, but it’s frequently a tough thing to make decisions about business data without understanding the business drivers behind the data.

[image courtesy of Datadobi]

What’s The Real Fix?

The real fix, according to Datadobi, is to add a bit more automation and smarts to the process, and this relies heavily on accurate tagging of the data you’re storing. D’Halluin pointed out to me that they don’t suggest you create complex tags for individual files, as you could be there for years trying to sort that out. Rather, you add tags to shares or directories, and let the StorageMAP engine make recommendations and move stuff around for you.

[image courtesy of Datadobi]

Tags can represent business ownership, the role of the data, any action to be taken, or other designations, and they’re user definable.
[image courtesy of Datadobi]

How Does This Fix It?

You’ll notice that the process above looks awfully similar to the one before – so how does this fix anything? The key, in my opinion at least, is that StorageMAP takes away the requirement for intervention from the end user. Instead of going through some process every quarter to “clean up the server”, you’ve got a process in place to do the work for you. As a result, you’ll hopefully see improved cost control, better storage efficiency across your estate, and (hopefully) you’ll be getting a little bit more value from your data.

 

Thoughts

Tools that take care of everything for you have always had massive appeal in the market, particularly as organisations continue to struggle with data storage at any kind of scale. Gone are the days when your admins had an idea where everything on a 9GB volume was stored, or why it was stored there. We now have data stored all over the place (both officially and unofficially), and it’s becoming impossible to keep track of it all.

The key things to consider with these kinds of solutions is that you need to put in the work with tagging your data correctly in the first place. So there needs to be some thought put into what your data looks like in terms of business value. Remember that mp4 video files might not be warranted in the Accounting department, but your friends in Marketing will be underwhelmed if you create some kind of rule to automatically zap mp4s. The other thing to consider is that you need to put some faith in the system. This kind of solution will be useless if folks insist on not deleting anything, or not “believing” the output of the analytics and reporting. I used to work with customers who didn’t want to trust a vendor’s automated block storage tiering because “what does it know about my workloads?”. Indeed. The success of these kind of intelligence and automation tools relies to a certain extent on folks moving away from faith-based computing as an operating model.

But enough ranting from me. I’ve covered Datadobi a bit over the last few years, and it makes sense that all of these announcements have finally led to the StorageMAP product. These guys know data, and how to move it.

StorCentric Announces Nexsan Unity NV10000

Nexsan (a StorCentric company) recently announced the Nexsan Unity NV10000. I thought I’d share a few of my thoughts here.

What Is It? 
In the immortal words of Silicon Valley: “It’s a box“. But the Nexsan Unity NV10000 is a box with some fairly decent specifications packed in a small form-factor, including support for various 1DWPD NVMe SSDs and the latest Intel Xeon processors.
Protocol Support
Protocol support, as would be expected with the Unity, is broad, with support for File (NFS, SMB), Block (iSCSI, FC), and Object (S3) data storage protocols within the one unified platform.
Performance Enhancements
These were hinted at with the release of Unity 7.0, but the Nexsan Unity NV10000 boosts performance by increasing bandwidths of up to 25GB/s, enabling you to scale performance up as your application needs evolve.

Other Useful Features

As you’d expect from this kind of storage array, the Nexsan Unity NV10000 also delivers features such as:

  • High availability (HA);
  • Snapshots;
  • ESXi integration;
  • In-line compression;
  • FASTier™ caching;
  • Asynchronous replication;
  • Data at rest encryption; and
  • Storage pool scrubbing to protect against bit rot, avoiding silent data corruption.

Backup Target?

Unity supports a comprehensive Host OS matrix and is certified as a Veeam Ready Repository for backups. Interestingly, the Nexsan Unity NV10000 also provides data security, regulations compliance, and ransomware recoverability. The platform also supports immutable block and file and S3 object locking, for data backup that is unchangeable and cannot be encrypted, even by internal bad actors.

Thoughts

I’m not as much of a diskslinger as I used to be, but I’m always interested to hear about what StorCentric / Nexsan has been up to with its storage array releases. It strikes me that the company does well by focussing on those features that customers are looking for (fast storage, peace of mind, multiple protocols) and also by being able to put it in a form-factor that appeals in terms of storage density. While the ecosystem around StorCentric is extensive, it makes sense for the most part, with the various components coming together well to form a decent story. I like that the company has really focussed on ensuring that Unity isn’t just a cool product name, but also a key part of the operating environment that powers the solution.

Random Short Take #70

Welcome to Random Short Take #70. Let’s get random.

Random Short Take #69

Welcome to Random Short Take #69. Let’s get random.