Random Short Take #46

Welcome to Random Short Take #46. Not a great many players have worn 46 in the NBA, but one player who has is one of my favourite Aussie players: Aron “Bangers” Baynes. So let’s get random.

  • Enrico recently attended Cloud Field Day 9, and had some thoughts on NetApp’s identity in the new cloud world. You can read his insights here.
  • This article from Chris Wahl on multi-cloud design patterns was fantastic, and well worth reading.
  • I really enjoyed this piece from Russ on technical debt, and some considerations when thinking about how we can “future-proof” our solutions.
  • The Raspberry Pi 400 was announced recently. My first computer was an Amstrad CPC 464, so I have a real soft spot for jamming computers inside keyboards.
  • I enjoyed this piece from Chris M. Evans on hybrid storage, and what it really means nowadays.
  • Working from home a bit this year? Me too. Tom wrote a great article on some of the security challenges associated with the new normal.
  • Everyone has a quadrant nowadays, and Zerto has found itself in another one recently. You can read more about that here.
  • Working with VMware Cloud Director and wanting to build a custom theme? Check out this article.

Random Short Take #45

Welcome to Random Short Take #45. The number 45 has taken a bit of a beating in terms of popularity in recent years, but a few pretty solid players have nonetheless worn 45 in the NBA, including MJ and The Rifleman. My favourite from this list is A.C. Green (“slam so hard, break your TV screen“). So let’s get random.

StorCentric Announces Data Mobility Suite

StorCentric recently announced its Data Mobility Suite (DMS). I had the opportunity to talk to Surya Varanasi (StorCentric CTO) about the news, and thought I’d share some of my notes here.

 

What Is It?

DMS is being positioned as a suite of “data cloud services” by StorCentric, with a focus on:

  • Data migration;
  • Data consistency; and
  • Data operation.

It has the ability to operate across heterogeneous storage, clouds, and protocols. It’s a software solution based on subscription licensing and uses a policy-driven engine to manage data in the enterprise. It can run on bare-metal or as a VM appliance. Object storage platform / cloud support if fairly robust, with AWS, Backblaze B2, and Wasabi, amongst others, all being supported.

[image courtesy of StorCentric]

Use Cases

There are a number of scenarios where a solution like DMS makes sense. You might have a bunch of NFS storage on-premises, for example, and want to move it to a cloud storage target using S3. Another use case cited involved collaboration across multiple sites, with the example being a media company creating content in three places, and working in different time zones, and wanting to move the data back to a centralised location.

Big Ideas

Speaking to StorCentric about the announcement, it was clear that there’s a lot more on the DMS roadmap. Block storage is something the team wants to tackle, and they’re also looking to deliver analytics and ransomware alerting. There’s also a strong desire to provide governance as well. For example, if I want to copy some data somewhere and keep it for 10 years, I’ll configure DMS to take care of that for me.

 

Thoughts and Further Reading

Data management means a lot of things to a lot of people. Storage companies often focus on moving blocks and files from one spot to another, but don’t always do a solid job of capturing data needs to be stored where it does. Or how, for that matter. There’s a lot more to data management than keeping ones and zeroes in a safe place. But it’s not just about being able to move data from one spot to another. It’s about understanding the value of your data, and understanding where it needs to be to deliver the most value to your organisation. Whilst it seems like DMS is focused primarily on moving data from one spot to another, there’s plenty of potential here to develop a broader story in terms of data governance and mobility. There’s built-in security, and the ability to apply levels of data governance to data in various locations. The greater appeal here is also the ability to automate the movement of data to different places based on policy. This policy-driven approach becomes really interesting when you start to look at complicated collaboration scenarios, or need to do something smart with replication or data migration.

Ultimately, there are a bunch of different ways to get data from one point to another, and a bunch of different reasons why you might need to do that. The value in something like DMS is the support for heterogeneous storage platforms, as well as the simple to use GUI support. Plenty of data migration tools come with extremely versatile command line interfaces and API support, but the trick is delivering an interface that is both intuitive and simple to navigate. It’s also nice to have a few different use cases met with one tool, rather than having to reach into the bag a few different times to solve very similar problems. StorCentric has a lot of plans for DMS moving forward, and if those plans come to fruition it’s going to form a very compelling part of the typical enterprise’s data management toolkit. You can read the press release here.

Quobyte Announces 3.0

Quobyte recently announced Release 3.0 of its software. I had the opportunity to speak to Björn Kolbeck (Co-Founder and CEO) about the release, and thought I’d share some thoughts here.

 

About Quobyte

If you haven’t heard of Quobyte before, it was founded in 2013 by some ex-Googlers and HPC experts. The folks at Quobyte were heavily influenced by Google’s scale-out software model and wanted to bring that to the enterprise. Quobyte has had software in production since 2016 and has customers across a range of industry verticals, including financial services and media streaming. It’s not really object storage, more a parallel file system or, at a stretch, scale-out NAS.

 

The Tech

Kolbeck describes Quobyte as “storage for Generation Scale-Out” and is focussed on “getting storage out of the ugly corner of specialised appliances”.

Unlimited Performance

  • Linear scaling delivers unlimited performance
  • No bottlenecks – scale from small to 1000s of servers
  • No more NFS – it’s part of the problem

Deploy Anywhere

  • True software storage runs anywhere – bare metal, containers, cloud
  • Almost any x86t server – no appliances

Unconditional Simplicity

  • Anyone can do storage, it’s just another Linux application
  • All in user space, installs in minutes

 

The Announcement

Free Edition

The first part of the announcement is that there’s a free edition (previously there was a 45 day trial on offer). It’s limited in terms of capacity, support, and file system clients, but could be useful in labs and smaller environments.

[image courtesy of Quobyte]

3.0 Release

The 3.0 release is also a big part of Quobyte’s news, with the new version delivering a bunch of new features, most of which are outlined below.

360 Security

  • Holistic data protection
  • End to end AES encryption (in transit / at rest / untrusted storage nodes)
  • Selective TLS support
  • Access keys for the file system
  • X.509 certificates
  • Event stream (metadata, file access)

Policy Engine

Powerful Policy Engine

  • For: Tenant, volume, file, client
  • Control: Layout, tiering, QoS, recoding, caching
  • Dynamic: Runtime re-configurable

Automated

  • Auto file layout: replication + EC and Flash + HDD
  • Auto selection of replication factor, EC schema

Self-Service

Quobyte is looking to deliver a “cloud-like experience” with its self-service capabilities.

Login for users

  • Manage access keys
  • Check resource consumption

Authenticate using access keys

  • S3
  • File system driver
  • K8s / CSI
  • User-space drivers: HDFS, TF, MPI-IO

Multi-Cluster

Data Mover

  • Bi-directional sync (evental consistency)
  • Policy-based data tiering between clusters
  • Recoding

TLS between clusters

More Native Drivers

HDFS

MPI-IO

Benefit of kernel bypass

  • Lower latency
  • Less memory bandwidth

 

Thoughts and Further Reading

One of the challenges with software-defined storage is invariably the constraint that poor hardware choices can put on performance. Kolbeck acknowledged that Quobyte is “as fast as your hardware”. I asked him whether Quobyte provided guidance on hardware choices that worked well with the platform. There is a bunch of recommended (and tested) hardware listed on this page. He did mention that whichever way you decided to go, it was recommended to stick with either Mellanox or Broadcom NICs due to issues observed with other vendors’ Linux drivers. There’re also recommendations on the site for public cloud instance sizing covering AWS, GCP, and Oracle.

Quobyte is being deployed to support scale-out workloads in the enterprise across a number of sectors including financial services, life sciences, media and entertainment, and manufacturing in Europe and Asia. Kolbeck noted that one of the interesting things about the advent of smart everything is that “car manufacturers are suddenly in the machine learning field” and looking for new ways to support their businesses.

There are a lot of reasons to like software-defined storage offerings. You can generally run them on anything, and performance enhancements can frequently be had via code upgrades. That’s not to say that you don’t get that with the big box slingers, but the flexibility of hardware choice has tremendous appeal, particularly in the enterprise market where it can feel like the margin on commodity hardware can be exorbitant. Quobyte hasn’t been around forever, but the folks over there seem to have a pretty solid heritage in software-defined and scale-out storage solutions – a good sign if you’re in the market for a software-defined, scale-out storage solution. Some folks are going to rue the lack of NFS support, but I’m sure Kolbeck and the team would be happy to sit down and discuss with them why that’s no great loss. There’s some pretty cool stuff in this release, and the free edition is definitely worth taking for a spin. I’m looking forward to hearing more from Quobyte over the next little while.

StorONE Q3-2020 Update

StorONE recently announced details of its Q3-2020 software release. I had the opportunity to talk about the announcement with George Crump and thought I’d share some brief thoughts here.

 

Release Highlights

Performance Improvements

One of the key highlights of this release is significant performance improvements for the platform based purely on code optimisations. Crump tells me that customers with Intel Optane and NVMe SSDs will be extremely happy with what they see. What’s also notable is that customers still using high latency media such as hard disk drives will still see a performance improvement of 15 – 20%.

Data Protection

StorONE has worked hard on introducing some improved resilience for the platform as well, with two key features being made available:

  • vRack; and
  • vReplicate.

vRack provides the ability to split S1 storage across more than one rack (or row, for that matter) to mitigate any failures impacting the rack hosting the controllers and disk enclosures. You can now also set tolerance for faults at an enclosure level, not just a drive level.

[image courtesy of StorONE]

vReplicate extends S1:Replicate’s capabilities to provide cascading replication. You can now synchronously replicate between data centres or campus sites and then asynchronously send that data to another site, hundreds of kilometres away if necessary. Primary systems can be an All-Flash Array.next, traditional All-Flash Array, or a Hybrid Array, and the replication target can be an inexpensive hard disk only S1 system.

[image courtesy of StorONE]

There’s now full support for Volume Shadow Copy Service (VSS) for S1:Snap users.

 

Other Enhancements

Some of the other enhancements included with this release are:

  • Improved support for NVMe-oF (including the ability to simultaneously support iSCSI and FC along with NVMe);
  • Improved NAS capability, with support for quotas and NIS / LDAP; and
  • Downloadable stats for increased insights.

 

Thoughts

Some of these features might seem like incremental improvements, but this is an incremental release. I like the idea of supporting legacy connections while supporting the ability to add newer tech to the platform, and providing a way forward in terms of hardware migration. The vRack resiliency concept is also great, and a salient reminder that the ability to run this code on commodity hardware makes some of these types of features a little more accessible. I also like the idea of being able to download analytics data and do things with it to gain greater insights into what the system is doing. Sure, it’s an incremental improvement, but an important one nonetheless.

I’ve been a fan of the StorONE story for some time now (and not just because the team slings a few dollars my way to support the site every now and then). I think the key to much of StorONE’s success has been that it hasn’t gotten caught up trying to be a storage appliance vendor, and has instead focussed on delivering reliable code on commodity systems that results in a performance-oriented storage platform that continues to improve from a software perspective without being tied to a particular hardware platform. The good news is though, when new hardware becomes available (such as Optane), it’s not a massive problem to incorporate it into the solution.

StorONE has always talked a big game in terms of raw performance numbers, but I think it’s the addition of features such as vRack and improvements to the replication capability that really makes it a solution worth investigating. It doesn’t hurt that you can check the pricing calculator out for yourself before you decide to go down the path of talking to StorONE’s sales team. I’m looking forward to seeing what StorONE has in store in the next little while, as I get the impression it’s going to be pretty cool. You can read details of the update here.

Pure Storage Acquires Portworx

Pure Storage announced its intention to acquire Portworx in mid-September. Around that time I had the opportunity to talk about the news with Goutham Rao (Portworx CTO) and Matt Kixmoeller (Pure Storage VP, Strategy) and thought I’d share some brief thoughts here.

 

The News

Pure and Portworx have entered an agreement that will see Pure pay approximately $370M US in cash. Portworx will form a new Cloud Native Business Unit inside Pure to be led by Portworx CEO Murli Thirumale. All Portworx founders are joining Pure, with Pure investing significantly to grow the new business unit. According to Pure, “Portworx software to continue as-is, supporting deployments in any cloud and on-premises, and on any bare metal, VM, or array-based storage”. It was also noted that “Portworx solutions to be integrated with Pure yet maintain a commitment to an open ecosystem”.

About Portworx

Described as the “leading Kubernetes data services platform”, Portworx was founded in 2014 in Los Altos, CA. It runs a 100% software, subscription, and cloud business model with development and support sites in California, India, and Eastern Europe. The product has been GA since 2017, and is used by some of the largest enterprise and Cloud / SaaS companies globally.

 

What’s A Portworx?

The idea behind Portworx is that it gives you data services for any application, on any Kubernetes distribution, running on any cloud, any infrastructure, and at any stage of the application lifecycle. To that end, it’s broken up into a bunch of different components, and runs in the K8s control plane adjacent to the applications.

PX-Store

  • Software-defined storage layer that automates container storage for developers and admins
  • Consistent storage APIs: cloud, bare metal, or arrays

PX-Migrate

  • Easily move applications between clusters
  • Enables hybrid cloud and multi-cloud mobility

PX-Backup

  • Application-consistent backup for cloud native apps with all k8s artefacts and state
  • Backup to any cloud or on-premises object storage

PX-Secure

  • Implement consistent encryption and security policies across clouds
  • Enable multi-tenancy with access controls

PX-DR

  • Sync and async replication between Availability Zones and regions
  • Zero RPO active / active for high resiliency

PX-Autopilot

  • GitOps-driven automation allows for easier platform for non-storage experts to deploy stateful applications, monitors everything about an application, reacts and prevents problems from happening
  • Auto-scale storage as your app grows to reduce costs

 

How It Fits Together

When you bring Portworx into the Pure Storage picture, you start to see that it fits well with the existing Pure Storage picture. In the picture below you’ll also see support for the standard container storage interface (CSI) to work with other vendors.

[image courtesy of Pure Storage]

Also worth noting is that PX-Essentials remains free forever for workloads under 5TB and 5 nodes).

 

Thoughts and Further Reading

I think this is a great move by Pure, mainly because it lends them a whole lot more credibility with the DevOps folks. Pure was starting to make inroads with Pure Storage Orchestrator, and I think this move will strengthen that story. Giving Portworx access to Pure’s salesforce globally is also going to broaden its visibility in the market and open up doors to markets that may have been difficult to get into previously.

Persistent storage for containers is heating up. As Rao pointed out in our discussion, “as container adoption grows, storage becomes a problem”. Portworx already had a good story to tell in this space, and Pure is no slouch when it comes to delivering advanced storage capabilities across a variety of platforms. I like that the messaging has been firmly based in maintaining the openness of the platform and I’m interested to see what other integrations happen as the two companies start working more closely together. If you’d like another perspective on the news, check out Chris Evans’s article here.

Random Short Take #44

Welcome to Random Short Take #44. A few players have worn 44 in the NBA, including Danny Ainge and Pistol Pete, but my favourite from this list is Keith Van Horn.  A nice shooting touch and strong long sock game. Let’s get random.

  • ATMs are just computers built to give you money. And it’s scary to think of the platforms that are used to deliver that functionality. El Reg pointed out a recent problem with one spotted in the wild in Ngunnawal.
  • Speaking of computing at the edge, I found this piece from Keith interesting. As much as things change they stay the same. I think he’s spot on when he says “[m]anufacturers and technology companies must come together with modular solutions that enable software upgrades for these assets’ lives”. We need to be demanding more from the industry when it comes to some of this stuff.
  • Heard about Project Monterey at VMworld and wanted to know more? Pensando has you covered.
  • I enjoyed this article from Preston about the difference between bunkers and vaults – worth checking out even if you’re not a Dell EMC customer.
  • Cloud – it can be tough to know which way to go. And a whole bunch of people have an interest in you using their particular solution. This article from Chris Evans was particularly insightful.
  • DH2i has launched DxOdyssey for IoT – you can read more about that here.
  • Speaking of news, Retrospect recently announced Backup 17.5 too. There are some cloud improvements, and support for macOS Big Sur beta.
  • It’s the 30th anniversary of Vanilla Ice’s “Ice Ice Baby“, and like me you were probably looking for a comprehensive retrospective on Vanilla Ice’s career. Look no further than this article over at The Ringer.

Storage Field Day 20 – Wrap-up and Link-o-rama

Disclaimer: I recently attended Storage Field Day 20.  My flights, accommodation and other expenses were paid for by Tech Field Day. There is no requirement for me to blog about any of the content presented and I am not compensated in any way for my time at the event.  Some materials presented were discussed under NDA and don’t form part of my blog posts, but could influence future discussions.

This is a quick post to say thanks once again to Stephen and Ben, and the presenters at Storage Field Day 20. I had a super fun and educational time. For easy reference, here’s a list of the posts I did covering the events (they may not match the order of the presentations).

Storage Field Day 20 – I’ll Be At Storage Field Day 20

Storage Field Day 20 – (Fairly) Full Disclosure

Cisco MDS, NVMe, and Flexibility

Qumulo – Storage Your Way

Pure Storage Announces Second Generation FlashArray//C with QLC

Nebulon – It’s Server Storage Jim, But Not As We Know It

VAST Data – The Best Is Yet To Come

Intel Optane And The DAOS Storage Engine

 

Also, here’s a number of links to posts by my fellow delegates (in no particular order). They’re all very smart people, and you should check out their stuff, particularly if you haven’t before. I’ll attempt to keep this updated as more posts are published. But if it gets stale, the Storage Field Day 20 landing page will have updated links.

 

Jason Benedicic (@JABenedicic)

Nebulon Shadow Storage

 

David Chapa (@DavidChapa)

“High Optane” Fuel For Performance

 

Becky Elliott (@BeckyLElliott)

Guess Who’s Attending Storage Field Day 20?

 

Ray Lucchesi (@RayLucchesi)

Storage that provides 100% performance at 99% full

106: Greybeards talk Intel’s new HPC file system with Kelsey Prantis, Senior Software Eng. Manager, Intel

 

Vuong Pham (@Digital_KungFu)

Storage Field Day 20.. oh yeah!!

 

Keiran Shelden (@Keiran_Shelden)

Let’s Zoom to SFD20

 

Enrico Signoretti (@esignoretti)

An Intriguing Approach to Modern Data Center Infrastructure

Is Scale-Out File Storage the New Black?

 

Paul Stringfellow (@TechStringy)

Storage Field Day and The Direction of Travel

 

Keith Townsend (@CTOAdvisor)

Will the DPU kill the Storage Array?

 

[image courtesy of Stephen Foskett]

Random Short Take #43

Welcome to Random Short Take #43. A few players have worn 43 in the NBA, including Frank Brickowski, but my favourite from this list is Red Kerr (more for his commentary chops than his game, I think).  Let’s get random.

  • Mike Wilson has published Part 2 of his VMware VCP 2020 Study Guide and it’s a ripper. Check it out here. I try to duck and weave when it comes to certification exams nowadays, but these kind of resources are invaluable.
  • It’s been a while since I had stick time with Data Domain OS, but Preston’s article on password hardening was very useful.
  • Mr Foskett bought a cloud, of sorts. Read more about that here. Anyone who knows Stephen knows that he’s all about what’s talking about what’s happening in the industry, but I do enjoy reading about these home projects as well.
  • Speaking of clouds, Rancher was named “A Leader” in multi-cloud container development platforms by an independent research firm. You can read the press release here.
  • Datadobi had a good story to share about what it did with UMass Memorial Health Care. You can read the story here.
  • Steve O has done way too much work understanding how to change the default theme in Veeam Enterprise Manager 10 and documenting the process so you don’t need to work it out. Read about the process here.
  • Speaking of data protection, Zerto has noticed Azure adoption increasing at quite a pace, amongst other things.
  • This was a great article on open source storage from Chin-Fah.

Intel Optane And The DAOS Storage Engine

Disclaimer: I recently attended Storage Field Day 20.  My flights, accommodation and other expenses were paid for by Tech Field Day. There is no requirement for me to blog about any of the content presented and I am not compensated in any way for my time at the event.  Some materials presented were discussed under NDA and don’t form part of my blog posts, but could influence future discussions.

Intel recently presented at Storage Field Day 20. You can see videos of the presentation here, and download my rough notes from here.

 

Intel Optane Persistent Memory

If you’re a diskslinger, you’ve very likely heard of Intel Optane. You may have even heard of Intel Optane Persistent Memory. It’s a little different to Optane SSD, and Intel describes it as “memory technology that delivers a unique combination of affordable large capacity and support for data persistence”. It looks a lot like DRAM, but the capacity is greater, and there’s data persistence across power losses. This all sounds pretty cool, but isn’t it just another form factor for fast storage? Sort of, but the application of the engineering behind the product is where I think it starts to get really interesting.

 

Enter DAOS

Distributed Asynchronous Object Storage (DAOS) is described by Intel as “an open source software-defined scale-out object store that provides high bandwidth, low latency, and high I/O operations per second (IOPS) storage containers to HPC applications”. It’s ostensibly a software stack built from the ground up to take advantage of the crazy speeds you can achieve with Optane, and at scale. There’s a handy overview of the architecture available on Intel’s website. Traditional object (and other storage systems) haven’t really been built to take advantage of Optane in quite the same way DAOS has.

[image courtesy of Intel]

There are some cool features built into DAOS, including:

  • Ultra-fine grained, low-latency, and true zero-copy I/O
  • Advanced data placement to account for fault domains
  • Software-managed redundancy supporting both replication and erasure code with online rebuild
  • End-to-end (E2E) data integrity
  • Scalable distributed transactions with guaranteed data consistency and automated recovery
  • Dataset snapshot capability
  • Security framework to manage access control to storage pools
  • Software-defined storage management to provision, configure, modify, and monitor storage pools

Exciting? Sure is. There’s also integration with Lustre. The best thing about this is that you can grab it from Github under the Apache 2.0 license.

 

Thoughts And Further Reading

Object storage is in its relative infancy when compared to some of the storage architectures out there. It was designed to be highly scalable and generally does a good job of cheap and deep storage at “web scale”. It’s my opinion that object storage becomes even more interesting as a storage solution when you put a whole bunch of really fast storage media behind it. I’ve seen some media companies do this with great success, and there are a few of the bigger vendors out there starting to push the All-Flash object story. Even then, though, many of the more popular object storage systems aren’t necessarily optimised for products like Intel Optane PMEM. This is what makes DAOS so interesting – the ability for the storage to fundamentally do what it needs to do at massive scale, and have it go as fast as the media will let it go. You don’t need to worry as much about the storage architecture being optimised for the storage it will sit on, because the folks developing it have access to the team that developed the hardware.

The other thing I really like about this project is that it’s open source. This tells me that Intel are both focused on Optane being successful, and also focused on the industry making the most of the hardware it’s putting out there. It’s a smart move – come up with some super fast media, and then give the market as much help as possible to squeeze the most out of it.

You can grab the admin guide from here, and check out the roadmap here. Intel has plans to release a new version every 6 months, and I’m really looking forward to seeing this thing gain traction. For another perspective on DAOS and Intel Optane, check out David Chapa’s article here.