Pure//Accelerate 2019 – Cloud Block Store for AWS

Disclaimer: I recently attended Pure//Accelerate 2019.  My flights, accommodation, and conference pass were paid for by Pure Storage. There is no requirement for me to blog about any of the content presented and I am not compensated by Pure Storage for my time at the event.  Some materials presented were discussed under NDA and don’t form part of my blog posts, but could influence future discussions.

Cloud Block Store for AWS from Pure Storage has been around for a little while now. I had the opportunity to hear about it in more depth at the Storage Field Day Exclusive event at Pure//Accelerate 2019 and thought I’d share some thoughts here. You can grab a copy of my rough notes from the session here, and video from the session is available here.

 

Cloud Vision

Pure Storage have been focused on making everything related to their products effortless from day 1. An example of this approach is the FlashArray setup process – it’s really easy to get up and running and serving up storage to workloads. They wanted to do the same thing with anything they deliver via cloud services as well. There is, however, something of a “cloud divide” in operation in the industry. If you’re familiar with the various cloud deployment options, you’ll likely be aware that on-premises and hosted cloud is a bit different to public cloud. They:

  • Deliver different application architectures;
  • Deliver different management and consumption experience; and
  • Use different storage.

So what if Pure could build application portability and deliver common shared data services?

Pure have architected their cloud service to leverage what they call “Three Pillars”:

  • Build Your Cloud
  • Run anywhere
  • Protect everywhere

 

What Is It?

So what exactly is Cloud Block Store for AWS then? Well, imagine if you will, that you’re watching an episode of Pimp My Ride, and Xzibit is talking to an enterprise punter about how he or she likes cloud, and how he or she likes the way Pure Storage’s FlashArray works. And then X says, “Hey, we heard you liked these two things so we put this thing in the other thing”. Look, I don’t know the exact situation where this would happen. But anyway …

  • 100% software – deploys instantly as a virtual appliance in the cloud, runs only as long as you need it;
  • Efficient – deduplication, compression, and thin provisioning deliver capacity and performance economically;
  • Hybrid – easily migrate data bidirectionally, delivering data portability and protection across your hybrid cloud;
  • Consistent APIs – developers connect to storage the same way on-premises and in the cloud. Automated deployment with Cloud Formation templates;
  • Reliable, secure – delivers industrial-strength perfromance, reliability & protection with Multi-AZ HA, NDU, instant snaps and data at rest encryption; and
  • Flexible – pay as you go consumption model to best match your needs for production and development.

[image courtesy of Pure Storage]

Architecture

At the heart of it, the architecture for CVS is not dissimilar to the FlashArray architecture. There’re controllers, drives, NVRAM, and a virtual shelf.

  • EC2: CBS Controllers
  • EC2: Virtual Drives
  • Virtual Shelf: 7 Virtual drives in Spread Placement Group
  • EBS IO1: NVRAM, Write Buffer (7 total)
  • S3: Durable persistent storage
  • Instance Store: Non-Persistent Read Mirror

[image courtesy of Pure Storage]

What’s interesting, to me at least, is how they use S3 for persistent storage.

Procurement

How do you procure CBS for AWS? I’m glad you asked. There are two procurement options.

A – Pure as-a-Service

  • Offered via SLED / CLED process
  • Minimums 100TiB effective used capacity
  • Unified hybrid contracts (on-premises and CBS, CBS)
  • 1 year to 3 year contracts

B – AWS Marketplace

  • Direct to customer
  • Minimum, 10 TiB effective used capacity
  • CBS only
  • Month to month contract or 1 year contract

 

Use Cases

There are a raft of different use cases for CBS. Some of them made sense to me straight away, some of them took a little time to bounce around in my head.

Disaster Recovery

  • Production instance on-premises
  • Replicate data to public cloud
  • Fail over in DR event
  • Fail back and recover

Lift and shift

  • Production instance on-premises
  • Replicate data to public cloud
  • Run the same architecture as before
  • Run production on CBS

Use case: Dev / test

  • Replicate data to public cloud
  • Instantiate test / dev instances in public cloud
  • Refresh test / dev periodically
  • Bring changes back on-premises
  • Snapshots are more costly and slower to restore in native AWS

ActiveCluster

  • HA within an availability zone and / or across availability zones in an AWS region (ActiveCluster needs <11ms latency)
  • No downtime when a Cloud Block Store Instance goes away or there is a zone outage
  • Pure1 Cloud Mediator Witness (simple to manage and deploy)

Migrating VMware Environments

VMware Challenges

  • AWS does not recognise VMFS
  • Replicating volumes with VMFS will not do any good

Workaround

  • Convert VMFS datastore into vVOLs
  • Now each volume has the Guest VM’s file system (NTFS, EXT3, etc)
  • Replicate VMDK vVOLs to CBS
  • Now the volumes can be mounted to EC2 with matching OS

Note: This is for the VM’s data volumes. The VM boot volume will not be usable in AWS. The VM’s application will need to be redeployed in native AWS EC2.

VMware Cloud

VMware Challenges

  • VMware Cloud does not support external storage, it only supports vSAN

Workaround

  • Connect Guest VMs directly to CBS via iSCSI

Note: I haven’t verified this myself, and I suspect there may be other ways to do this. But in the context of Pure’s offering, it makes sense.

 

Thoughts and Further Reading

There’s been a feeling in some parts of the industry for the last 5-10 years that the rise of the public cloud providers would spell the death of the traditional storage vendor. That’s clearly not been the case, but it has been interesting to see the major storage slingers evolving their product strategies to both accommodate and leverage the cloud providers in a more effective manner. Some have used the opportunity to get themselves as close as possible to the cloud providers, without actually being in the cloud. Others have deployed virtualised versions of their offerings inside public cloud and offered users the comfort of their traditional stack, but off-premises. There’s value in these approaches, for sure. But I like the way that Pure have taken it a step further and optimised their architecture to leverage some of the features of what AWS can offer from a cloud hardware perspective.

In my opinion, the main reason you’d look to leverage something like CBS on AWS is if you have an existing investment in Pure and want to keep doing things a certain way. You’re also likely using a lot of traditional VMs in AWS and want something that can improve the performance and resilience of those workloads. CBS is certainly a great way to do this. If you’re already running a raft of cloud-native applications, it’s likely that you don’t necessarily need the features on offer from CBS, as you’re already (hopefully) using them natively. I think Pure understand this though, and aren’t pushing CBS for AWS as the silver bullet for every cloud workload.

I’m looking forward to seeing what the market uptake on this product is like. I’m also keen to crunch the numbers on running this type of solution versus the cost associated with doing something on-premises or via other means. In any case, I’m looking forward to see how this capability evolves over time, and I think CBS on AWS is definitely worthy of further consideration.

Random Short Take #23

Want some news? In a shorter format? And a little bit random? This listicle might be for you.

  • Remember Retrospect? They were acquired by StorCentric recently. I hadn’t thought about them in some time, but they’re still around, and celebrating their 30th anniversary. Read a little more about the history of the brand here.
  • Sometimes size does matter. This article around deduplication and block / segment size from Preston was particularly enlightening.
  • This article from Russ had some great insights into why it’s not wise to entirely rule out doing things the way service providers do just because you’re working in enterprise. I’ve had experience in both SPs and enterprise and I agree that there are things that can be learnt on both sides.
  • This is a great article from Chris Evans about the difficulties associated with managing legacy backup infrastructure.
  • The Pure Storage VM Analytics Collector is now available as an OVA.
  • If you’re thinking of updating your Mac’s operating environment, this is a fairly comprehensive review of what macOS Catalina has to offer, along with some caveats.
  • Anthony has been doing a bunch of cool stuff with Terraform recently, including using variable maps to deploy vSphere VMs. You can read more about that here.
  • Speaking of people who work at Veeam, Hal has put together a great article on orchestrating Veeam recovery activities to Azure.
  • Finally, the Brisbane VMUG meeting originally planned for Tuesday 8th has been moved to the 15th. Details here.

Pure//Accelerate 2019 – (Fairly) Full Disclosure

Disclaimer: I recently attended Pure//Accelerate 2019.  My flights, accommodation, and conference pass were paid for by Pure Storage. There is no requirement for me to blog about any of the content presented and I am not compensated by Pure Storage for my time at the event.  Some materials presented were discussed under NDA and don’t form part of my blog posts, but could influence future discussions.

Here are my notes on gifts, etc, that I received as an attendee at Pure//Accelerate 2019. Apologies if it’s a bit dry but I’m just trying to make it clear what I received during this event to ensure that we’re all on the same page as far as what I’m being influenced by. I’m going to do this in chronological order, as that was the easiest way for me to take notes during the week. Whilst every attendee’s situation is different, I was paid by me employer to be at this event.

 

Saturday

My wife kindly dropped me at the airport. I flew Qantas economy class from BNE – LAX – AUS courtesy of Pure Storage. I had a 5 hour layover at LAX. I stopped at the Rolling Stone Bar and Grill in the Terminal 7 and had a breakfast burrito. It wasn’t the best, but anything is pretty good after the smell of airplane food. When I got to Austin I was met by a driver that Pure had organised. I grabbed my suitcase and we travelled to the Fairmont Austin (paid for by Pure) in one of those big black SUVs that are favoured by many of the limousine companies.

I got presentable and then went down to the hotel bar to catch up with Alastair Cooke and his wife Tracey, Matt Leib, Gina Minks, and Leah Schoeb. I had a gin and tonic, paid for by Alastair, and then took the hotel courtesy car to Austin City Limits with Matt to see Joe Russo’s Almost Dead. It’s not the sort of gig I’d normally go to, but I appreciate live music in most forms, the crowd was really into it, and it’s always great to spend time with Matt. Matt also very kindly paid for my entry to the gig and bought me a beer there (a 16oz can of Land Shark Lager). I had a second beer and bought one for Matt too.

 

Sunday

I hadn’t really eaten since LAX, so I hit up Matt to come to lunch with me. We went for a wander downtown in Austin and ended up on 6th Street at Chupacabra Cantina y Tacqueria. I had one of the West Coast Burritos, a huge flour tortilla stuffed with refried beans, green chilli rice, jack cheese, crispy potato, lettuce, tomato, onion and chicken Tinga filling. It was delicious. I also had two Twisted X Austin Lager beers to wash it down.

In the afternoon I caught up with Matt and Chris Evans in the hotel bar. I had 3 Modelo Especial beers – these were kindly paid for by Emily Gallagher from Touchdown PR.

The Tech Field Day people all got together for dinner at Revue in the hotel. I had 3 Vista Adair Kolsch beers, some shrimp gyoza, chilli wonton dumplings, and okonomiyaki. This was paid for by Tech Field Day.

 

Monday

On Monday morning I had breakfast at the hotel. This was a buffet-style affair and I had scrambled eggs, huevo rancheros, bacon, jalapeño sausage, charcuterie, salmon and cream cheese, and coffee. This was paid for by Pure Storage. I received a gift bag at registration. This included a:

  • Pure//Accelerate cloth tote bag;
  • Rocketbook Everlast notebook;
  • “Flash Was Only The Beginning” hardcover book;
  • Porter 12 oz portable ceramic mug;
  • h2go Concord 25 oz stainless steel bottle; and
  • 340g bag of emporium medium house blend cuvée coffee.

For lunch I had beef brisket, BBQ sauce and some green salad. I also picked up a Pure FlashArray//C t-shirt during the Storage Field Day Exclusive event.

Before dinner I had a Modelo in the hotel – this was paid for by Tech Field Day. We then attended an Analysts and Influencers reception at Banger’s. I had 3 beers there (some kind of Pilsner) and a small amount of BBQ. I then made my way over to Parkside on 6th Street for an APJ event. I had 4 Austin Limits Lagers there and some brisket and macaroni and cheese. I should have smoke-bombed at that point but didn’t and ended up getting my phone swiped from a bar. Lesson learnt.

 

Tuesday

I skipped breakfast in favour of some more sleep. For lunch I had beef tacos in the Analysts area. Dinner was an Analyst and Influencer and Executive Program reception at the hotel. I had 3 Modelo beers, some dumplings, and some beef skewers. I turned in relatively early as the jet-lag was catching up with me.

 

Wednesday

For breakfast we were in the Solutions Exchange area for a private tour of the Pure setup. I had a greasy ham, cheese and egg croissant, some fruit, and 2 coffees. After the keynote I picked up some Rubrik socks.

In the afternoon I took a taxi to the Austin PD to attempt to report my phone. I then grabbed lunch with Matt Leib at P. Terry’s Burger Stand downtown. I had a hamburger and a chocolate shake. Matt paid for this. Matt then paid for a ride-sharing service to the local Apple Store where I picked up a new handset. We then took another car back to the hotel, which Matt kindly paid for.

We had dinner at Banger’s with the remaining Tech Field Day crew. I had 3 Austin Beerworks Pearl-Snap beers, boiled peanuts, chilli fries, and jalapeño sausage. It was delicious. This was paid for by Tech Field Day. I then headed to Austin City Limits for the Pure//Accelerate appreciation party. Weezer were playing, and I was lucky enough to get a photo with them (big thanks to Stephen Foskett and Armi Banaria for sorting me out!).

I had 3 Landshark Lager beers during the concert. After the show we retired to the hotel bar where I had 2 more Modelo beers before calling it a night.

 

Thursday

On Thursday morning I ran into Craig Waters and Justin Warren and joined them for a coffee at Houndstooth Coffee (I had the iced latte to try and fight off the heat). This was paid for by Craig. We then headed to Fareground. I had a burger with bacon and cheese from Contigo. It was delicious. This was also paid for by Craig.

Returning to the hotel, I bumped into my old mentor Andrew Fisher and he bought me a few Modelos in the bar while re-booking his flights due to some severe weather issues in Houston. I then took a Pure-provided car service to the airport and made my way home to Brisbane via LAX.

Big thanks to Pure Storage for having me over for the week, and big thanks to everyone who spent time with me at the event (and after hours) – it’s a big part of why I keep coming back to these types of events.

Random Short Take #22

Oh look, another semi-regular listicle of random news items that might be of some interest.

  • I was at Pure Storage’s //Accelerate conference last week, and heard a lot of interesting news. This piece from Chris M. Evans on FlashArray//C was particularly insightful.
  • Storage Field Day 18 was a little while ago, but that doesn’t mean that the things that were presented there are no longer of interest. Stephen Foskett wrote a great piece on IBM’s approach to data protection with Spectrum Protect Plus that’s worth read.
  • Speaking of data protection, it’s not just for big computers. Preston wrote a great article on the iOS recovery process that you can read here. As someone who had to recently recover my phone, I agree entirely with the idea that re-downloading apps from the app store is not a recovery process.
  • NetApp were recently named a leader in the Gartner Magic Quadrant for Primary Storage. Say what you will about the MQ, a lot of folks are still reading this report and using it to help drive their decision-making activities. You can grab a copy of the report from NetApp here. Speaking of NetApp, I’m happy to announce that I’m now a member of the NetApp A-Team. I’m looking forward to doing a lot more with NetApp in terms of both my day job and the blog.
  • Tom has been on a roll lately, and this article on IT hero culture, and this one on celebrity keynote speakers, both made for great reading.
  • VMworld US was a little while ago, but Anthony‘s wrap-up post had some great content, particularly if you’re working a lot with Veeam.
  • WekaIO have just announced some work their doing Aiden Lab at the Baylor College of Medicine that looks pretty cool.
  • Speaking of analyst firms, this article from Justin over at Forbes brought up some good points about these reports and how some of them are delivered.

Pure Storage Expands Portfolio, Adds Capacity And Performance

Disclaimer: I recently attended Pure//Accelerate 2019.  My flights, accommodation, and conference pass were paid for by Pure Storage. There is no requirement for me to blog about any of the content presented and I am not compensated by Pure Storage for my time at the event.  Some materials presented were discussed under NDA and don’t form part of my blog posts, but could influence future discussions.

Pure Storage announced two additions to its portfolio of products today: FlashArray//C and DirectMemory Cache. I had the opportunity to hear about these two products at the Storage Field Day Exclusive event at Pure//Accelerate 2019 and thought I’d share some thoughts here.

 

DirectMemory Cache

DirectMemory Cache is a high-speed caching system that reduces read latency for high-locality, performance-critical applications.

  • High speed: based on Intel Optane SCM drives
  • Caching system: repeated accesses to “hot data” are sped up automatically – no tiering = no configuration
  • Read latency: only read performance is affected – no changes to latency
  • High-locality: only workloads that reuse often a dates that fits in the cache will benefit
  • Performance-Critical: high-throughput latency sensitive workloads

According to Pure, “DirectMemory Cache is the functionality within Purity that provides direct access to data and accelerates performance critical applications”. Note that this is only for read data, write caching is still done via DRAM.

How Can This Help?

Pure has used Pure1 Meta analysis to arrive at the following figures:

  • 80% of arrays can achieve 20% lower latency
  • 40% of arrays can achieve 30-50% lower latency (up to 2x boost)

So there’s some real potential to improve existing workloads via the use of this read cache.

DirectMemory Configurations

Pure Storage DirectMemory Modules plug directly into FlashArray//X70 and //X90, are inserted into the chassis, and are available in the following configurations:

  • 3TB (4x750GB) DirectMemory Modules
  • 6TB (8x750GB) DirectMemory Modules

Top of Rack Architecture

Pure are positioning the “top of rack” architecture as a way to compete some of the architectures that have jammed a bunch of flash in DAS or in compute to gain increased performance. The idea is that you can:

  • Eliminate data locality;
  • Bring storage and compute closer;
  • Provide storage services that are not possible with DAS;
  • Bring the efficiency of FlashArray to traditional DAS applications; and
  • Offload storage and networking load from application CPUs.

 

FlashArray//C

Typical challenges in Tier 2

Things can be tough in the tier 2 storage world. Pure outlined some of the challenges they were seeking to address by delivering a capacity optimised product.

Management complexity

  • Complexity / management
  • Different platforms and APIs
  • Interoperability challenges

Inconsistent Performance

  • Variable app performance
  • Anchored by legacy disk
  • Undersized / underperforming

Not enterprise class

  • <99.9999% resiliency
  • Disruptive upgrades
  • Not evergreen

The C Stands For Capacity Optimised All-Flash Array

Flash performance at disk economics

  • QLC architecture enables tier 2 applications to benefit from the performance of all-flash – predictable 2-4ms latency, 5.2PB (effective) in 9U delivers 10x consolidation for racks and racks of disk.

Optimised end-to-end for QLC Flash

  • Deep integration from software to QLC NAND solves QLC wear concerns and delivers market-leading economics. Includes the same evergreen maintenance and wear replacement as every FlashArray

“No Compromise” enterprise experience

  • Built for the same 99.9999%+ availability, Pure1 cloud management, API automation, and AI-driven predictive support of every FlashArray

Flash for every data workflow

  • Policy driven replication, snapshots, and migration between arrays and clouds – now use Flash for application tiering, DR, Test / Dev, Backup, and retention

Configuration Details

Configuration options include:

  • 366TB RAW – 1.3PB effective
  • 878TB RAW – 3.2PB effective
  • 1.39PB RAW – 5.2PB effective

Use Cases

  • Policy based VM tiering between //X and //C
  • Multi-cloud data protection and DR – on-premises and multi-site
  • Multi-cloud test / dev – workload consolidation

*File support (NFS / SMB) coming in 2020 (across the entire FlashArray family, not just //C)

 

Thoughts

I’m a fan of companies that expand their portfolio based on customer requests. It’s a good way to make more money, and sometimes it’s simplest to give the people what they want. The market has been in Pure’s ear for some time about delivering some kind of capacity storage solution. I think it was simply a matter of time before the economics and the technology intersected at a point where it made sense for it to happen. If you’re an existing Pure customer, this is a good opportunity to deploy Pure across all of your tiers of storage, and you get the benefit of Pure1 keeping an eye on everything, and your “slow” arrays will still be relatively performance-focused thanks to NVMe throughout the box. Good times in IT isn’t just about speeds and feeds though, so I think this announcement is more important in terms of simplifying the story for existing Pure customers that may be using other vendors to deliver Tier 2 capabilities.

I’m also pretty excited about DirectMemory Cache, if only because it’s clear that Pure has done its homework (i.e. they’ve run the numbers on Pure1 Meta) and realised that they could improve the performance of existing arrays via a reasonably elegant solution. A lot of the cool kids do DAS, because that’s what they’ve been told will yield great performance. And that’s mostly true, but DAS can be a real pain in the rear when you want to move workloads around, or consolidate performance, or do useful things like data services (e.g. replication). Centralised storage arrays have been doing this stuff for years, and it’s about time they were also able to deliver the performance required in order for those companies not to have to compromise.

You can read the press release here, and the Tech Field Day videos can be viewed here.

Random Short Take #21

Here’s a semi-regular listicle of random news items that might be of some interest.

  • This is a great article covering QoS enhancements in Purity 5.3. Speaking of Pure Storage I’m looking forward to attending Pure//Accelerate in Austin in the next few weeks. I’ll be participating in a Storage Field Day Exclusive event as well – you can find more details on that here.
  • My friends at Scale Computing have entered into an OEM agreement with Acronis to add more data protection and DR capabilities to the HC3 platform. You can read more about that here.
  • Commvault just acquired Hedvig for a pretty penny. It will be interesting to see how they bring them into the fold. This article from Max made for interesting reading.
  • DH2i are presenting a webinar on September 10th at 11am Pacific, “On the Road Again – How to Secure Your Network for Remote User Access”. I’ve spoken to the people at DH2i in the past and they’re doing some really interesting stuff. If your timezone lines up with this, check it out.
  • This was some typically insightful coverage of VMworld US from Justin Warren over at Forbes.
  • I caught up with Zerto while I was at VMworld US last week, and they talked to me about their VAIO announcement. Justin Paul did a good job of summarising it here.
  • Speaking of VMworld, William has posted links to the session videos – check it out here.
  • Project Pacific was big news at VMworld, and I really enjoyed this article from Joep.

NetApp, Workloads, and Pizza

Disclaimer: I recently attended VMworld 2019 – US.  My flights and accommodation were paid for by Digital Sense, and VMware provided me with a free pass to the conference and various bits of swag. There is no requirement for me to blog about any of the content presented and I am not compensated by VMware for my time at the event.  Some materials presented were discussed under NDA and don’t form part of my blog posts, but could influence future discussions.

 

As part of my attendance at VMworld US 2019 I had the opportunity to attend Tech Field Day Extra sessions. You can view the videos from the NetApp session here, and download my rough notes from here.

 

Enhanced DC Workloads

In The Beginning There Were Workloads

Andy Banta started his presentation by talking about the evolution of the data centre (DC). The first-generation DCs were resource-constrained. As long as there was something limiting (disk, CPU, memory), things didn’t get done. The later first-generation DCs were comprised of standalone hosts with applications. Andy called “2nd-generation DCs” those hosts that were able to run multiple workloads. The evolution of these 2nd-generation DCs was virtualisation – now you could run multiple applications and operating systems on one host.

The DC though, is still all about compute, memory, throughput, and capacity. As Andy described it, “the DC is full of boxes”.

[image courtesy of NetApp]

 

But There’s Cool Stuff Happening

Things are changing in the DC though, primarily thanks to a few shifts in key technologies that have developed in recent times.

Persistent Memory

Persistent memory has become more mainstream, and application vendors are developing solutions that can leverage this technology effectively. There’s also technology out there that will let you slice this stuff up and share it around, just like you would a pizza. And it’s resilient too, so if you drop your pizza, there’ll be some still left on your plate (or someone else’s plate). Okay I’ll stop with the tortured analogy.

Microvisors

Microvisors are being deployed more commonly in the DC (and particularly at the edge). What’s a microvisor? “Imagine a Hypervisor stripped down to only what you need to run modern Linux based containers”. The advent of the microvisor is leading to different types of workloads (and hardware) popping up in racks where they may not have previously been found.

Specialised Cores on Demand

You can now also access specialised cores on demand from most service providers. You need access to some GPUs to get some particular work done? No problem. There are a bunch of different ways you can slice this stuff up, and everyone’s hip to the possibility that you might only need them for a short time, but you can pay a consumption fee for however long that time will be.

HPC

Even High Performance Compute (HPC) is doing stuff with new technology (in this case NVMeoF). What kinds of workloads?

  • Banking – low-latency transactions
  • Fluid dynamics – lots of data being processed quickly in a parallel stream
  • Medical and nuclear research

 

Thoughts

My favourite quote from Andy was “NVMe is grafting flesh back on to the skeleton of fibre channel”. He (and most of us in the room) are of the belief that FC (in its current incantation at least) is dead. Andy went on to say that “[i]t’s out there for high margin vendors” and “[t]he more you can run on commodity hardware, the better off you are”.

The DC is changing, and not just in the sense that a lot of organisations aren’t running their own DCs any more, but also in the sense that the types of workloads in the DC (and their form factor) are a lot different to those we’re used to running in first-generation DC deployments.

Where does NetApp fit in all of this? The nice thing about having someone like Andy speak on their behalf is that you’re not going to get a product pitch. Andy has been around for a long time, and has seen a lot of different stuff. What he can tell you, though, is that NetApp have started developing (or selling) technology that can accommodate these newer workloads and newer DC deployments. NetApp will be happy to sell you storage that runs over IP, but they can also help you out with compute workloads (in the core and edge), and show you how to run Kubernetes across your estate.

The DC isn’t just full of apps running on hosts accessing storage any more – there’s a lot more to it than that. Workload diversity is becoming more and more common, and it’s going to be really interesting to see where it’s at in ten years from now.

VMware – VMworld 2019 – HBI3516BUS – Scaling Virtual Infrastructure for the Enterprise: Truths, Beliefs and the Real World

Disclaimer: I recently attended VMworld 2019 – US.  My flights and accommodation were paid for by Digital Sense, and VMware provided me with a free pass to the conference and various bits of swag. There is no requirement for me to blog about any of the content presented and I am not compensated by VMware for my time at the event.  Some materials presented were discussed under NDA and don’t form part of my blog posts, but could influence future discussions.

These are my rough notes from “HBI3516BUS – Scaling Virtual Infrastructure for the Enterprise: Truths, Beliefs and the Real World” was a sponsored panel session hosted by George Crump (of Storage Switzerland fame) and sponsored by Tintri by DDN. The panellists were:

JP: Hyper-V is not really for the enterprise. Configuration, and automation were a challenge. Tintri made it easier to deal with the hypervisor.

JD: You put a bunch of disks and connect it up to what you want to. It’s really simple to setup. “Why would you want to go complex if you didn’t have to?”

MB: When we had block storage, we were beholden to the storage team. We’ve never had problems with their [Tintri’s] smallest hybrid arrays.

AA: Back in the ESX 2.5 days – single LUN per VM. We would buy our arrays half-populated – ready to grow. We’re now running 33 – 34 devices. Tintri was great with QoS for VMs. It became a great troubleshooting tool for VMware.

GC: Reporting and analytics with Tintri has always been great.

MB: We use Tintri analytics to create reports for global infrastructure. Tintri will give you per-VM allocation by default. Performance like a Tivo – you can go back and look at analytics at a very granular level.

GC: How did the addition of new arrays go with Global Center?

MB: We manage our purchases based on capacity or projects. 80 – 85% we consider additional capacity. Global Center has a Pools function. It does a storage vMotion “like” feature to move data between arrays. There’s no impact.

JP: We used a UCS chassis, Tintri arrays, and Hyper-V hypervisor. We used a pod architecture. We knew how many users we wanted to host per pod. We have 44000 users globally. VDI is the only thing the bank uses.

AA: We’re more of a compute / core based environment, rather than users.  One of the biggest failings of Tintri is that it just works. When you’re not causing problems – people aren’t paying attention to it.

MB: HCI in general has a problem with very large VMs.

AA: We use a lot of scripting, particularly on the Red Hat (RHV) side of things. Tintri is fixing a lot of those at a different level.

GC: What would you change?

JP: I would run VMware.

MB: The one thing that can go wrong is the network. It was never a standardised network deployment. We had different network people in different regions doing different things.

JP: DR in the cloud. How do you do bank infrastructure in the cloud? Can we DR into the cloud? Tested Tintri replicating into Azure.

AA: We’re taking on different people. Moving “up” the stack.

Consistency in environments. It’s still a hard thing to do.

Wishlist?

  • Containers
  • A Virtual Appliance

 

Thoughts

Some folks get upset about these sponsored sessions at VMworld. I’ve heard it said before that they’re nothing more than glorified advertising for the company that sponsors the session. I’m not sure that it’s really any different to a vendor holding a four day conference devoted to themselves, but some people like to get ornery about stuff like that. One of my favourite things about working with technology is hearing from people out in the field about how they use that technology to do their jobs better / faster / more efficiently.

Sure, this session was a bit of a Tintri fan panel, but I think the praise is warranted. I’ve written enthusiastically in the past about how I thought Tintri has really done some cool stuff in terms of storage for virtualisation. I was sad when things went south for them as a company, but I have hopes that they’ll recover and continue to innovate under the control of DDN.

When everything I’ve been hearing from the keynote speakers at this conference revolved around cloud-native tools and digital transformation, it was interesting to come across a session where the main challenges still involved getting consistent, reliable and resilient performance from block storage to serve virtual desktop workloads to the enterprise. That’s not to say that we shouldn’t be looking at what’s happening with Kubernetes, etc, but I think there’s still room to understand what’s making these bigger organisations tick in terms of successful storage infrastructure deployments.

Useful session. 4 stars.

VMware – VMworld 2019 – HBI2537PU – Cloud Provider CXO Panel with Cohesity, Cloudian and PhoenixNAP

Disclaimer: I recently attended VMworld 2019 – US.  My flights and accommodation were paid for by Digital Sense, and VMware provided me with a free pass to the conference and various bits of swag. There is no requirement for me to blog about any of the content presented and I am not compensated by VMware for my time at the event.  Some materials presented were discussed under NDA and don’t form part of my blog posts, but could influence future discussions.

Here are my rough notes from “HBI2537PU – Cloud Provider CXO Panel with Cohesity, Cloudian and PhoenixNAP”, a panel-type presentation with the following people:

You can grab a PDF copy of my notes from here.

Introductions are done.

YR: William, given your breadth of experience, what are some of the emerging trends you’ve been seeing?

WB: Companies are struggling to keep up with the pace of information generation. Understanding the data, storing and retaining it, and protecting it. Multi-cloud adds a lot of complexity. We’ve heard studies that say 22% of data generated is actually usable. It’s just sitting there. Public cloud is still hot, but it’s settling down a little.

YR: William comes from a massive cloud provider. What are you guys using?

WB: We’ve standardised on vCloud Director (vCD) and vSphere. We came from build our own but it wasn’t providing the value that we hoped it would. Customers want a seamless way to manage multiple cloud resources.

YR: Are you guys familiar with VCPP?

AP: VCPP is the crown jewel of our partner program at VMware. 4000+ providers, 120+ countries, 10+ million VMs, 10000+ DCs. We help you save money, make money (things are services ready). We’re continuing to invest in vCD. Kubernetes, GPUs, etc. Lots of R&D.

YR: William, you mentioned you standardised on the VMware platform. Talk to us about your experience. Why vCD?

WB: It’s been a checkered past for vCD. We were one of the first five on the vCloud Express program in 2010 / 11. We didn’t like vCD in its 1.0 version. We thought we can do this better. And we did. We launched the first on-demand, pay by the hour public cloud for enterprise in 2011. But it didn’t really work out. 2012 / 13 we started to see investments being made in vCD. 5.0 / 5.5 improved. Many people thought vCD was gong to die. We now see a modern, flexible portal that can be customised. And we can take our devs and have them customise vCD, rather than build a customised portal. That’s where we can put our time and effort. We’ve always done things differently. Always been doing other things. How do we bring our work in visual cloud into that cloud provider portal with vCD?

YR: You have an extensive career at VMware.

RR: I was one of the first people to take vCD out to the world. But Enterprise wasn’t mature enough. When we focused on SPs, it was the right thing to do. DIY portals needs a lot of investment. VMware allows a lot of extensibility now. For us, as Cohesity, we want to be able to plug in to that as well.

WB: At one point we had 45 devs working on a proprietary portal.

YR: We’ve been doing a lot on the extensibility side. What role are services playing in cloud providers?

AP: It takes away the complexities of deploying the stack.

JT: We’re specifically in object. A third of our customers are service providers. You guys know that object is built for scale, easy to manage, cost-effective. 20% of the data gets used. We hear that customers want to improve on that. People are moving away from tape. There’s a tremendous opportunity for services built on storage. Amazon has shown that. Data protection like Cohesity. Big data with Splunk. You can offer an industry standard, but differentiate based on other services.

YR: As we move towards a services-oriented world, William how do you see cloud management services evolving?

WB: It’s not good enough to provide some compute infrastructure any more. You have to do something more. We’re stubbornly focussed on different types of IaaS. We’re not doing generic x86 on top of vSphere. Backup, DR – those are in our wheelhouse. From a platform perspective, more and more customers want some kind of single pane of glass across their data. For some that’s on-premises, for some its public, for some it’s SaaS. You have to be able to provide value to the customer, or they will disappear. Object storage, backup with Cohesity. You need to keep pace with data movement. Any cloud, any data, any where.

AP: I’ve been at VMware long enough not to drink the Kool-Aid. Our whole cloud provider business is rooted in some humility. vCD can help other people doing better things to integrate. vCD has always been about reducing OPEX. Now we’re hitting the top line. Any cloud management platform today needs to open, extensible, not try to do anything.

YR: Is the crowd seeing pressure on pure IaaS?

Commentator: Coming from an SP to enterprise is different. Economics. Are you able to do a show back with vCD 9 and vROps?

WB: We’re putting that in the hands of customers. Looking at CloudHealth. There’s a benefit to being in the business management space. You have the opportunity to give customers a better service. That, and more flexible business models. Moving into flexible billing models – gives more freedom to the enterprise customer. Unless you’re the largest of the large – enterprises have difficulty acting as a service provider. Citibank are an exception to this. Honeywell do it too. If you’re Discount Tire – it’s hard. You’re the guy providing the service, and you’re costing them money. There’s animosity – and there’s no choice.

Commentator: Other people have pushed to public because chargeback is more effective than internal show back with private cloud.

WB: IT departments are poorly equipped to offer a breadth of services to their customers.

JT: People are moving workloads around. They want choice and flexibility. VMware with S3 compatible storage. A common underlying layer.

YR: Economics, chargeback. Is VMware (and VCPP) doing enough?

WB: The two guys to my right (RR and JT) have committed to building products that let me do that. I’ve been working on object storage use cases. I was talking to a customer. They’re using our IaaS and connected to Amazon S3. You’ve gone to Amazon. They didn’t know about it though. Experience and cost that can be the same or better. Egress in Amazon S3 is ridiculous. You don’t know what you don’t know. You can take that service and deliver it cost-effectively.

YR: RR talk to us about the evolution of data protection.

RR: Information has grown. Data is fragmented. Information placement is almost unmanageable. Services have now become available in a way that can be audited, secured, managed. At Cohesity, first thing we did was data protection, and I knew the rest was coming. Complexity’s a problem.

YR: JT. We know Cloudian’s a leader in object storage. Where do you see object going?

JT: It’s the underlying storage layer of the cloud. Brings down cost of your storage layer. It’s all about TCO. What’s going to help you build more revenue streams? Cloudian has been around since 2011. New solutions in backup, DR, etc, to help you build new revenue streams. S3 users on Amazon are looking for alternatives. Many of Cloudian’s customers are ex-Amazon customers. What are we doing? vCD integration. Search Cloudian and vCD on YouTube. Continuously working to drive down the cost of managing storage. 1.5PB in a 4RU box in collaboration with Seagate.

WB: Expanding service delivery, specifically around object storage, is important. You can do some really cool stuff – not just backup, it’s M&E, it’s analytics. Very few of our customers are using object just to store files and folders.

YR: We have a lot of providers in the room. JT can you talk more about these key use cases?

JT: It runs the gamut. You can break it down by verticals. M&E companies are offering editing suites via service providers. People are doing that for the legal profession. Accounting – storing financial records. Dental records and health care. The back end is the same thing – compute with S3 storage behind it. Cloudian provides multi-tenanted, scalable performance. Cost is driven down as you get larger.

YR: RR your key use cases?

RR: DRaaS is hot right now. When I was at VMware we did stuff with SRM. DR is hard. It’s so simple now. Now every SP can do it themselves. Use S3 to move data around from the same interface. And it’s very needed too. Everyone should have ubiquitous access to their data. We have that capability. We can now do vulnerability scans on the data we store on the platform. We can tell you if a VM is compromised. You can orchestrate the restoration of an environment – as a service.

YR: WB what are the other services you want us to deliver?

WB: We’re an odd duck. One of our major practices is information security. The idea that we have intelligent access to data residing in our infrastructure. Being able to detect vulnerabilities, taking action, sending an email to the customer, that’s the type of thing that cloud providers have. You might not be doing it yet – but you could.

YR: Security, threat protection. RR – do you see Cohesity as the driver to solve that problem?

RR: Cohesity will provide the platform. Data is insecure because it’s fragmented. Cohesity lets you run applications on the platform. Virus scanners, run books, all kinds of stuff you can offer as a service provider.

YR: William, where does the onus lie, how do you see it fitting together?

WB: The key for us is being open. Eg Cohesity integration into vCD. If I don’t want to – I don’t have to. Freedom of choice to pick and choose where we went to deliver our own IP to the customer. I don’t have to use Cohesity for everything.

JT: That’s exactly what we’re into. Choice of hardware, management. That’s the point. Standards-based top end.

YR: Security

*They had 2 minutes to go but I ran out of time and had to get to another meeting. Informative session. 4 stars.

Pure Storage – Configuring ObjectEngine Bucket Security

This is a quick post as a reminder for me next time I need to do something with basic S3 bucket security. A little while I ago I was testing Pure Storage’s ObjectEngine (OE) device with a number of data protection products. I’ve done a few articles previously on what it looked like from the Cohesity and Commvault perspective, but thought it would be worthwhile to document what I did on the OE side of things.

The first step is to create the bucket in the OE dashboard.

You’ll need to call it something, and there are rules around the naming convention and length of the name.

In this example, I’m creating a bucket for Commvault to use, so I’ve called this one “commvault-test”.

Once the bucket has been created, you should add a security policy to the bucket.

Click on “Add” and you’ll be prompted to get started with the Bucket Policy Editor.

I’m pretty hopeless with this stuff, but fortunately there’s a policy generator on the AWS site you can use.

Once you’ve generated your policy, click on Save and you’ll be good to go. Keep in mind that any user you reference in the policy will need to exist in OE for the policy to work.

Here’s the policy I applied to this particular bucket. The user is commvault, and the bucket name is commvault-test.

{
  "Id": "Policy1563859773493",
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Stmt1563859751962",
      "Action": "s3:*",
      "Effect": "Allow",
      "Resource": "arn:aws:s3:::commvault-test",
      "Principal": {
        "AWS": [
          "arn:aws:iam::0:user/commvault"
        ]
      }
    },
    {
      "Sid": "Stmt1563859771357",
      "Action": "s3:*",
      "Effect": "Allow",
      "Resource": "arn:aws:s3:::commvault-test/*",
      "Principal": {
        "AWS": [
          "arn:aws:iam::0:user/commvault"
        ]
      }
    }
  ]
}

You can read more about the policy elements here.