Pure Storage – Configuring ObjectEngine Bucket Security

This is a quick post as a reminder for me next time I need to do something with basic S3 bucket security. A little while I ago I was testing Pure Storage’s ObjectEngine (OE) device with a number of data protection products. I’ve done a few articles previously on what it looked like from the Cohesity and Commvault perspective, but thought it would be worthwhile to document what I did on the OE side of things.

The first step is to create the bucket in the OE dashboard.

You’ll need to call it something, and there are rules around the naming convention and length of the name.

In this example, I’m creating a bucket for Commvault to use, so I’ve called this one “commvault-test”.

Once the bucket has been created, you should add a security policy to the bucket.

Click on “Add” and you’ll be prompted to get started with the Bucket Policy Editor.

I’m pretty hopeless with this stuff, but fortunately there’s a policy generator on the AWS site you can use.

Once you’ve generated your policy, click on Save and you’ll be good to go. Keep in mind that any user you reference in the policy will need to exist in OE for the policy to work.

Here’s the policy I applied to this particular bucket. The user is commvault, and the bucket name is commvault-test.

{
  "Id": "Policy1563859773493",
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Stmt1563859751962",
      "Action": "s3:*",
      "Effect": "Allow",
      "Resource": "arn:aws:s3:::commvault-test",
      "Principal": {
        "AWS": [
          "arn:aws:iam::0:user/commvault"
        ]
      }
    },
    {
      "Sid": "Stmt1563859771357",
      "Action": "s3:*",
      "Effect": "Allow",
      "Resource": "arn:aws:s3:::commvault-test/*",
      "Principal": {
        "AWS": [
          "arn:aws:iam::0:user/commvault"
        ]
      }
    }
  ]
}

You can read more about the policy elements here.

Formulus Black Announces Forsa 3.0

Formulus Black recently announced version 3.0 of its Forsa product. I had the opportunity to speak with Mark Iwanowski and Jing Xie about the announcement and wanted to share some thoughts here.

 

So What’s A Forsa Again?

It’s a software solution for running applications in memory without needing to re-tool your applications or hardware. You can present persistent storage (think Intel Optane) or non-persistent memory (think DRAM) as a block device to the host and run your applications on that. Here’s a look at the architecture.

[image courtesy of Formulus Black]

Is This Just a Linux Thing?

No, not entirely. There’s Ubuntu and CentOS support out of the box, and Red Hat support is imminent. If you don’t use those operating systems though, don’t stress. You can also run this using a KVM-based hypervisor. So anything supported by that can be supported by Forsa.

But What If My Memory Fails?

Formulus Black has a technology called “BLINK” which provides the ability to copy your data down to SSDs, or you can failover the data to another host.

Won’t I Need A Bunch Of RAM?

Formulus Black uses Bit Markers – a memory efficient technology (like deduplication) – to make efficient use of the available memory. They call it “amplification” as opposed to deduplication, as it amplifies the available space.

Is This Going To Cost Me?

A little, but not as much as you’d think (because nothing’s ever free). The software is licensed on a per-socket basis, so if you decide to add memory capacity you’re not up for additional licensing costs.

 

Thoughts and Further Reading

I don’t do as much work with folks requiring in-memory storage solutions as much as I’d like to do, but I do appreciate the requirement for these kinds of solutions. The big appeal here is the lack of requirement to re-tool your applications to work in-memory. All you need is something that runs on Linux or KVM and you’re pretty much good to go. Sure, I’m over-simplifying things a little, but it looks like there’s a good story here in terms of the lack of integration required to get some serious performance improvements.

Formulus Black came out of stealth around 4 and a bit months ago and have already introduced a raft of improvements over version 2.0 of their offering. It’s great to see the speed with which they’ve been able to execute on new features in their offering. I’m curious to see what’s next, as there’s obviously been a great focus on performance and simplicity.

The cool kids are all talking about the benefits of NVMe-based, centralised storage solutions. And they’re right to do this, as most applications will do just fine with these kinds of storage platforms. But there are still going to be minuscule bottlenecks associated with these devices. If you absolutely need things to run screamingly fast, you’ll likely want to run them in-memory. And if that’s the case, Formulus Black’s Forsa solution might be just what you’re looking for. Plus, it’s a pretty cool name for a company, or possibly an aspiring wizard.

Random Short Take #20

Here are some links to some random news items and other content that I recently found interesting. You might find them interesting too. Episode 20 – feels like it’s becoming a thing.

  • Scale Computing seems to be having a fair bit of success with their VDI solutions. Here’s a press release about what they did with Harlingen WaterWorks System.
  • I don’t read Corey Quinn’s articles enough, but I am glad I read this one. Regardless of what you think about the enforceability of non-compete agreements (and regardless of where you’re employed), these things have no place in the modern workforce.
  • If you’re getting along to VMworld US this year, I imagine there’s plenty in your schedule already. If you have the time – I recommend getting around to seeing what Cody and Pure Storage are up to. I find Cody to be a great presenter, and Pure have been doing some neat stuff lately.
  • Speaking of VMworld, this article from Tom about packing the little things for conferences in preparation for any eventuality was useful. And if you’re heading to VMworld, be sure to swing past the VMUG booth. There’s a bunch of VMUG stuff happening at VMworld – you can read more about that here.
  • I promise this is pretty much the last bit of news I’ll share regarding VMworld. Anthony from Veeam put up a post about their competition to win a pass to VMworld. If you’re on the fence about going, check it out now (as the competition closes on the 19th August).
  • It wouldn’t be a random short take without some mention of data protection. This article about tiering protection data from George Crump was bang on the money.
  • Backblaze published their quarterly roundup of hard drive stats – you can read more here.
  • This article from Paul on freelancing and side gigs was comprehensive and enlightening. If you’re thinking of taking on some extra work in the hopes of making it your full-time job, or just wanting to earn a little more pin money, it’s worthwhile reading this post.

Burlywood Tech Announces TrueFlash Insight

Burlywood Tech came out of stealth a few years ago, and I wrote about their TrueFlash announcement here. I had another opportunity to speak to Mike Tomky recently about Burlywood’s TrueFlash Insight announcement and thought I’d share some thoughts here.

 

The Announcement

Burlywood’s “TrueFlash” product delivers what they describe as a “software-defined SSD” drive. Since they’ve been active in the market they’ve gained traction in what they call the Tier 2 service provider segments (not the necessarily the “Big 7” hyperscalers).

They’ve announced TrueFlash Insight because, in a number of cases, customers don’t know what their workloads really look like. The idea behind TrueFlash Insight is that it can be run in a production environment for a period of time to collect metadata and drive telemetry. Engineers can also be sent on site if required to do the analysis. The data collected with TrueFlash Insight helps Burlywood with the process of designing and tuning the TrueFlash product for the desired workload.

How It Works

  • Insight is available only on Burlywood TrueFlash drives
  • Enabled upon execution of a SOW for Insight analysis services
  • Run your application as normal in a system with one or more Insight-enabled TrueFlash drives
  • Follow the instructions to download the telemetry files
  • Send telemetry data to Burlywood for analysis
  • Burlywood parses the telemetry, analyses data patterns, shares performance information, and identifies potential bottlenecks and trouble spots
  • This information can then be used to tune the TrueFlash SSDs for optimal performance

 

Thoughts and Further Reading

When I wrote about Burlywood previously I was fascinated by the scale that would be required for a company to consider deploying SSDs with workload-specific code sitting on them. And then I stopped and thought about my comrades in the enterprise space struggling to get the kind of visibility into their gear that’s required to make these kinds of decisions. But when your business relies so heavily on good performance, there’s a chance you have some idea of how to get information on the performance of your systems. The fact that Burlywood are making this offering available to customers indicates that even those customers that are on board with the idea of “Software-defined SSDs (SDSSDs?)” don’t always have the capabilities required to make an accurate assessment of their workloads.

But this solution isn’t just for existing Burlywood customers. The good news is it’s also available for customers considering using Burlywood’s product in their DC. It’s a reasonably simple process to get up and running, and my impression is that it will save a bit of angst down the track. Tomky made the comment that, with this kind of solution, you don’t need to “worry about masking problems at the drive level – [you can] work on your core value”. There’s a lot to be said for companies, even the ones with very complex technical requirements, not having to worry about the technical part of the business as much as the business part of the business. If Burlywood can make that process easier for current and future customers, I’m all for it.

StorONE Announces S1-as-a-Service

StorONE recently announced its StorONE-as-a-Service (S1aaS) offering. I had the opportunity to speak to Gal Naor about it and thought I’d share some thoughts here.

 

The Announcement

StorONE’s S1-as-a-Service (S1aaS), is a use-based solution integrating StorONE’s S1 storage services with Dell Technologies and Mellanox hardware. The idea is they’ll ship you an appliance (available in a few different configurations) and you plug it in and away you go. There’s not a huge amount to say about it as it’s fairly straightforward. If you need more that the 18TB entry-level configuration, StorONE can get you up and running with 60TB thanks to overnight shipping.

Speedonomics

The as-a-Service bit is what most people are interested in, and S1aaS starts at $999 US per month for the 18TB all-flash array that delivers up to 150000 IOPS. There are a couple of other configurations available as well, including 36TB at $1797 per month, and 54TB at $2497 per month. If, for some reason, you decide you don’t want the device any more, or you no longer have that particular requirement, you can cancel your service with 30 days’ notice.

 

Thoughts and Further Reading

The idea of consuming storage from vendors on-premises via flexible finance plans isn’t a new one. But S1aaS isn’t a leasing plan. There’s no 60-month commitment and payback plan. If you want to use this for three months for a particular project and then cancel your service, you can. Just as you could with cable. From that perspective, it’s a reasonably interesting proposition. A number of the major storage vendors would struggle to put that much capacity and speed in such a small footprint on-premises for $999 per month. This is the major benefit of a software-based storage product that, by all accounts, can get a lot out of commodity server hardware.

I wrote about StorONE when they came out of stealth mode a few years ago, and noted the impressive numbers they were posting. Are numbers the most important thing when it comes to selecting storage products? No, not always. There’s plenty to be said for “good enough” solutions that are more affordable. But it strikes me that solutions that go really fast and don’t cost a small fortune to run are going to be awfully compelling. One of the biggest impediments to deploying on-premises storage solutions “as-a-Service” is that there’s usually a minimum spend required to make it worthwhile for the vendor or service provider. Most attempts previously have taken more than 2RU of rack space as a minimum footprint, and have required the customer to sign up for minimum terms of 36 – 60 months. That all changes (for the better) when you can run your storage on a server with NVMe-based drives and an efficient, software-based platform.

Sure, there are plenty of enterprises that are going to need more than 18TB of capacity. But are they going to need more than 54TB of capacity that goes at that speed? And can they build that themselves for the monthly cost that StorONE is asking for? Maybe. But maybe it’s just as easy for them to look at what their workloads are doing and decide whether they want everything to on that one solution. And there’s nothing to stop them deploying multiple configurations either.

I was impressed with StorONE when they first launched. They seem to have a knack for getting good performance from commodity gear, and they’re willing to offer that solution to customers at a reasonable price. I’m looking forward to seeing how the market reacts to these kinds of competitive offerings. You can read more about S1aaS here.

Brisbane VMUG – August 2019

hero_vmug_express_2011

The August edition of the Brisbane VMUG meeting will be held on Tuesday 20th August at Fishburners from 4 – 6pm. It’s sponsored by Dell EMC and should to be a great afternoon.

Here’s the agenda:

  • VMUG Intro
  • VMware Presentation: TBA
  • Dell EMC Presentation: Protecting Your Critical Assets With Dell EMC
  • Q&A
  • Refreshments and drinks.

Dell EMC have gone to great lengths to make sure this will be a fun and informative session and I’m really looking forward to hearing about their data protection portfolio. You can find out more information and register for the event here. I hope to see you there. Also, if you’re interested in sponsoring one of these events, please get in touch with me and I can help make it happen.

Random Short Take #18

Here are some links to some random news items and other content that I recently found interesting. You might find them interesting too. Episode 18 – buckle up kids! It’s all happening.

  • Cohesity added support for Active Directory protection with version 6.3 of the DataPlatform. Matt covered it pretty comprehensively here.
  • Speaking of Cohesity, Alastair wrote this article on getting started with the Cohesity PowerShell Module.
  • In keeping with the data protection theme (hey, it’s what I’m into), here’s a great article from W. Curtis Preston on SaaS data protection, and what you need to consider to not become another cautionary tale on the Internet. Curtis has written a lot about data protection over the years, and you could do a lot worse than reading what he has to say. And that’s not just because he signed a book for me.
  • Did you ever stop and think just how insecure some of the things that you put your money into are? It’s a little scary. Shell are doing some stuff with Cybera to improve things. Read more about that here.
  • I used to work with Vincent, and he’s a super smart guy. I’ve been at him for years to start blogging, and he’s started to put out some articles. He’s very good at taking complex topics and distilling them down to something that’s easy to understand. Here’s his summary of VMware vRealize Automation configuration.
  • Tom’s take on some recent CloudFlare outages makes for good reading.
  • Google Cloud has announced it’s acquiring Elastifile. That part of the business doesn’t seem to be as brutal as the broader Alphabet group when it comes to acquiring and discarding companies, and I’m hoping that the good folks at Elastifile are looked after. You can read more on that here.
  • A lot of people are getting upset with terms like “disaggregated HCI”. Chris Mellor does a bang up job explaining the differences between the various architectures here. It’s my belief that there’s a place for all of this, and assuming that one architecture will suit every situation is a little naive. But what do I know?

NetApp Wants You To See The Whole Picture

Disclaimer: I recently attended Tech Field Day 19.  My flights, accommodation and other expenses were paid for by Tech Field Day. There is no requirement for me to blog about any of the content presented and I am not compensated in any way for my time at the event.  Some materials presented were discussed under NDA and don’t form part of my blog posts, but could influence future discussions.

NetApp recently presented at Tech Field Day 19. You can see videos of their presentation here, and download my rough notes from here.

 

Management or Monitoring?

James Holden (Director, Cloud Analytics) delivered what I think was a great presentation on NetApp Cloud Insights. Early on he made the comment that “[w]e’re as read-only as we possibly can be. Being actionable puts you in a conversation where you’re doing something with the infrastructure that may not be appropriate.” It’s a comment that resonated with me, particularly as I’ve been on both sides of the infrastructure management and monitoring fence (yes, I know, it sounds like a weird fence – just go with it). I remember vividly providing feedback to vendors that I wanted their fancy single pane of glass monitoring solution to give me more management capabilities as well. And while they were at it, it would be great if they could develop software that would automagically fix issues in my environment as they arose.

But do you want your cloud monitoring tools to really have that much control over your environment? Sure, there’s a lot of benefit to be had deploying solutions that can reduce the stick time required to keep things running smoothly, but I also like the idea that the software won’t just dive in a fix what it perceives as errors in an environment based on a bunch of pre-canned constraints that have been developed by people that may or may not always have a good grip on what’s really happening in these types of environments.

Keep Your Cloud Happy

So what can you do with Cloud Insights? As it turns out, all kinds of stuff, including cost optimisation. It doesn’t always sound that cool, but customers are frequently concerned with the cost of their cloud investment. What they get with Cloud Insights is:

Understanding

  • What’s my last few months cost?
  • What’s my current month running cost
  • Cost broken down by AWS service, account, region?
  • Does it meet the budget?

Analysis

  • Real time cost analysis to alert on sudden rise in cost
  • Project cost over period of time

Optimisation

  • Save costs by using “reserved instances”
  • Right sizing compute resources
  • Remove waste: idle EC2 instances, unattached EBS volumes, unused reserved instances
  • Spot instance use

There are a heap of other features, including:

  • Alerting and impact analysis; and
  • Forensic analysis.

It’s all wrapped up in an alarmingly simple SaaS solution meaning quick deployment and faster time to value.

The Full Picture

One of my favourite bits of the solution though is that NetApp are striving to give you access to the full picture:

  • There are application services running in the environment; and
  • There are operating systems and hardware underneath.

“The world is not just VMs on compute with backend storage”, and NetApp have worked hard to ensure that the likes of micro services are also supported.

 

Thoughts and Further Reading

One of the recurring themes of Tech Field Day 19 was that of management and monitoring. When you really dig into the subject, every vendor has a different take on what can be achieved through software. And it’s clear that every customer also has an opinion on what they want to achieve with their monitoring and management solutions. Some folks are quite keen for their monitoring solutions to take action as events arise to resolve infrastructure issues. Some people just want to be alerted about the problem and have a human intervene. And some enterprises just want an easy way to report to their C-level what they’re spending their money on. With all of these competing requirements, it’s easy to see how I’ve ended up working in enterprises running 10 different solutions to monitor infrastructure. They also had little idea what the money was being spent on, and had a large team of operations staff dealing with issues that weren’t always reported by the tools, or they got buried in someone’s inbox.

IT operations has been a hard nut to crack for a long time, and it’s not always the fault of the software vendors that it isn’t improving. It’s not just about generating tonnes of messages that no-one will read. It’s about doing something with the data that people can derive value from. That said, I think NetApp’s solution is a solid attempt at providing a useful platform to deliver on some pretty important requirements for the modern enterprise. I really like the holistic view they’ve taken when it comes to monitoring all aspects of the infrastructure, and the insights they can deliver should prove invaluable to organisations struggling with the myriad of moving parts that make up their (private and public) cloud footprint. If you’d like to know more, you can access the data sheet here, and the documentation is hosted here.

Updated Articles Page

I recently had the opportunity to deploy a Nasuni NMC and filer in the lab and thought I’d run through the basics. There’s a new document outlining the process on the articles page.

Spectra Logic – BlackPearl Overview

I recently had the opportunity to take a briefing with Jeff Braunstein and Susan Merriman from Spectra Logic (one of those rare occasions where getting your badge scanned at a conference proves valuable), and thought I’d share some of my notes here.

 

BlackPearl Family

Spectra Logic sell a variety of products, but this briefing was focused primarily on the BlackPearl series. Braunstein described it as a “gateway” device, with both NAS and object front end interfaces, and backend capability that can move data to multiple types of archives.

[image courtesy of Spectra Logic]

It’s a hardware box, but at its core the value is in the software product. The idea is that the BlackPearl acts as a disk cache, and you configure policies to send the data to one or more storage targets. The cool thing is that it supports multiple retention policies, and these can be permanent too. By that I mean you could spool one copy to tape for long term storage, and have another copy of your data sit on disk for 90 days (or however long you wanted).

 

Local vs Remote Storage

Local

There are a few different options for local storage, including BlackPearl Object Storage Disk, functioning as “near line archive”. This is configured with 107 enterprise quality SATA drives, (and they’re looking at introducing 16TB drives next month), providing roughly 1.8PB RAW capacity. They function as power-down archive drives (using the drive spin down settings), and delivers a level of resilience and reliability by using ZFS as the file system,. There are also customer-configurable parity settings. Alternatively, you can pump data to Spectra Tape Libraries, for those of you who still want to use tape as a storage format.

 

Remote Storage Targets

In terms of remote storage targets, BlackPearl can leverage either public cloud, or other BlackPearl devices as replication targets. Replication to BlackPearl can be one way or bi-directional. Public Cloud support is available via Amazon S3 (and S3-like products such as Cloudian and Wasabi), and MS Azure. There is a concept of data immutability in the product, and you can turn on versioning to prevent your data management applications (or users) from accidentally clobbering your data.

Braunstein also pointed out that tape generations evolve, and BlackPearl has auto-migration capabilities. You can potentially have data migrate transparently from tape to tape (think LTO-6 to LTO-7), tape to disk, and tape to cloud.

 

[image courtesy of Spectra Logic]

In terms of how you leverage BlackPearl, some of that is dependent on the workflows you have in place to move your data. This could be manual, semi-automated, or automated (or potentially purpose built into existing applications). There’s a Spectra S3 RESTful API, and there’s heaps of information on developer.spectralogic.com on how to integrate BlackPearl into your existing applications and media workflows.

 

Thoughts

If you’re listening to the next-generation data protection vendors and big box storage folks, you’d wonder why companies such as Spectra Logic still focus on tape. It’s not because they have a rich heritage and deep experience in the tape market (although they do). There are plenty of use cases where tape still makes sense in terms of its ability to economically store large amounts of data in a relatively secure (off-line if required) fashion. Walk into any reasonably sized film production house and you’ll still see tape in play. From a density perspective (and durability), there’s a lot to like about tape. But BlackPearl is also pretty adept at getting data from workflows that were traditionally file-based and putting them on public cloud environments (the kind of environments that heavily leverage object storage interfaces). Sure, you can pump the data up to AWS yourself if you’re so inclined, but the real benefit of the BlackPearl approach, in my opinion, is that it’s policy-driven and fully automated. There’s less chance that you’ll fat finger the transfer of critical data to another location. This gives you the ability to focus on your core business, and not have to worry about data management.

I’ve barely scratched the surface of what BlackPearl can do, and I recommend checking out their product site for more information.