Qumulo – Storage Your Way

Disclaimer: I recently attended Storage Field Day 20.  My flights, accommodation and other expenses were paid for by Tech Field Day. There is no requirement for me to blog about any of the content presented and I am not compensated in any way for my time at the event.  Some materials presented were discussed under NDA and don’t form part of my blog posts, but could influence future discussions.

Qumulo recently presented at Storage Field Day 20. You can see videos of the presentation here, and download my rough notes from here.

 

Solving Problems The Qumulo Way

Extreme, Efficient Scalability

Legacy problem

  • Inefficient file system architectures only use 60-80% of purchased capacity
  • Limited capacity and file count scalability
  • Adding capacity and performance complex and often required downtime

Qumulo Solution

  • Get 100% out of your investment
  • Scale to hundreds of billions of files
  • Expand capacity and performance seamlessly

No More Tiers

Legacy problem

  • Data not available in fastest storage when users and applications request it
  • Tiering jobs were slow to execute and often didn’t complete, leaving risk of filling the performance tier

Qumulo Solution

  • Single-tier solution with predictive caching ensures performance SLAs are met
  • Simplified administration and full confidence in access to data

Visibility

Legacy problem

  • Troubleshooting complexity, unforeseen growth expenditures, and usage bottlenecks plague IT
  • Impossible to track who is using data and for which projects

Qumulo Solution

  • Gain graphical, real-time visibility into user performance, usage trends and performance bottlenecks
  • Proactively plan for future requirements

 

Flash! Ah-ah

The development of an all-NVMe solution also means Qumulo can do even more to address legacy storage problems.

Intelligent Caching

  • Enables fast random reads from limited and expensive media (SSD)
  • Identifies read I/O patterns and promotes data from disk to SSD
  • Cache eviction policy leverages heat-based strategy

Predictive Prefetch

  • Enables very fast streaming reads
  • Proactively moves data to RAM by anticipating files that will likely be read in large, parallel batches or within a file
  • Constantly automatically adjusting if data is not used at per client granularity

All Writes Go To Flash

  • No special hardware components required
  • Data de-staged to optimise for HDD performance
  • Low latency NVMe makes writes much faster

 

Thoughts and Further Reading

We’ve been talking about the end of disk-based storage systems for some time now. But there still seems to be an awful lot of spinning drives being used around the globe to power a variety of different workloads. Hybrid storage still has a place in storage world, particularly when you need “good enough” performance in price-sensitive environments. What All-Flash does, however, is provide the ability to deliver some very, very good performance for those applications that need it. Doing high resolution video editing or need to render those files at high speed? An all-NVMe solution is likely what you’re after. But if you just need a bunch of capacity to store video surveillance files, or archives, then a hybrid solution will quite likely meet your requirements. The key to the Qumulo solution is that it can do both of those things whilst using a bunch of software smarts to help you get your unstructured data under control. It’s not just about throwing a new storage protocol at the problem and hoping things will run better / faster though. It’s also important to understand how the solution can scale out, and what kind of visibility you get with said solution. These are two critical aspects of storage solutions used in media production environments, particularly when being able to squeeze every last bit of performance out of the system is a key requirement, and you might be in a position where you need to throw a bunch of workload at the system in a hurry.

Qumulo strikes me as being a super popular solution for video editing, broadcast, production, and so forth. This is one of my favourite market segments, if only because the variety of workloads and solutions that cater to those workloads is pretty insane. That said, when you dig into other market segments, such as Artificial Intelligence and analytics workloads, you’ll also notice that unstructured data access is a common requirement. The delivery of an all-NVMe solution helps Qumulo provide the resources required to satisfy those high-performance applications. But the cool thing isn’t just the performance, or even the ability to scale. It’s the visibility you can get into the platform to work out what’s going on. Managing petabytes of unstructured data is a daunting task at the best of times, so it’s nice to see a company paying attention to making both the end user and the storage administrator happy. I’m the first to admit I haven’t been paying as much attention to Qumulo as I should have, but I will be doing so from now on. For another perspective, check out Ray Lucchesi’s article on Qumulo here.

Storage Field Day 8 – Wrap-up and Link-o-rama

Disclaimer: I recently attended Storage Field Day 8.  My flights, accommodation and other expenses were paid for by Tech Field Day. There is no requirement for me to blog about any of the content presented and I am not compensated in any way for my time at the event.  Some materials presented were discussed under NDA and don’t form part of my blog posts, but could influence future discussions.

This is a quick post to say thanks once again to Stephen, Claire and the presenters at Storage Field Day 8. I had a fantastic time and learnt a lot. For easy reference, here’s a list of the posts I did covering the event (not necessarily in chronological order).

Storage Field Day – I’ll be at SFD8

Storage Field Day 8 – Day 0

Storage Field Day 8 – (Fairly) Full Disclosure

Cohesity – There’s more to this than just “Secondary Storage”

Violin Memory – Sounds a lot better than it used to

Pure Storage – Orange is the new black, now what?

INFINIDAT – What exactly is a “Moshe v3.0”?

Nimble Storage – InfoSight is still awesome

Primary Data – Because we all want our storage to do well

NexGen Storage – Storage QoS and other good things

Coho Data – Numerology, but not as we know it

Intel – They really are “Inside”

Qumulo – Storage for people who care about their data

 

Also, here’s a number of links to posts by my fellow delegates. They’re all switched-on people, and you’d do well to check out what they’re writing about. I’ll try and update this list as more posts are published. But if it gets stale, the SFD8 landing page has updated links.

 

Ray Lucchesi (@RayLucchesi)

Coho Data, the packet processing squeeze and working set exploits

Primary data’s path to better data storage presented at SFD8

PB are the new TB, GreyBeards talk with Brian Carmody, CTO Inifinidat

 

Mark May (@CincyStorage)

Can Violin Step back From the Brink?

Storage software can change enterprise workflow

Redefining Secondary Storage

 

Scott D. Lowe (@OtherScottLowe)

IT as a Change Agent: It’s Time to Look Inward, Starting with Storage

Overcoming “New Vendor Risk”: Pure Storage’s Techniques

So, What is Secondary Storage Cohesity-Style?

Data Awareness Is Increasingly Popular in the Storage Biz

 

Jon Klaus (@JonKlaus)

Storage Field Day – I will be attending SFD8!

Wow it’s early – Traveling to Storage Field Day 8

Coho Data: storage transformation without disruption

Pure Storage: Non Disruptive Everything

Cohesity is changing the definition of secondary storage

Qumulo: data-aware scale-out NAS

Nimble Storage – InfoSight VMVision

NexGen Storage: All-Flash Arrays can be hybrids too!

Infinidat: Enterprise reliability and performance

 

Alex Galbraith (@AlexGalbraith)

Looking Forward to Storage Field Day 8

Without good Analytics you dont have a competitive storage product

How often do you upgrade your storage array software?

Where and why is my data growing?…

Why are storage snapshots so painful?

 

Jarett Kulm (@JK47TheWeapon)

Storage Field Day 8 – Here I come!

 

Enrico Signoretti (@ESignoretti)

#SFD8, it’s storage prime time!

Analytics, the key to (storage) happiness

We are entering the Data-aware infrastructure era

Has the next generation of monolithic storage arrived?

Juku.beats 25: Qumulo, data-aware scale-out NAS

Infinidat: awesome tech, great execution

Juku.beats 27: NexGen Storage, QoS made easy.

Software defined? No no no, it’s poorly defined storage (and why Primary Data is different)

Juku.beats 28 – Infinidat storage: multiple nine resiliency, high performance, $1/GB

Are you going All-Flash? Nah, the future is hybrid

 

Vipin V.K. (@VipinVK111)

Tech Field Day Calling…! – #SFD8

Infinibox – Enterprise storage solution from Infinidat

Understanding NVMe (Non-Volatile Memory Express)

All-Flash symphony from Violin Memory

Cohesity – Secondary storage consolidation

With FLASH, things are changing ‘in a flash’ !?

 

Josh De Jong (@EuroBrew)

Storage Field Day Here I Come!

Thoughts in the Airport

NexGen Storage – The Future is Hybrid

Pure Storage – Enterprise Ready, Pure and Simple

 

Finally, thanks again to Stephen, Claire (and Tom in absentia). It was an educational and enjoyable few days and I really valued the opportunity I was given to attend.

SFD8_Group

 

Qumulo – Storage for people who care about their data

Disclaimer: I recently attended Storage Field Day 8.  My flights, accommodation and other expenses were paid for by Tech Field Day. There is no requirement for me to blog about any of the content presented and I am not compensated in any way for my time at the event.  Some materials presented were discussed under NDA and don’t form part of my blog posts, but could influence future discussions.

For each of the presentations I attended at SFD8, there are a few things I want to include in the post. Firstly, you can see video footage of the Qumulo presentation here. You can also download my raw notes from the presentation here. Finally, here’s a link to the Qumulo website that covers some of what they presented.

 

logo_qumulo_fullcolor_use_on_light

Qumulo is a 3.5 year old company from Seattle, WA that officially launched in March 2015. Interestingly, the founders invented OneFS and Isilon scale-out NAS. They pointed out that nowadays managing storage isn’t the big problem, it’s managing the data. Qumulo want to “[b]uild invisible storage that makes data visible”.

Qumulo_Invisible_Storage

Ostensibly, Qumulo’s QSFS is a software-only layer that can work either with Qumulo’s appliances, roll your own hardware, VMs or cloud.

 

Qumulo sells storage to people that care a great deal about their data

Qumulo stated that their key design goals were to be:

  • Hardware agnostic;
  • Flexible;
  • Fast and scalable; and
  • File and object aware.

They want to achieve this by building “data-aware, primary scale-out storage systems that provide real-time analytics to give visibility into data usage and performance at scale”. Sounds like big ambitions, but Qumulo seem confident they can pull it off.

The solution is ideally suited to commercial HPC and large-scale unstructured data environments, with real-time analytics that provides the ability to curate and manage the data.

Qumulo have taken a flash-first hybrid design approach with their hardware. In their opinion this maximises both price / performance and price / capacity ratio. I tend to agree with this approach, given the relative sluggishness that we’ve seen with regard to the drop in price of flash.

Qumulo also employ a SaaS-type software delivery model, leveraging a subscription model not perpetual licenses to provide pay-as-you-go access to “continual software innovation”.Everything about the solution is 100% programmable, with a public and self-documenting REST API with interactive API explorer built-in to the product. Interestingly, the development and release cycle is tight, with new iterations of the platform being released every 2 weeks.

If you’re into hardware, you might be interested in reading some more about the Qumulo QC24 Hybrid Storage and QC208.

QC24-Hardware

[image courtesy of Qumulo]

You can check the specs for yourself here, but think of the QC24 as the fast one and the QC208 as the big, dense one. Of note, Qumulo state that the supported cluster size is 4 – 1000 nodes. I think someone asked them about that and it hadn’t been tested with physical nodes yet. But the argument from Qumulo was that it was technically supported. Connectivity is also slightly different, with the QC24 Hybrid model sporting 2 * 10GbE SFP+ and the QC208 sporting 4 x 40GbE QSFP+. Cache is also slightly different to support the different capacities. Fine, so they’re not really the same.

 

Closing Thoughts and Further Reading

Qumulo state that they do “100% sales through channel”. I’ll be interested to see how long that model lasts. It’s something I always like to hear, as I work for a VAR, but oftentimes the reality is never quite as expected as the vendor grows and seeks new revenue opportunities.

In any case, as Qumulo pointed out in their presentation, we’re witnessing a shift in enterprise storage, moving from

  • Hardware-based to software-based;
  • Proprietary operating environments to linux-based OS platforms;
  • Hard drives -> flash-first hybrid appliances;
  • Petabytes of data to billions of files / objects; and
  • Storage management to data management.

Qumulo seem well-positioned, on paper at least, to flourish in this new world of enterprise storage. I like their approach to data rather than storage management. I’m really interested to see how they go over the next 12 months, particularly with the channel-only model and the rapid software development cycles.

Storage Field Day 8 – (Fairly) Full Disclosure

Disclaimer: I recently attended Storage Field Day 8.  My flights, accommodation and other expenses were paid for by Tech Field Day. There is no requirement for me to blog about any of the content presented and I am not compensated in any way for my time at the event.  Some materials presented were discussed under NDA and don’t form part of my blog posts, but could influence future discussions.

My full disclosure post will never reach the heady heights of Justin’s, all I can do is try and get somewhere near that level. Here are my notes on gifts, etc, that I received as a delegate at Storage Field Day 8. I’d like to point out that I’m not trying to play presenters / companies off against each other. I don’t have feelings one way or another about receiving gifts at these events (although I generally prefer small things I can fit in my suitcase). Rather, I just trying to make it clear what I received during this event to ensure that we’re all on the same page as far as what I’m being influenced by. Some presenters didn’t provide any gifts as part of their session – which is totally fine. I’m going to do this in chronological order, as that was the easiest way for me to take notes during the week. While every delegate’s situation is different, I’d also like to clarify that I took 2 days of vacation time and 3 days of training / work time to be at this event.

 

Saturday

I left my house Saturday morning at 4am and travelled BNE -> SYD -> SFO. I paid for my own cab to the airport in Brisbane, with the transfer in Sydney covered by my ticket. A period of time passed and I was given “dinner” and “breakfast”. This was included in the price of the ticket (paid for by Tech Field Day). Alcoholic beverages were not included, but I stuck with water. United wanted to sell me snacks at some stage too, but I politely declined. I had a spare seat next to me and the opportunity to watch “Office Space” for the umpteenth time on the tiny screen, so it wasn’t all bad.

 

Tuesday

On Tuesday night we had the delegate dinner at Antonella’s Ristorante in San Jose. It’s a nice Italian place. I had a bit of everything. As part of the gift exchange I received a pair of socks and some very nice beef jerky. I also had two Modelos and a Montejo at the hotel bar after dinner. Claire gave us all a bag of snacks when we got to the hotel. I particularly enjoyed the cashews and Reese’s things.

SW_BAG

 

Wednesday

We had breakfast at Mikayla’s. I had a breakfast roll. Coho Data provided us with doughnuts with bacon on top. While you can put bacon on all kinds of things, sometimes it’s best if you don’t. They also gave each of us a Coho-flavoured Raspberry Pi 2. We had lunch at Pure Storage provided by DishDash. They also gave us some brightly-coloured socks and a 8GB USB stick. Cohesity  provided us with a Hawaiian-themed food and beverages, including Mai Tais. This may or may not have been a mistake given the feistiness of some of the delegates during the ensuing session. They also gave us customised Amazon Echo devices. I gave mine to Claire as it wouldn’t work in Australia. We also received a Cohesity sleeveless fleece top – arguably these don’t work in Australia either, as it doesn’t get that cold where I live :). We had a social event at the Hotel Valencia on Santana Row. I had some sliders and a few Heinekens. I then went to the bar at the hotel and had a Fat Tire before retiring for the night.

 

Thursday

We had a buffet breakfast at the hotel. I had some bacon and eggs and fruit. Qumulo provided us with a Qumulo-branded notepad. We had a Californian BBQ-style lunch at the hotel. After a few more sessions, we then had dinner at Picasso’s in downtown San Jose. It was a tapas place, so I had a bit of everything. And two glasses of Sangria. I then had a Fat Tire at the hotel bar upon our return.

 

Friday

On Friday morning we had breakfast at Violin Memory. I had a breakfast burrito and coffee (I loves me some Mexican in the morning). They also gave us a T-shirt, stubbie holder, bottle opener, mints and toothpicks. Intel provided us with a nice buffet lunch at the Intel HQ Executive Briefing Centre. Nimble Storage gave us each a small tote bag, a webcam privacy screen, some headphones and a travel set. This was followed by “Happy Hour” at Nimble. I had a sparkling water. To finish off we had dinner at Mexicali in Santa Clara. I had a prawn burrito. I didn’t eat anything on the flight home.

 

Conclusion

I’d like to extend my thanks once again to the Storage Field Day organisers and the companies presenting at the event. I had an extremely enjoyable and educational time. Here’s a photo.

FINAL

 

Storage Field Day 8 – Day 0

Disclaimer: I recently attended Storage Field Day 8.  My flights, accommodation and other expenses were paid for by Tech Field Day. There is no requirement for me to blog about any of the content presented and I am not compensated in any way for my time at the event.  Some materials presented were discussed under NDA and don’t form part of my blog posts, but could influence future discussions.

This is just a quick post to share some thoughts on day zero at Storage Field Day. I thought I’d share some touristy snaps before we get into the meat and potatoes stuff later in the week. Here’s the obligatory wing shot – this one was taken as we were about to start the descent into SFO.

I spent a few days relaxing with a friend in the Bay Area before meeting up with my fellow delegates on Tuesday afternoon. Tuesday evening we all got together and headed to Antonella’s Ristorante in San Jose. It was great.

IMG_2343

Anyway, enough with the holiday snaps. I just wanted to thank Stephen and Claire for having me back, making sure everything is running according to plan and for just being really very decent people. I’ve really enjoyed catching up with the people I’ve met before and meeting the new guys. Look out for some posts related to the Tech Field Day sessions in the next few weeks. And if you’re in a useful timezone, check out the live streams from the event here, or the recordings afterwards.