Category Archives: Cloudian

VMware – VMworld 2019 – HBI2537PU – Cloud Provider CXO Panel with Cohesity, Cloudian and PhoenixNAP

Posted on by

Disclaimer: I recently attended VMworld 2019 – US.  My flights and accommodation were paid for by Digital Sense, and VMware provided me with a free pass to the conference and various bits of swag. There is no requirement for me to blog about any of the content presented and I am not compensated by VMware for my time at the event.  Some materials presented were discussed under NDA and don’t form part of my blog posts, but could influence future discussions.

Here are my rough notes from “HBI2537PU – Cloud Provider CXO Panel with Cohesity, Cloudian and PhoenixNAP”, a panel-type presentation with the following people:

You can grab a PDF copy of my notes from here.

Introductions are done.

YR: William, given your breadth of experience, what are some of the emerging trends you’ve been seeing?

WB: Companies are struggling to keep up with the pace of information generation. Understanding the data, storing and retaining it, and protecting it. Multi-cloud adds a lot of complexity. We’ve heard studies that say 22% of data generated is actually usable. It’s just sitting there. Public cloud is still hot, but it’s settling down a little.

YR: William comes from a massive cloud provider. What are you guys using?

WB: We’ve standardised on vCloud Director (vCD) and vSphere. We came from build our own but it wasn’t providing the value that we hoped it would. Customers want a seamless way to manage multiple cloud resources.

YR: Are you guys familiar with VCPP?

AP: VCPP is the crown jewel of our partner program at VMware. 4000+ providers, 120+ countries, 10+ million VMs, 10000+ DCs. We help you save money, make money (things are services ready). We’re continuing to invest in vCD. Kubernetes, GPUs, etc. Lots of R&D.

YR: William, you mentioned you standardised on the VMware platform. Talk to us about your experience. Why vCD?

WB: It’s been a checkered past for vCD. We were one of the first five on the vCloud Express program in 2010 / 11. We didn’t like vCD in its 1.0 version. We thought we can do this better. And we did. We launched the first on-demand, pay by the hour public cloud for enterprise in 2011. But it didn’t really work out. 2012 / 13 we started to see investments being made in vCD. 5.0 / 5.5 improved. Many people thought vCD was gong to die. We now see a modern, flexible portal that can be customised. And we can take our devs and have them customise vCD, rather than build a customised portal. That’s where we can put our time and effort. We’ve always done things differently. Always been doing other things. How do we bring our work in visual cloud into that cloud provider portal with vCD?

YR: You have an extensive career at VMware.

RR: I was one of the first people to take vCD out to the world. But Enterprise wasn’t mature enough. When we focused on SPs, it was the right thing to do. DIY portals needs a lot of investment. VMware allows a lot of extensibility now. For us, as Cohesity, we want to be able to plug in to that as well.

WB: At one point we had 45 devs working on a proprietary portal.

YR: We’ve been doing a lot on the extensibility side. What role are services playing in cloud providers?

AP: It takes away the complexities of deploying the stack.

JT: We’re specifically in object. A third of our customers are service providers. You guys know that object is built for scale, easy to manage, cost-effective. 20% of the data gets used. We hear that customers want to improve on that. People are moving away from tape. There’s a tremendous opportunity for services built on storage. Amazon has shown that. Data protection like Cohesity. Big data with Splunk. You can offer an industry standard, but differentiate based on other services.

YR: As we move towards a services-oriented world, William how do you see cloud management services evolving?

WB: It’s not good enough to provide some compute infrastructure any more. You have to do something more. We’re stubbornly focussed on different types of IaaS. We’re not doing generic x86 on top of vSphere. Backup, DR – those are in our wheelhouse. From a platform perspective, more and more customers want some kind of single pane of glass across their data. For some that’s on-premises, for some its public, for some it’s SaaS. You have to be able to provide value to the customer, or they will disappear. Object storage, backup with Cohesity. You need to keep pace with data movement. Any cloud, any data, any where.

AP: I’ve been at VMware long enough not to drink the Kool-Aid. Our whole cloud provider business is rooted in some humility. vCD can help other people doing better things to integrate. vCD has always been about reducing OPEX. Now we’re hitting the top line. Any cloud management platform today needs to open, extensible, not try to do anything.

YR: Is the crowd seeing pressure on pure IaaS?

Commentator: Coming from an SP to enterprise is different. Economics. Are you able to do a show back with vCD 9 and vROps?

WB: We’re putting that in the hands of customers. Looking at CloudHealth. There’s a benefit to being in the business management space. You have the opportunity to give customers a better service. That, and more flexible business models. Moving into flexible billing models – gives more freedom to the enterprise customer. Unless you’re the largest of the large – enterprises have difficulty acting as a service provider. Citibank are an exception to this. Honeywell do it too. If you’re Discount Tire – it’s hard. You’re the guy providing the service, and you’re costing them money. There’s animosity – and there’s no choice.

Commentator: Other people have pushed to public because chargeback is more effective than internal show back with private cloud.

WB: IT departments are poorly equipped to offer a breadth of services to their customers.

JT: People are moving workloads around. They want choice and flexibility. VMware with S3 compatible storage. A common underlying layer.

YR: Economics, chargeback. Is VMware (and VCPP) doing enough?

WB: The two guys to my right (RR and JT) have committed to building products that let me do that. I’ve been working on object storage use cases. I was talking to a customer. They’re using our IaaS and connected to Amazon S3. You’ve gone to Amazon. They didn’t know about it though. Experience and cost that can be the same or better. Egress in Amazon S3 is ridiculous. You don’t know what you don’t know. You can take that service and deliver it cost-effectively.

YR: RR talk to us about the evolution of data protection.

RR: Information has grown. Data is fragmented. Information placement is almost unmanageable. Services have now become available in a way that can be audited, secured, managed. At Cohesity, first thing we did was data protection, and I knew the rest was coming. Complexity’s a problem.

YR: JT. We know Cloudian’s a leader in object storage. Where do you see object going?

JT: It’s the underlying storage layer of the cloud. Brings down cost of your storage layer. It’s all about TCO. What’s going to help you build more revenue streams? Cloudian has been around since 2011. New solutions in backup, DR, etc, to help you build new revenue streams. S3 users on Amazon are looking for alternatives. Many of Cloudian’s customers are ex-Amazon customers. What are we doing? vCD integration. Search Cloudian and vCD on YouTube. Continuously working to drive down the cost of managing storage. 1.5PB in a 4RU box in collaboration with Seagate.

WB: Expanding service delivery, specifically around object storage, is important. You can do some really cool stuff – not just backup, it’s M&E, it’s analytics. Very few of our customers are using object just to store files and folders.

YR: We have a lot of providers in the room. JT can you talk more about these key use cases?

JT: It runs the gamut. You can break it down by verticals. M&E companies are offering editing suites via service providers. People are doing that for the legal profession. Accounting – storing financial records. Dental records and health care. The back end is the same thing – compute with S3 storage behind it. Cloudian provides multi-tenanted, scalable performance. Cost is driven down as you get larger.

YR: RR your key use cases?

RR: DRaaS is hot right now. When I was at VMware we did stuff with SRM. DR is hard. It’s so simple now. Now every SP can do it themselves. Use S3 to move data around from the same interface. And it’s very needed too. Everyone should have ubiquitous access to their data. We have that capability. We can now do vulnerability scans on the data we store on the platform. We can tell you if a VM is compromised. You can orchestrate the restoration of an environment – as a service.

YR: WB what are the other services you want us to deliver?

WB: We’re an odd duck. One of our major practices is information security. The idea that we have intelligent access to data residing in our infrastructure. Being able to detect vulnerabilities, taking action, sending an email to the customer, that’s the type of thing that cloud providers have. You might not be doing it yet – but you could.

YR: Security, threat protection. RR – do you see Cohesity as the driver to solve that problem?

RR: Cohesity will provide the platform. Data is insecure because it’s fragmented. Cohesity lets you run applications on the platform. Virus scanners, run books, all kinds of stuff you can offer as a service provider.

YR: William, where does the onus lie, how do you see it fitting together?

WB: The key for us is being open. Eg Cohesity integration into vCD. If I don’t want to – I don’t have to. Freedom of choice to pick and choose where we went to deliver our own IP to the customer. I don’t have to use Cohesity for everything.

JT: That’s exactly what we’re into. Choice of hardware, management. That’s the point. Standards-based top end.

YR: Security

*They had 2 minutes to go but I ran out of time and had to get to another meeting. Informative session. 4 stars.

Cloudian Announces HyperStore 7 – Gets Super Cloudy

Posted on by

Cloudian recently announced HyperStore 7, and I was fortunate enough to grab a few minutes with John Toor to run through the announcement.

 

The Announcement

The key features of HyperStore 7 include:

  • Multi-cloud access via a common API: Manage all cloud and on-premises storage assets, including Amazon AWS, Google GCP, and Microsoft Azure via a common API
  • Merge Files and Objects:  Combine file and object management to a single namespace, accessed via SMB (CIFS) / NFS protocols and the S3 API
  • Scale-out architecture: Multiple distributed controllers can manage a single namespace across on-premises and cloud environments for performance scaling, increased availability and simplified data access
  • Converged Data Access: Permits data stored as files to be retrieved as objects, or vice versa, providing full data interchangeability

I’ll run through these in a little more detail below.

 

Multi-cloud via Common API

The cool thing about HyperStore 7 is that it’s delivered as a single software image. This means you can manage your HyperStore environment from a common interface, regardless of whether it’s an appliance located on-premises, or a virtual image running in Azure, GCP or AWS.

[image courtesy of Cloudian]

 

The common image also means you can start out small and build up. You can deploy on-premises first, then work up to a hybrid cloud deployment, and then, if you’re so inclined, you can deploy HyperStore 7 natively in the cloud. The best thing about this feature is that you don’t need to undo the work you’ve already done on-premises, you can just build on it.

 

Files and Objects, Together

Once of the most exciting features, in my opinion, is “Converged Data Access”. The recent introduction of HyperFile ramps up the file and object play considerably, with a single namespace across multiple environments, and files and objects being stored in that namespace. You can access data in object or file format interchangeably as well.

[image courtesy of Cloudian]

 

Note also that data stored in its native cloud format. So if you’re using Azure, for example, your data is stored in blob format, and is thus accessible to other applications that can leverage that format.

 

Other Notes

The basic edition of HyperFile is included with HyperStore at no charge. The hardware appliance remains the primary model for on-premises deployments, with Cloudian noting that a lot of customers are still most comfortable buying hardware from a vendor for their storage deployments.

 

Thoughts

With the introduction of HyperFile, Cloudian made some leaps ahead in terms of breadth of offering. In my opinion, the ability to deploy HyperStore 7 on your favourite public cloud platform, and have it running a shared data pool with your on-premises HyperStore storage, is simply icing on the cake. A lot of people are talking about how they are all in with multi-cloud solutions, but it seems that Cloudian have come up with a fairly simple solution to the problem. You’ll need to do a little work to make sure your networking is set up in the way you need it to meet your requirements, but you’d need to do that if you were looking to do file or object in public cloud in any case. There are a bunch of use cases for this type of technology, and it’s nice to see that it’s not a bunch of different products glued together and called a solution.

It’s no secret that I think Cloudian have been doing some pretty cool stuff in the object space for a while now. The addition of HyperFile capability last year, and this multi-cloud capability in HyperStore 7, gets me all kinds of excited to see what they’ve got in store for the future. If you’re after a scalable object (and file) solution works well on-premises and off-premises, you’d do well to check out what Cloudian has to offer.

Cloudian Announces HyperFile, Makes Object Better

Posted on by

Cloudian recently announced an addition to their HyperStore appliance. I had the opportunity to be briefed by Jon Toor and thought I’d share the highlights of the announcement here. I’ve had the pleasure of talking to Cloudian at a few Storage Field Day events. If you’re unfamiliar with the HyperStore 4000, you can read my coverage of it here. In short, it’s 840TB of object storage in 4RU with really, really, comprehensive S3 compliance, amongst other things.

 

HyperFile You Say?

HyperFile is the new file front-end controller for the HyperStore appliance. It supports the following features:

  • SMB3 and NFS3;
  • High Availabilty with active / passive controllers;
  • Non-disruptive failover;
  • POSIX compliance;
  • Active Direcotry / LDAP authentication;
  • Write Once Read Many (WORM); and
  • Snapshots.

It wouldn’t be a product announcement without a bezel shot. I can’t say whether this is actually what it looks like, but if it does, it’s kind of cool.

[image courtesy of Cloudian]

The appliance itself is 2RU with dual controllers and a shared backplane. The cool thing is that it can be deployed as VMs, making it appealing for service providers looking to setup multiple environments for customers. Supported hypervisors include vSphere 5.1 (or later) and KVM. Replication is handled at the HyperStore level.

Multi-tenancy is supported with dedicated controllers.

[image courtesy of Cloudian]

There’s a global namespace between file and object and it also supports a shared namespace across multiple NAS controllers, meaning you can up your number of controllers to increase bandwidth or replication performance. From a scalability perspective, it supports up to 64 namespaces per controller. One of my favourite features is what Cloudian call “converged access” between file and object, meaning you could use S3 for storing files. It also supports Microsoft Azure, Google Cloud Platform and Amazon S3 formats, opening up some interesting possibilities for file consumption on-premises and in the cloud.

There are two editions available. The Basic HyperFile NAS Controller includes

  • Full protocol support;
  • High-availability;
  • Converged data access; and
  • Data migration.

The Enterprise HyperFile NAS Controller adds

  • Snapshot;
  • WORM; and
  • Geo-distribution with file versioning/locking.

 

Thoughts

I’ve been a fan of Cloudian’s products for some time, and this addition to the HyperStore platform makes them a compelling option for file and object storage in the data centre. With this approach they’re looking to push further into Media Asset Management (MAM) and video surveillance solutions. The title of the post is misleading. Object is already pretty cool, and a very suitable solution for a number of workloads. So why would an object vendor need to add file to work in these industries? Isn’t object ideally suited to these kinds of workloads? Yes, but sometimes the leading software vendors and people in charge of workflows are focused on other things, like only supporting file. So Cloudian have adapted to take a bigger piece of the pie. In much the same way that some data protection solutions are still file oriented, the HyperFile allows Cloudian to play in areas where it’s traditionally been excluded.

I’m also a fan of the appliance as VM approach and I like the breadth of protocol support and cloud integration available. If you’re going to put cloud in the name of your company the expectation will be there that you know what you’re doing. Cloudian haven’t disappointed thus far. If you’re in the market for a solid object (and now file) solution, you could do worse than talking to the folks at Cloudian.

Cloudian Announces HyperStore 4000 – Gets Super Dense

Posted on by

I’ve written about Cloudian before, having had the good fortune of seeing them present at Storage Field Day 7 and Storage Field Day 10. I recently had the opportunity to be briefed by Jon Toor, Cloudian’s CMO, on the HyperStore 4000 (the datasheet can be found here and Cloudian’s press release can be found here).

 

Super Dense Spec

[image via Cloudian]

The HyperStore 4000 comes with two nodes per chassis, each with 2 Intel E5-2620 v4, 8 core CPUs and 128GB RAM. There are 35 hot-swap 3.5″ drives per node, with 2 800GB hot-swap SSDs per node and 10GbE networking (same as the HyperStore 1500).

With raw capacity of 700TB in a 4RU enclosure, Cloudian tell me the HyperStore 4000 reduces storage costs by 40 percent versus their previous solutions.

It also offers:

  • Data availability is enhanced by the resilient hardware architecture (two compute nodes per chassis), delivering 99.999999 percent data durability from a three-appliance cluster; and
  • Built-in hybrid cloud tiering (to Amazon S3 or Google Cloud) enables customers to optimize their storage model with a combination of on-premises Cloudian storage and public cloud storage.

 

Thoughts

I’ve written about Cloudian’s S3 guarantee before – and this is still a big part of the value proposition. I spoke to Jon about where he saw uptake in Cloudian (and object storage in general) in the enterprise. He talked about data protection on-premises being the obvious problem being solved with object and spoke of object storage cost approaching the same economics as tape. But there’s other stuff you can use object storage for too, including media asset managers (using S3 compliance) and video surveillance. Most of the software on the market has some kind of hook into S3, and this is heavily leveraged by Cloudian to provide the same services on-premises.

But object storage (and Cloudian in particular) is not just about cheap and deep. Object does distributed storage really well too. Jon spoke about Cloudian providing good, verifiable visibility into data locality. This can be extremely important for government authorities and businesses with a requirement to focus not just on how their data is stored, but also where.

If you’re in the market for some nice, dense, smart object storage, I’d encourage you to look further into the Cloudian offering. For another perspective on the HyperStore 4000 announcement, El Reg has a good summary here.

Cloudian Does Object Smart and at Scale

Posted on by

Disclaimer: I recently attended Storage Field Day 10.  My flights, accommodation and other expenses were paid for by Tech Field Day. There is no requirement for me to blog about any of the content presented and I am not compensated in any way for my time at the event.  Some materials presented were discussed under NDA and don’t form part of my blog posts, but could influence future discussions.

cloudian-logo

Before I get started, you can find a link to my raw notes on Cloudian‘s presentation here. You can also see videos of the presentation here.

I’m quite keen on Cloudian’s story, having seen them in action at Storage Field Day 7. I also got to have a beer with Michael Tso at the SFD10 mixer and talk all things Australian.

SFD10_Cloudian_MichaelTso_Cropped

 

Smart and at Scale

Cloudian took us through some of their driving design principles, and I thought it was worth covering these off again. You’ll notice the word “scale” gets used a lot, and this has been a particularly important capability for Cloudian. They did a blog post on it too.

One of the key features of the HyperStore solution is that it needed to support what Cloudian term “Smart Operations at Scale”. This requires the tech to:

  • Be simple and intuitive;
  • Be fully automated from an operations perspective (e.g. add/remove drives/nodes, upgrades);
  • Provide visual storage analytics to automatically see hot spots; and
  • Offer self service consumption (via a policy based approach).

Cloudian have also worked hard to ensure they can provide “Extreme Durability at Scale”, with the HyperStore solution offering the ability to:

  • Be always repaired, always verified;
  • Offer automated failure avoidance (through the use of Dynamic Object Routing); and
  • Be “enterprise” grade.

One of the keys to being able deliver a scaleable solution has been the ability to provide the end user with “Smart support at Scale”, primarily through the use of:

  • Proactive (not reactive) support;
  • Continuous monitoring; and
  • Global analytics.

The analytics piece is a big part of the Cloudian puzzle, and something they’ve been working hard on recently. With their visual analytics you can analyse your data across globe and plan for future based on your demand. Cloudian not only performs analytics at scale, but also designed to facilitate operations at scale, with:

  • One screen for hundreds of nodes (in a kind of “beehive” layout);
  • Instant view of a node’s health;
  • The ability to add nodes with one click; and
  • The ability to dynamically rebalance the cluster.

When it comes to software defined storage platforms, the simple things matter, particularly as it relates to your interactions with the hardware platform. To that end, with HyperStore you’ve got the ability to do some basic stuff, like:

  • Identifying node types;
  • Blinking suspect servers; and
  • Blinking suspect drives.

When you’re running a metric s**t-tonne of these devices in a very big data centre, this kind of capability is really important, especially when it comes to maintenance. As is the ability to perform rolling upgrades of the platform with no downtime and in an automated fashion. When it comes to rebuilds, Cloudian provides insight into both data rebuild information and cluster rebalance information – both handy things to know when something’s gone sideways.

The Cloudian platform also does “Smart Disk Balancing”. If there’s a disk imbalance it will change the tokens pointing from “highly used disk to low used disk”. If there’s a disk failure, new data automatically routes to newly assigned resources. Makes sense, and nice to see they’ve thought it through.

 

Further Reading and Conclusion

Cloudian make quite a big deal of their S3 compatibility. They even give me a sticker that says it’s guaranteed. It looks a lot like this:

Badge_S3YourDataCenter_transparent2

Chris Evans also did a series of posts on S3 and Cloudian that you can read here, here and here. He also did a great preview post prior to SFD10 which is also worth a look. He’s a good lad, he is. Particularly when I need to point you, my loyal reader, to well written articles on topics I’m a little sketchy on.

S3 compatibility is a big thing for a lot of people looking at deploying object storage, primarily because AWS are leaps and bounds ahead of the pack in terms of object storage functionality, deployed instances, and general mindshare. Cloudian haven’t just hitched their wagon to S3 compatibility though. In my opinion they’ve improved on the S3 experience through clever design and a solid approach to some fundamental issues that arise when you’re deploying a whole bunch of devices in data centres that don’t often have staff members present.

Storage Field Day 7 – Day 3 – Cloudian

Posted on by

Disclaimer: I recently attended Storage Field Day 7.  My flights, accommodation and other expenses were paid for by Tech Field Day. There is no requirement for me to blog about any of the content presented and I am not compensated in any way for my time at the event.  Some materials presented were discussed under NDA and don’t form part of my blog posts, but could influence future discussions.

For each of the presentations I attended at SFD7, there are a few things I want to include in the post. Firstly, you can see video footage of the Cloudian presentation here. You can also download my raw notes from the presentation here. Finally, here’s a link to the Cloudian website that covers some of what they presented.

 

Overview

Michael Tso, CEO and co-founder of Cloudian, provided us with a brief overview of the company. It was founded about 4 years ago, and a lot of the staff’s background was experience with hyper-scale messaging systems for big telcos. They now have about 65 staff.

Cloudian  offers a software version as well as a hardware appliance that runs their HyperStore software. The hardware appliance comes in 3 different flavours:

  • Entry Level;
  • Capacity Optimised; and
  • Performance Optimised.

The software is supported on RedHat and CentOS.

 

Architecture

Paul Turner, Chief Marketing and Product Officer, gave us an introduction to the architecture behind Cloudian. Their focus is on using commodity servers, that provide scale out capability, are durable, and simple to use. “If you don’t make it dead easy to add nodes or remove nodes on the fly you don’t have a good platform”.

The platform uses

  • Erasure Coding;
  • Replication; and
  • Compression

Here’s a picture of what’s inside:

SFD7_Day3_Cloudian_What's_Inside

Features include:

  • Natively S3;
  • Hybrid Storage Cloud;
  • Extreme durability;
  • Multi-tenant;
  • Geo-distribution;
  • Scale out;
  • Intelligence in Software;
  • Smart Support;
  • Data Protection;
  • QoS;
  • Programmable; and
  • Billing and Reporting.

They also make use of an Adaptive Policy Engine (multi-tenant, continuous, adaptive, policy engine), which offers:

  • Policy controlled virtual storage pools (buckets like Amazon);
  • Scale / reduce storage on demand;
  • Multi-tenanted with many application tenants on same infrastructure;
  • Dynamically adjust protection policies;
  • Optimise for small objects by policy; and
  • Cloud archiving by virtual pool.

 

Here’s a diagram of the logical architecture.

SFD7_Day3_Cloudian_Architecture

They use Cassandra as the core metadata and distribution mechanism. Why Cassandra? Well it’s

Scalable

  • Supports 1000s of nodes
  • Adds capacity by adding nodes to running system
  • Distributed shared-nothing P2P architecture, with no single point of failure

Reliable

  • Data durability, synced to disk
  • Resilient to network or hardware failures
  • Multi-DC replication
  • Tuneable data consistency level

Provides Features such as

  • Vnodes, TTL, secondary indexes, compression, encryption

Performant

  • Write path especially fast

Multiple data protection policies, including:

  • NoSQL DB, Replicas, Erasure Coding

Policy features

  • ACL, QoS, Tiering, versioning, etc.

vnodes

  • Nodes remapped to physical disks. then one disk failure only affects those nodes;
  • Maximum 256 nodes per physical node. no token management. tokens randomly assigned;
  • Parallel I/O across nodes;
  • Increased repair speed in case of disk or node failure; and
  • Allows heterogeneous machines in a cluster.

 

Further Reading and Final Thoughts

If you’re doing a bit with cloud storage, I think these guys are worth checking out. I particularly like the use case for Cloudian deployed as an on-premises S3 cloud behind the firewall. There’s also a Community Edition available for download. You can use HyperStore Community Edition software for:

  • For product evaluation;
  • Testing HyperStore software features in a single or multi-node install; and
  • Building 10TB object storage systems free of charge.

I think that’s pretty neat. I also recommend checking out Keith’s preview of Cloudian.