EMC – Sometimes RAID 6 can be a PITA

Posted on by

This is really a quick post to discuss how RAID 6 can be a bit of a pain to work with when you’re trying to combine traditional CLARiiON / VNX DAEs and Storage Pool best practices. It’s no secret that EMC strongly recommend using RAID 6 when you’re using SATA-II / NL-SAS drives that are 1TB or greater. Which is a fine and reasonable thing to recommend. However, as you’re no doubt aware, the current implementation of FAST VP uses Storage Pools that require homogeneous RAID types. So you need multiple tools if you want to run both RAID 1/0 and RAID 6. If you want a pool that can leverage FAST to move slices between EFD, SAS, and NL-SAS, it all needs to be RAID 6. There are a couple of issues with this. Firstly, given the price of EFDs, a RAID 6 (6+2) of EFDs is going to feel like a lot of money down the drain. Secondly, if you stick with the default RAID 6 implementation for Storage Pools, you’ll be using 6+2 in the private RAID groups. And then you’ll find yourself putting private RAID groups across backend ports. This isn’t as big an issue as it was with the CX4, but it still smells a bit ugly.

What I have found, however, is that you can get the CLARiiON to create non-standard sized RAID 6 private RAID groups. If you create a pool with 10 spindles in RAID 6, it will create a private RAID groups in a 8+2 configuration. This seems to be the magic number at the moment. If you add 12 disks to the pool it will create 2 4+2 private RAID groups, and if you use 14 disks it will do a 6+2 and a 4+2 RAID group. Now, the cool thing about 10 spindles in a private RAID group is that you could, theoretically (I’m extrapolating from the VNX Best Practices document here), split the 8+2 across two DAEs in a 5+5. In this fashion, you can increase the rebuild times slightly in the event of a disk failure, and you can also draw some sensible designs that fit well in a traditional DAE4P. Of course, creating your pools in increments of 10 disks is going to be a pain, particularly for larger Storage Pools, and particularly as there is no re-striping of data done after a pool expansion. But I’m sure EMC are focussing on this issue in the future, as a lot of customers have had a problem with the initial approach. The downside to all this, of course, is that you’re going to suffer a capacity and, to a lesser extent, performance penalty by using RAID 6 across the board. In this instance you need to consider whether FAST VP is going to give you the edge over split RAID pools or traditional RAID groups.

I personally like the idea of Storage Pools, and I’m glad EMC have gotten on-board with them in their midrange stuff. I’m also reasonably optimistic that they’re working on addressing a lot of issues that have come up in the field. I just don’t know when that will be.

10 Comments

  1. RAID 6 is a find option vs. RAID 10 except for slower writes. Being able to lose 2 drives is a nice plus.

  2. Dan,

    I’d like to emphasize that extending pools today is a very bad idea because – as you already mentioned – there is no restriping going on after adding a new private RAID group / bunch of disks to a pool. New LUNs migrated to or created in this pool will be placed only on that fresh and empty disks. The striping over all disks in the pool will only begin from the point where all disk have the same capacity filling level.

    That might result in a significant performance decrease.

    I agree with you: The idea of pools is a consequent way of implementing automation and virtualization concepts. But at this point of maturity you have to look out for pitfalls because of limitations.

    AND @EMC: speak, blog, chat, present about the limitations existing today. That would make your customers more happy than finding themselves stuck in a pitfall.

  3. I’d like to emphasize that extending pools today is a very bad idea because – as you already mentioned – there is no restriping going on after adding a new private RAID group / bunch of disks to a pool. New LUNs migrated to or created in this pool will be placed only on that fresh and empty disks. The striping over all disks in the pool will only begin from the point where all disk have the same capacity filling level.
    +1

  4. Hi Daniel,

    I agree with everything you’ve said. I’ve found my local EMC guys are fairly open when limitations in their product are discoveredp – unfortunately it always seems to be after we’ve already purchased said product. That said, at least in a smaller market like the one I’m in, the locals only get to really know about stuff when a customer implements it. I imagine in America and Europe this is not as much of an issue.

  5. The key is that it’s not a problem extending the pool when you increase the pool by the same number of disks. In addition, most people don’t see the ramifications from doing said restripe. I assume in small environments this may be an option, however, in larger environments a restripe would probably not be an option anyway. Hence, increase the pool by the same amount of disks is the solution.

  6. Hey Dave, I was under the (perhaps mistaken) impression that, even if I expand the pool by the same number of spindles, I could still run into performance issues, given the way that the pools allocate the 1GB slices after it is expanded. Or did I misread http://virtualeverything.wordpress.com/2011/03/05/emc-storage-pool-deep-dive-design-considerations-caveats/ ? I’d still prefer to allocate all of the spindles up front, sidestepping the issue entirely. But that’s not always an option. Feel free to correct me on this one.

  7. Hi Dan. I’ve never implemented pools with a small amount of disks. I’ve always had a decent number of disks in the pool. If you had 40 disks in the pool, the solution is to increase the pool by 40 disks to maintain performance you originally had. In addition, as of late, they have all been 2 tiers with FAST VP enabled. I usually take the EFD and put it towards FAST cache. With all that said, for something that needs a certain amount of IOPS, ie. Exchange 2007, I always go the traditional route and create a 15K 1+0. Or in the case of Recoverpoint, a 10K 1+0. You can do a traditional RG or create a homogeneous pool. If you have something that needs the performance, a pool might not be the answer for the requirement. Best practice is to use traditional RG’s for things that have specific performance requirements.

  8. Dave, I agree it always comes down to the use case as to whether you use Pools or traditional RAID Groups. And you’re invariably better off if you can deploy the appropriate amount of disks up front when using Pools. I guess I’ve just been a bit underwhelmed with EMC’s initial approach pushing Pools and then backing out and saying “oh no, you need to use Metas in RAID Groups to really make best use of these disks”. Which has led to some awkward conversations with CxOs around return on investment for particular projects that may have involved a significant investment in EFDs. Still, I’m hopeful that they’ll keep working on the technology and this time next year I’ll have something different to whinge about.

  9. I just found your blog and I have to say am really impressed with the quality of posts. This particular post highlights one of the most under-communicated issues with FAST that gets glossed over during a sales campaign. There was a lot of talk earlier this year from EMC folks that it would be fixed in Q4. Well, the end of Q4 is rapidly approaching and I haven’t heard a peep about it lately. That doesn’t make me very hopeful. I got burned recommending pools too early to customers when FLARE 30 came out. Now that I know how many architectural changes have to happen to accommodate multiple-protection type RAID pools, I wouldn’t recommend any customer think about implementing them for a good 6 months so the code can get patched a couple times.

  10. Thanks for the kind words and for stopping by the blog. Rest assured I’m hearing talk from EMC folks about a Q1 fix, so we’re suffering from a similar problem here too. I was burnt on pools too, specifically mixing EFDs with FC and SATA in a pool that was retrospectively deemed to be inappropriate for the workload. That said, I think pools are very good when a) you have the spindles you need up front, and b) you don’t have super high I/O requirements. I’ve used some rather large pools on CX4s to do what I needed for some run-rate type VM workloads and, combined with FAST VP, it’s gone well. And it’s certainly less of a headache than a whole bunch of Metas :)

Comments are closed.