VMware – VMworld 2014 – BCO2701 – vSphere HA Best Practices and FT Tech Preview

Disclaimer: I recently attended VMworld 2014 – SF.  My flights and accommodation were paid for by myself, however VMware provided me with a free pass to the conference and various bits of swag. There is no requirement for me to blog about any of the content presented and I am not compensated in any way for my time at the event.  Some materials presented were discussed under NDA and don’t form part of my blog posts, but could influence future discussions.

banner-hero-mon-stream

BCO2701 – vSphere HA Best Practices and FT Tech Preview

BCO2701

If you’ve read my other posts from this week it’s probably pretty clear  that I should give up the charade of not providing a transcript of the session. So let’s just call them notes and not talk about it any more, yeah?

This session was presented by:

  • GS Khalsa – VMware Senior Technical Marketing Manager
  • Manoj Krishnan – VMware Engineer – vSphere HA

Agenda:

  • What’s New
  • Failure Events
  • Best Practices (Network, HA and VSAN, Host Isolation Response and Admission Control)
  • Tech Previews

What’s New
In vSphere 5.5

  • Protection for VSAN VMs (major)
  • AppHA Integration
  • VM-VM Anti-affinity rule

What is HA?

  • Minimises downtime
  • Auto VM recovery in minutes
  • Protects against 3 types of failures:

– Infrastructure – host, vm
– Connectivity – host network iso, DS incurs PDL
– App – GuestOS hangs / crashes, app hangs / crashes

  • Cluster with a master and multiple slaves
  • Heartbeats via network and storage
  •  HA network can be management network or VSAN network (if using VSAN)

Failure Events
Once the HA agents are up and running, they go into election mode.

What if a slave dies? Master declares and restarts affected VMs on another host. If the Master fails, slaves go into election mode and restart VMs. Note that vCenter only talks to one master, so if you have a network partition, this can impact where VMs will startup in a failure event.

Host Isolation
The host declares itself isolated if it can’t talk to Master via network. Lets the master know via the datastore heartbeat

Best Practices

Network

  • Redundant HA network
  • Fewest hops possible
  • Consistent portgroup names, network labels
  • Route based on originating port ID
  • Failback policy = no
  • Enable portfast, Edge, etc
  •  Same MTU Size
  •  Disable host monitoring during NW maint
  •  vmknics on separate subnets

Storage

  • heartbeats – all hosts should see the same datastores
  • choose a heartbeat datastore that is fault isolated from ha network and resilient to failures
  • override auto-selected options as required

HA and VSAN

  • Heartbeat datastore is not necessary for VSAN cluster
  • Add non-VSAN datastore to cluster hosts if VM MAC address collisions on the VM network are a significant concern
  • Choose a datastore that is fault isolated from the VSAN network
  • Set the isolation address with das.isolationAddressX
  • Configure HA to not use the default management network gateway
  • If VSAN is non-routable, use a pingable isolation address somewhere else in the network

Host Isolation Response

To delay response in 5.1+, use das.config.fdm.isolationPolicyDelaySec

There are a few options:

  • Leave Powered On (default with 5.1+)
  • Shutdown (default with 4.x)
  • Power Off

Which to use? It depends.

  • What type of VM?
  • Will host likely retain access to storage?
  • Will VM likely retain access to VM?

Admission Control
If you turn admission control off, you’re limiting what HA can do for you.

You can reserve resources in case of host failures

  • Ensures resources are available
  • No guarantee VMs will be happy after a failure though
  • Working on closing this gap

How to do it?

  • Select the appropriate policy
  • Enable DRS
  • Simulate failures to test by using maintenance mode and the Impact Assessment fling

Make adjustments if:

  • VMs are not restarted
  • Desired performance is not realised

Reducing Impediments

  • Maximise utility by enabling DRS in automatic mode
  • Maximise hosts a given VM can run on
  • Ensure sufficient resources
  • Ensure critical VMs get what they need (HA restart priority – this doesn’t guarantee VM restart order)

There are 3 Admission Control Policies
1. Percentage of Cluster Resources

  • Often best choice
  • Maximises cluster resource use
  • Use when reservation varies considerably

2. Number of hosts policy (slot algorithm)

  • Maximises chances of restarting VMs with reservations
  • Avoids fragmentation
  • Often conservative – if this is a concern, use dedicated failover host policy.

3. Dedicated Failover Host(s)

  • Best if VMs have large resource reservations
  • Impact to VMs on other hosts minimised
  • Restart time can be longer
  • Failover hosts are idle prior to failure
  • If you use this, select largest hosts to be failover hosts.

Tech Previews – FT and HA
FT for zero-RPO, zero-RTO (The version in the beta)

  • 64GB RAM per protected VM
  • 4 vCPUs
  • Uses individual, separate storage (no longer shared) for Primary and Secondary VM
  • Complete “re-imagining” of how FT functioned.
  • Lockstep was hard to scale, now uses “incredibly fast check-pointing”.

HA – VM Component Protection

  • Problem: Host has storage connectivity issue – APD or PDL
  • Difficult to manage VMs
  • Approach: Move VMs to a healthier host
  • You can choose how you want to protect and under what circumstances (sounds a bit like Australia’s border protection policy – sigh).

Admission Control Fling – vRAS

  • Coming in the next month or two
  • vSphere Resource and Availability Service
  • Assess impact of host failures and VM migration on resources using DRS dump files
  • Provides sample what-if scenarios

And that’s it. Top session, very informative. 4 stars.

VMware ESX 3.5 Update 2 / Update 3

With all of the hullabaloo around the licensing issue in 3.5 Update 2, some people might have missed the fact that there have been some really neat improvements in the product.

One of my pet irritations is that a lot of environments have dysfunctional DNS. In the past, this has caused HA to break, unless we put entries in the hosts files of the ESX hosts and the VirtualCenter host. Starting with ESX/ESXi 3.5 Update 2, DNS resolution or /etc/hosts file entries are no longer required to configure VMware HA. It’s now down to VirtualCenter to do that for you. This is no excuse to not have functioning DNS in place, but it does make situations where the ESX environment is the first thing going in a little more palatable.

The Remote Command Line Interface (Remote CLI), which was previously supported only on ESXi (with support for ESX only in conjunction with Storage VMotion), is now fully supported for both ESXi 3.5 Update 2 and ESX 3.5 Update 2.

As of ESX/ESXi 3.5 Update 2, VMware now supports up to 192 logical (virtual) CPUs per host, provided the host has no more than 170 VMs, and there are no more than three virtual floppy devices or virtual CDROM devices configured on the host at any given time. This is cool, although unfortunately it’s not something I’ve had time to test in the lab.

Hot virtual disk extension support has been added to this release, which makes the CLI-averse a little more comfortable with virtual disk expansion. Hot extend is supported for monolithic disks in VMFS that do not have a VM snapshot.

So, there are some cool things, besides the obvious boneheaded error mentioned previously. But don’t get too excited, as I noticed that VirtualCenter 2.5 Update 3 has just been released. Get it here and have a look through the release notes here. This release seems to focus on resolving issues, of which there are many in VirtualCenter, and this is no bad thing. There is a FLEX license server upgrade included, but (and I imagine they’ve poached coders from Sun for this functionality) “the license server will not be automatically upgraded when using the VMware Infrastructure Management Installer to upgrade an existing installation”. No, that would be too easy. There’s a standalone installer instead. Hopefully this will decrease the number of license server calls I get, but it’s still running FLEX isn’t it?

There’s still a lot of issues, but it wouldn’t be enterprise software without that, now, would it? things like “Storage Devices Connected to McData FC Switch Through QLogic Adapters Occasionally Do Not Reappear After Reboot“, “Incorrect Device Paths for LUNs Displayed in Storage Summary“, “ESX Server Host Might Lose Access to iSCSI Targets from EMC CX3 Series Array“, and so on. The sort of stuff that might just do your head in if you don’t have relaxation strategies in place. And upgrading VMware Update Manager is still hinky, while the whole experience, new install or upgrade, reeks of components slapped together rather than “unified”. But it’s a reasonably complicated product, and I’m no software engineer, so I can live with it. Enjoy.