VMware Live Cyber Recovery Link-O-Rama

Background

It’s been just over six months since VMware Live Recovery was announced. This is an umbrella product name for VMware Live Site Recovery (formerly known as Site Recovery Manager) and VMware Live Cyber Recovery (formerly known as VMware Cloud Disaster Recovery and Ransomware Recovery). Like many of the changes at VMware this year, this is a move designed to streamline the disaster recovery solutions being made available for VMware Cloud Foundation, and to make it simpler for you to choose which product works best for you.

To that end, I’ve put together a series of links from the documentation that I’ve found useful when talking with customers about VMware Live Cyber Recovery in particular. I figured if my customers were asking about these things, someone else might find the answers useful too. This isn’t comprehensive, but hopefully it’s of some use.

 

Link-O-Rama

Maximum Configs

Here are the current configuration maximums for VMware Live Site Recovery, VMware Live Cyber Recovery, and local vSAN snapshots. Why would you need to know these? There are useful things to know that differentiate the products, such as the recovery point objective values, virtual machine limits, and so on. Remember, these should be treated as limits, not targets, and if you have an environment that exceeds these limits, reach out to your account team to talk about it.

Networking Considerations

  • Does your security team want to know more about the Cyber Recovery (formerly DRaaS) connector? Check out this document.
  • Can you use Direct Connect instead of clobbering my Internet feed? Yes, here’s how. Want to know more about AWS Direct Connect (useful if you’re new to AWS)? This article should help.
  • What holes do you need to punch through my firewall for things like support and telemetry data uploads? This article on Service Public IP Addresses should help some.
  • Can you throttle the bandwidth that the replication uses? Yes, you can. The documentation is here.
  • Need to configure different IP addresses for your recovered virtual machines. This documentation should help.

How Do I Use It?

You can find (a slightly dated) copy of the VMware Cloud Disaster Recovery Admin Guide in PDF format here. Some things have changed with the switch to VMware Live Cyber Recovery, but the fundamentals remain the same, and it’s handy to have a downloadable copy of the admin guide, particularly when you’re in a situation where your data centre is on fire and your Internet connection is a melted mess.

Want to get a better feel for how VMware Live Cyber Recovery handles? Check out this Hands-on Lab.

Release Notes

The overall release notes for VMware Live Recovery are available here. You’ll see interesting bits of news, such as “VMware Cloud Director Availability is now part of VMware Live Recovery”. You can also drill down into the VMware Live Cyber Recovery documentation here, and the VMware Live Site Recovery 9.0 documentation can be found here. If you need to dig through some historical release notes (or just want to take a trip down memory lane), you can view the VMware Cloud Disaster Recovery release notes archive here.

But I Have More Questions

  • There’s a great set of Frequently Asked Questions on VMware Live Recovery in PDF format that can be found here.
  • How do you know if you need disaster recovery or ransomware recovery (or both)? This is a great article covering some of the considerations that you’ll need to think about.
  • Are you looking at backup solutions for VMware Cloud on AWS? This is a comprehensive article on using third-party backup and recovery tools with VMware Cloud on AWS.
  • Looking to leverage VMware Site Recovery on VMware Cloud on AWS? It has gone away, and you should consider moving to VMware Live Cyber Recovery.
  • Wondering about the overall security of the VMware Live Cyber Recovery product? The Cloud Security Alliance performed a VCDR Assessment that you can access here.

 

Conclusion

And that’s about it. I’ve published a few posts on this weblog covering VMware Cloud Disaster Recovery in the past, and you can find them here. While some of the screens might look a little different now, the vibe remains unchanged. If you have questions that I haven’t answered here, feel free to reach out and I’ll see if I can help.

VMware Cloud on AWS – TMCHAM – Part 13 – Delete the SDDC

*Update* 2025/01/30 – The VMware docs site has been moved to Broadcom’s Techdocs site. I won’t be updating all of the VMware links on this weblog, but this post will get the treatment.

Following on from my article on host removal, in this edition of Things My Customers Have Asked Me (TMCHAM), I’m going to cover SDDC removal on the VMware-managed VMware Cloud on AWS platform. Don’t worry, I haven’t lost my mind in a post-acquisition world. Rather, this is some of the info you’ll find useful if you’ve been running a trial or a proof of concept (as opposed to a pilot) deployment of VMware Cloud Disaster Recovery (VCDR) and / or VMware Cloud on AWS and want to clean some stuff up when you’re all done.

 

Process

Firstly, if you’re using VCDR and want to deactivate the deployment, the steps to perform are outlined here, and I’ve copied the main bits from that page below.

  1. Remove all DRaaS Connectors from all protected sites. See Remove a DRaaS Connector from a Protected Site (new link here).
  2. Delete all recovery SDDCs. See Delete a Recovery SDDC (new link here).
  3. Deactivate the recovery region from the Global DR Console. (Do this step last.) See Deactivate a Recovery Region (new link here). Usage charges for VMware Cloud DR are not stopped until this step is completed.

Funnily enough, as I was writing this, someone zapped our lab for reasons. So this is what a Region deactivation looks like in the VCDR UI.

Note that it’s important you perform these steps in that order, or you’ll have more cleanup work to do to get everything looking nice and tidy. I have witnessed firsthand someone doing it the other way and it’s not pretty. Note also that if your Recovery SDDC had services such as HCX connected, you should hold off deleting the Recovery SDDC until you’ve cleaned that bit up.

Secondly, if you have other workloads deployed in a VMware Cloud on AWS SDDC and want to remove a PoC SDDC, there are a few steps that you will need to follow.

If you’ve been using HCX to test migrations or network extension, you’ll need to follow these steps (new link here) to remove it. Note that this should be initiated from the source side, and your HCX deployment should be in good order before you start (site pairings functioning, etc). You might also wish to remove a vCenter Cloud Gateway, and you can find information on that process here (new link here).

Finally, there are some AWS activities that you might want to undertake to clean everything up. These include:

  • Removing VIFs attached to your AWS VPC.
  • Deleting the VPC (this will likely be required if your organisation has a policy about how PoC deployments  are managed).
  • Tidy up and on-premises routing and firewall rules that may have been put in place for the PoC activity.

And that’s it. There’s not a lot to it, but tidying everything up after a PoC will ensure that you avoid any unexpected costs popping up in the future.

VMware Cloud Disaster Recovery – Using A Script VM

This is a quick post covering the steps required to configure a script VM for use in a recovery plan with VMware Cloud Disaster Recovery (VCDR). Why would you want to do this? You might be running a recovery for a Linux VM and you need to run a script to update the DNS settings of the VM once it’s powered on at another site. Or you might have a site-specific application that needs to be installed. Whatever. The point is that VCDR gives you that ability to do that via the Script VM. You can read the documentation on the feature here.

Firstly, you configure the Script VM as part of the Recovery Plan creation process. Specify the name of the VM and the vCenter it’s hosted on.

Under Recovery steps, click on Add Step to add a step to the recovery process.

When you add the step, you’ll want to add an action for the post-recovery phase.

You can then select “Run script on the Script VM”.

At this point you can specify the full path to the script file, keeping in mind that Windows looks different to Linux. You can also set a timeout for the script.

And that’s pretty much it. Remember that you’ll need working DNS, or, failing that, valid IP addresses for things to work.

VMware Cloud Disaster Recovery – Ransomware Recovery Activation

One of the cool features of VMware Cloud Disaster Recovery (VCDR) is the Enhanced Ransomware Recovery capability. This is a quick post to talk through how to turn it on in your VCDR environment, and things you need to consider.

 

Organization Settings

The first step is to enable the ransomware services integration in your VCDR dashboard. You’ll need to be an Organisation owner to do this. Go to Settings, and click on Ransomware Recovery Services.

You’ll then have the option to select where the data analysis is performed.

You’ll also need to tick some boxes to ensure that you understand that an appliance will be deployed in each of your Recovery SDDCs, Windows VMs will get a sensor installed, and some preinstalled sensors may clash with Carbon Black.

Click on Activate and it will take a few moments. If it takes much longer than that, you’ll need to talk to someone in support.

Once the analysis integration is activated, you can then activate NSX Advanced Firewall. Page 245 of the PDF documentation covers this better than I can, but note that NSX Advanced Firewall is a chargeable service (if you don’t already have a subscription attached to your Recovery SDDC). There’s some great documentation here on what you do and don’t have access to if you allow the activation of NSX Advanced Firewall.

Like your favourite TV chef would say, here’s one I’ve prepared earlier.

Recovery Plan Configuration

Once the services integration is done, you can configure Ransomware Recovery on a per Recovery Plan basis.

Start by selecting Activate ransomware recovery. You’ll then need to acknowledge that this is a chargeable feature.

You can also choose whether you want to use integrated analysis (i.e. Carbon Black Cloud), and if you want to manually remove other security sensors when you recover. You can, also, choose to use your own tools if you need to.

And that’s it from a configuration perspective. The actual recovery bit? A story for another time.

VMware Cloud Disaster Recovery – Firewall Ports

I published an article a while ago on getting started with VMware Cloud Disaster Recovery (VCDR). One thing I didn’t cover in any real depth was the connectivity requirements between on-premises and the VCDR service. VMware has worked pretty hard to ensure this is streamlined for users, but it’s still something you need to pay attention to. I was helping a client work through this process for a proof of concept recently and thought I’d cover it off more clearly here. The diagram below highlights the main components you need to look at, being:

  • The Cloud File System (frequently referred to as the SCFS)
  • The VMware Cloud DR SaaS Orchestrator (the Orchestrator); and
  • VMware Cloud DR Auto-support.

It’s important to note that the first two services are assigned IP addresses when you enable the service in the Cloud Service Console, and the Auto-support service has three public IP addresses that you need to be able to communicate with. All of this happens outbound over TCP 443. The Auto-support service is not required, but it is strongly recommended, as it makes troubleshooting issues with the service much easier, and provides VMware with an opportunity to proactively resolve cases. Network connectivity requirements are documented here.

[image courtesy of VMware]

So how do I know my firewall rules are working? The first sign that there might be a problem is that the DRaaS Connector deployment will fail to communicate with the Orchestrator at some point (usually towards the end), and you’ll see a message similar to the following. “ERROR! VMware Cloud DR authentication is not configured. Contact support.”

How can you troubleshoot the issue? Fortunately, we have a tool called the DRaaS Connector Connectivity Check CLI that you can run to check what’s not working. In this instance, we suspected an issue with outbound communication, and ran the following command on the console of the DRaaS Connector to check:

drc network test --scope cloud

This returned a status of “reachable” for the Orchestrator and Auto-support services, but the SCFS was unreachable. Some negotiations with the firewall team, and we were up and running.

Note, also, that VMware supports the use of proxy servers for communicating with Auto-support services, but I don’t believe we support the use of a proxy for Orchestrator and SCFS communications. If you’re worried about VCDR using up all your bandwidth, you can throttle it. Details on how to do that can be found here. We recommend a minimum of 100Mbps, but you can go as low as 20Mbps if required.

Updated Articles Page

I recently had the opportunity to run through a VMware Cloud on Disaster Recovery deployment with a customer and thought I’d run through the basics. It’s important to note that there a variety of topologies supported with VCDR, and many things that need to be considered before you click deploy, and this is just one way of doing it. In any case, there’s a new document outlining the process on the articles page.

VMware Cloud on AWS – TMCHAM – Part 5 – VM Management

In this edition of Things My Customers Have Asked Me (TMCHAM), I’m going to delve into some questions around managing VMs running on the VMware-managed VMware Cloud on AWS platform, and talk about vCenter plugins and what that looks like when you move across to VMware Cloud on AWS.

How Can I Access vCenter?

VMware vCenter has been around since Hector was a pup, and the good news is that it can be used to manage your VMware Cloud on AWS environment. It’s accessible via a few different methods, including PowerCLI. If you want to access the HTML5 UI via the cloud console, you’ll need to ensure there’s a firewall rule in place to allow access via your Management gateway – the official documentation is here. If the rule has already been created and you just need to add your IP to the mix, here’s the process.

The first step is to find out your public IP address. I use WhatIsMyIP.com to do this.

In your console, go to Networking & Security -> Inventory -> Groups.

Under Groups, make sure you select Management Groups.

You’ll find a Group that was created that stores the IP information of folks wanting to access vCenter. In this example, we’ve called it “SET Home IP Addresses”.

Click on the vertical ellipsis and click Edit.

Click on the IPs section.

You’ll then see a spot where you can enter your IP address. You can do a single address or enter a range, as shown below.

Click Apply and then click Save to save the rule. Now you should be able to open vCenter.

Can I run RVTools and other scripts on my VMC environment?

Yes, you can run RVTools against your environment. In terms of privilege levels with VMware Cloud on AWS, you get CloudAdmin. The level of access is outlined here. It’s important to understand these privilege levels, because some things will and won’t work as a result of these.

Can I lockdown my VMs using PowerShell?

You will have the ability to set these advanced settings on your VMs in the SDDC, but this is limited to per-VM, rather than on a per-cluster basis. So if you normally ran a script on a pre-VM basis to harden the VM config, you’d need to run that on each VM individually, rather than on a per-cluster level.

What about vCenter plugins?

We don’t have a concept of vCenter plugins in VMware Cloud on AWS, so there are different ways to get the information you’d normally need. vROps, for example, has the ability to look at VMware Cloud on AWS, using either the on-premises version or the cloud version. There’s information on that here, but note that the plugin isn’t supported with VMC vCenter.

What about my Site Recovery Manager plugin? The mechanism for managing this will change depending on whether you’re using SRaaS or VCDR to protect your workloads. There’s some good info on SRaaS here, and some decent VCDR information here. Again, there is no plugin available, but the element managers are available via the cloud console.  

What about NSX-V? VMware Cloud on AWS is all NSX-T, and you can access the NSX Manager via the cloud console.

Conclusion

A big part of the reason people like VMware Cloud on AWS is that the management experience doesn’t differ significantly from what you get VMware Cloud Foundation of VMware Validated Designs on-premises. That said, there are a few things that do change when you move to VMware Cloud on AWS. Things like plugins don’t exist, but you can still run many of the scripts you know and love against the platform. Remember, though, it is a fully managed service, so some of the stuff you used to run against your on-premises environment is no longer necessary.

VMware Cloud on AWS – TMCHAM – Part 2 – VCDR Notes

In this episode of “Things My Customers Have Asked Me” (or TMCHAM for short), I’m going to dive into a few questions around VMware Cloud Disaster Recovery (VCDR), a service we offer as an add-on to VMware Cloud on AWS. If you’re unfamiliar with VCDR, you can read a bit more about it here.

VCDR Roles and Permissions

Can RBAC roles be customised? Not really, as these are cascaded down from the Cloud Services hub. As I understand it, I don’t believe you have granular control over it, just the pre-defined, default roles as outlined here, so you need to be careful about what you hand out to folks in your organisation. To see what Service Roles have been assigned to your account, in the VMware Cloud Services, go to My Account, and then click on My Roles. Under Service Roles, you’ll see a list of services, such as VCDR, Skyline, and so on. You can then check what roles have been assigned. 

VCDR Protection Groups

VCDR Protection Groups are the way that we logically group together workloads to be protected with the same RPO, schedule, and retention. There are two types of protection group: standard-frequency and high-frequency. Standard-frequency snapshots can be run as often as every 4 hours, while high-frequency snapshots can go as often as every 30 minutes. You can read more on protection groups here. It’s important to note that there are some caveats to be aware of with high-frequency snapshots. These are outlined here.

30-minute RPOs were introduced in late 2021, but there are some caveats that you need to be aware of. Some of these are straightforward, such as the minimum software levels for on-premises protection. But you also need to be mindful that VMs with existing vSphere snapshots will not be included, and, more importantly, high-frequency snapshots can’t be quiesced.

Can you have a VM instance in both a standard- and high-frequency snapshot protection group?  Would this allow us to get the best of both worlds – e.g. RPO could be as low as 30 minutes, but with a guaranteed snapshot of 4 hours?  Once you do a high-frequency snap on a VM, it keeps using that mechanism thereafter, even if it sits in a protection group using standard protection. Note also that you set a schedule for a protection group, so you can have snapshots running ever 30 mins and kept for a particular period of time (customer selects this). You could also run snapshots at 4 hours and keep those for a period of time too. While you can technically have a VM in multiple groups, what you’re better off doing is configuring a variety of schedules for your protection groups to meet those different RPOs.

Quiesced Snapshots

What happens to a VM during a quiesced state – would we experience micro service outages? The best answer I can give is “it depends”. The process for the standard, quiesced snapshot is similar to the one described here. The VM will be stunned by the process, so depending on what kind of activity is happening on the VM, there may be a micro outage to the service.

Other Considerations

The documentation talks about not changing anything when a scheduled snapshot is being run – how do we manage configuration of the SDDC if jobs are running 24/7?  Seems odd that nothing can be changed when a scheduled snapshot is being run? This refers more to the VM that is being snapped. i.e. Don’t change configs or make changes to the environment, as that would impact this VM. It’s not a blanket rule for the whole environment. 

Like most things, success with VCDR relies heavily on understanding the outcomes your organisation wants to achieve, and then working backwards from there. It’s also important to understand that this is a great way to do DR, but not necessarily a great way to do standard backup and recovery activities. Hopefully this article helps clarify some of the questions folks have around VCDR, and if it doesn’t, please don’t hesitate to get in contact.