VMware Cloud on AWS – TMCHAM – Part 13 – Delete the SDDC

Following on from my article on host removal, in this edition of Things My Customers Have Asked Me (TMCHAM), I’m going to cover SDDC removal on the VMware-managed VMware Cloud on AWS platform. Don’t worry, I haven’t lost my mind in a post-acquisition world. Rather, this is some of the info you’ll find useful if you’ve been running a trial or a proof of concept (as opposed to a pilot) deployment of VMware Cloud Disaster Recovery (VCDR) and / or VMware Cloud on AWS and want to clean some stuff up when you’re all done.

 

Process

Firstly, if you’re using VCDR and want to deactivate the deployment, the steps to perform are outlined here, and I’ve copied the main bits from that page below.

  1. Remove all DRaaS Connectors from all protected sites. See Remove a DRaaS Connector from a Protected Site.
  2. Delete all recovery SDDCs. See Delete a Recovery SDDC.
  3. Deactivate the recovery region from the Global DR Console. (Do this step last.) See Deactivate a Recovery Region. Usage charges for VMware Cloud DR are not stopped until this step is completed.

Funnily enough, as I was writing this, someone zapped our lab for reasons. So this is what a Region deactivation looks like in the VCDR UI.

Note that it’s important you perform these steps in that order, or you’ll have more cleanup work to do to get everything looking nice and tidy. I have witnessed firsthand someone doing it the other way and it’s not pretty. Note also that if your Recovery SDDC had services such as HCX connected, you should hold off deleting the Recovery SDDC until you’ve cleaned that bit up.

Secondly, if you have other workloads deployed in a VMware Cloud on AWS SDDC and want to remove a PoC SDDC, there are a few steps that you will need to follow.

If you’ve been using HCX to test migrations or network extension, you’ll need to follow these steps to remove it. Note that this should be initiated from the source side, and your HCX deployment should be in good order before you start (site pairings functioning, etc). You might also wish to remove a vCenter Cloud Gateway, and you can find information on that process here.

Finally, there are some AWS activities that you might want to undertake to clean everything up. These include:

  • Removing VIFs attached to your AWS VPC.
  • Deleting the VPC (this will likely be required if your organisation has a policy about how PoC deployments  are managed).
  • Tidy up and on-premises routing and firewall rules that may have been put in place for the PoC activity.

And that’s it. There’s not a lot to it, but tidying everything up after a PoC will ensure that you avoid any unexpected costs popping up in the future.

VMware Cloud Disaster Recovery – Using A Script VM

This is a quick post covering the steps required to configure a script VM for use in a recovery plan with VMware Cloud Disaster Recovery (VCDR). Why would you want to do this? You might be running a recovery for a Linux VM and you need to run a script to update the DNS settings of the VM once it’s powered on at another site. Or you might have a site-specific application that needs to be installed. Whatever. The point is that VCDR gives you that ability to do that via the Script VM. You can read the documentation on the feature here.

Firstly, you configure the Script VM as part of the Recovery Plan creation process. Specify the name of the VM and the vCenter it’s hosted on.

Under Recovery steps, click on Add Step to add a step to the recovery process.

When you add the step, you’ll want to add an action for the post-recovery phase.

You can then select “Run script on the Script VM”.

At this point you can specify the full path to the script file, keeping in mind that Windows looks different to Linux. You can also set a timeout for the script.

And that’s pretty much it. Remember that you’ll need working DNS, or, failing that, valid IP addresses for things to work.

VMware Cloud Disaster Recovery – Ransomware Recovery Activation

One of the cool features of VMware Cloud Disaster Recovery (VCDR) is the Enhanced Ransomware Recovery capability. This is a quick post to talk through how to turn it on in your VCDR environment, and things you need to consider.

 

Organization Settings

The first step is to enable the ransomware services integration in your VCDR dashboard. You’ll need to be an Organisation owner to do this. Go to Settings, and click on Ransomware Recovery Services.

You’ll then have the option to select where the data analysis is performed.

You’ll also need to tick some boxes to ensure that you understand that an appliance will be deployed in each of your Recovery SDDCs, Windows VMs will get a sensor installed, and some preinstalled sensors may clash with Carbon Black.

Click on Activate and it will take a few moments. If it takes much longer than that, you’ll need to talk to someone in support.

Once the analysis integration is activated, you can then activate NSX Advanced Firewall. Page 245 of the PDF documentation covers this better than I can, but note that NSX Advanced Firewall is a chargeable service (if you don’t already have a subscription attached to your Recovery SDDC). There’s some great documentation here on what you do and don’t have access to if you allow the activation of NSX Advanced Firewall.

Like your favourite TV chef would say, here’s one I’ve prepared earlier.

Recovery Plan Configuration

Once the services integration is done, you can configure Ransomware Recovery on a per Recovery Plan basis.

Start by selecting Activate ransomware recovery. You’ll then need to acknowledge that this is a chargeable feature.

You can also choose whether you want to use integrated analysis (i.e. Carbon Black Cloud), and if you want to manually remove other security sensors when you recover. You can, also, choose to use your own tools if you need to.

And that’s it from a configuration perspective. The actual recovery bit? A story for another time.

VMware Cloud Disaster Recovery – Firewall Ports

I published an article a while ago on getting started with VMware Cloud Disaster Recovery (VCDR). One thing I didn’t cover in any real depth was the connectivity requirements between on-premises and the VCDR service. VMware has worked pretty hard to ensure this is streamlined for users, but it’s still something you need to pay attention to. I was helping a client work through this process for a proof of concept recently and thought I’d cover it off more clearly here. The diagram below highlights the main components you need to look at, being:

  • The Cloud File System (frequently referred to as the SCFS)
  • The VMware Cloud DR SaaS Orchestrator (the Orchestrator); and
  • VMware Cloud DR Auto-support.

It’s important to note that the first two services are assigned IP addresses when you enable the service in the Cloud Service Console, and the Auto-support service has three public IP addresses that you need to be able to communicate with. All of this happens outbound over TCP 443. The Auto-support service is not required, but it is strongly recommended, as it makes troubleshooting issues with the service much easier, and provides VMware with an opportunity to proactively resolve cases. Network connectivity requirements are documented here.

[image courtesy of VMware]

So how do I know my firewall rules are working? The first sign that there might be a problem is that the DRaaS Connector deployment will fail to communicate with the Orchestrator at some point (usually towards the end), and you’ll see a message similar to the following. “ERROR! VMware Cloud DR authentication is not configured. Contact support.”

How can you troubleshoot the issue? Fortunately, we have a tool called the DRaaS Connector Connectivity Check CLI that you can run to check what’s not working. In this instance, we suspected an issue with outbound communication, and ran the following command on the console of the DRaaS Connector to check:

drc network test --scope cloud

This returned a status of “reachable” for the Orchestrator and Auto-support services, but the SCFS was unreachable. Some negotiations with the firewall team, and we were up and running.

Note, also, that VMware supports the use of proxy servers for communicating with Auto-support services, but I don’t believe we support the use of a proxy for Orchestrator and SCFS communications. If you’re worried about VCDR using up all your bandwidth, you can throttle it. Details on how to do that can be found here. We recommend a minimum of 100Mbps, but you can go as low as 20Mbps if required.

Updated Articles Page

I recently had the opportunity to run through a VMware Cloud on Disaster Recovery deployment with a customer and thought I’d run through the basics. It’s important to note that there a variety of topologies supported with VCDR, and many things that need to be considered before you click deploy, and this is just one way of doing it. In any case, there’s a new document outlining the process on the articles page.

VMware Cloud on AWS – TMCHAM – Part 5 – VM Management

In this edition of Things My Customers Have Asked Me (TMCHAM), I’m going to delve into some questions around managing VMs running on the VMware-managed VMware Cloud on AWS platform, and talk about vCenter plugins and what that looks like when you move across to VMware Cloud on AWS.

How Can I Access vCenter?

VMware vCenter has been around since Hector was a pup, and the good news is that it can be used to manage your VMware Cloud on AWS environment. It’s accessible via a few different methods, including PowerCLI. If you want to access the HTML5 UI via the cloud console, you’ll need to ensure there’s a firewall rule in place to allow access via your Management gateway – the official documentation is here. If the rule has already been created and you just need to add your IP to the mix, here’s the process.

The first step is to find out your public IP address. I use WhatIsMyIP.com to do this.

In your console, go to Networking & Security -> Inventory -> Groups.

Under Groups, make sure you select Management Groups.

You’ll find a Group that was created that stores the IP information of folks wanting to access vCenter. In this example, we’ve called it “SET Home IP Addresses”.

Click on the vertical ellipsis and click Edit.

Click on the IPs section.

You’ll then see a spot where you can enter your IP address. You can do a single address or enter a range, as shown below.

Click Apply and then click Save to save the rule. Now you should be able to open vCenter.

Can I run RVTools and other scripts on my VMC environment?

Yes, you can run RVTools against your environment. In terms of privilege levels with VMware Cloud on AWS, you get CloudAdmin. The level of access is outlined here. It’s important to understand these privilege levels, because some things will and won’t work as a result of these.

Can I lockdown my VMs using PowerShell?

You will have the ability to set these advanced settings on your VMs in the SDDC, but this is limited to per-VM, rather than on a per-cluster basis. So if you normally ran a script on a pre-VM basis to harden the VM config, you’d need to run that on each VM individually, rather than on a per-cluster level.

What about vCenter plugins?

We don’t have a concept of vCenter plugins in VMware Cloud on AWS, so there are different ways to get the information you’d normally need. vROps, for example, has the ability to look at VMware Cloud on AWS, using either the on-premises version or the cloud version. There’s information on that here, but note that the plugin isn’t supported with VMC vCenter.

What about my Site Recovery Manager plugin? The mechanism for managing this will change depending on whether you’re using SRaaS or VCDR to protect your workloads. There’s some good info on SRaaS here, and some decent VCDR information here. Again, there is no plugin available, but the element managers are available via the cloud console.  

What about NSX-V? VMware Cloud on AWS is all NSX-T, and you can access the NSX Manager via the cloud console.

Conclusion

A big part of the reason people like VMware Cloud on AWS is that the management experience doesn’t differ significantly from what you get VMware Cloud Foundation of VMware Validated Designs on-premises. That said, there are a few things that do change when you move to VMware Cloud on AWS. Things like plugins don’t exist, but you can still run many of the scripts you know and love against the platform. Remember, though, it is a fully managed service, so some of the stuff you used to run against your on-premises environment is no longer necessary.

VMware Cloud on AWS – TMCHAM – Part 2 – VCDR Notes

In this episode of “Things My Customers Have Asked Me” (or TMCHAM for short), I’m going to dive into a few questions around VMware Cloud Disaster Recovery (VCDR), a service we offer as an add-on to VMware Cloud on AWS. If you’re unfamiliar with VCDR, you can read a bit more about it here.

VCDR Roles and Permissions

Can RBAC roles be customised? Not really, as these are cascaded down from the Cloud Services hub. As I understand it, I don’t believe you have granular control over it, just the pre-defined, default roles as outlined here, so you need to be careful about what you hand out to folks in your organisation. To see what Service Roles have been assigned to your account, in the VMware Cloud Services, go to My Account, and then click on My Roles. Under Service Roles, you’ll see a list of services, such as VCDR, Skyline, and so on. You can then check what roles have been assigned. 

VCDR Protection Groups

VCDR Protection Groups are the way that we logically group together workloads to be protected with the same RPO, schedule, and retention. There are two types of protection group: standard-frequency and high-frequency. Standard-frequency snapshots can be run as often as every 4 hours, while high-frequency snapshots can go as often as every 30 minutes. You can read more on protection groups here. It’s important to note that there are some caveats to be aware of with high-frequency snapshots. These are outlined here.

30-minute RPOs were introduced in late 2021, but there are some caveats that you need to be aware of. Some of these are straightforward, such as the minimum software levels for on-premises protection. But you also need to be mindful that VMs with existing vSphere snapshots will not be included, and, more importantly, high-frequency snapshots can’t be quiesced.

Can you have a VM instance in both a standard- and high-frequency snapshot protection group?  Would this allow us to get the best of both worlds – e.g. RPO could be as low as 30 minutes, but with a guaranteed snapshot of 4 hours?  Once you do a high-frequency snap on a VM, it keeps using that mechanism thereafter, even if it sits in a protection group using standard protection. Note also that you set a schedule for a protection group, so you can have snapshots running ever 30 mins and kept for a particular period of time (customer selects this). You could also run snapshots at 4 hours and keep those for a period of time too. While you can technically have a VM in multiple groups, what you’re better off doing is configuring a variety of schedules for your protection groups to meet those different RPOs.

Quiesced Snapshots

What happens to a VM during a quiesced state – would we experience micro service outages? The best answer I can give is “it depends”. The process for the standard, quiesced snapshot is similar to the one described hereThe VM will be stunned by the process, so depending on what kind of activity is happening on the VM, there may be a micro outage to the service.

Other Considerations

The documentation talks about not changing anything when a scheduled snapshot is being run – how do we manage configuration of the SDDC if jobs are running 24/7?  Seems odd that nothing can be changed when a scheduled snapshot is being run? This refers more to the VM that is being snapped. i.e. Don’t change configs or make changes to the environment, as that would impact this VM. It’s not a blanket rule for the whole environment. 

Like most things, success with VCDR relies heavily on understanding the outcomes your organisation wants to achieve, and then working backwards from there. It’s also important to understand that this is a great way to do DR, but not necessarily a great way to do standard backup and recovery activities. Hopefully this article helps clarify some of the questions folks have around VCDR, and if it doesn’t, please don’t hesitate to get in contact.