Tag Archives: howto

Backblaze – Resolve bzfileids.dat File Error

Posted on by

This is a quick one. I recently had this Backblaze error pop up on one of my Mac computers.

Turns out you don’t need to contact support to resolve it, but you do need to follow the instructions in this article. I’ve copied the steps below  so you don’t need to read the article if that’s all too much for you.

Revalidating your Backup Data (Mac)

  1. Navigate to https://secure.backblaze.com/update.htm.
  2. Download the installer for macOS.
    Warning
    Do not uninstall the Backblaze Client, and do not skip the next step.
  3. Restart your computer.
  4. When your computer is back up, open the installer and click Install Now.
  5. Restart your computer.
  6. Click the Backblaze icon in your menu bar to open the Backup Client.
  7. Select Preferences.
  8. Press and hold the Option key, and click Restore Options.
  9. Let the process run for three to four hours.

Why Does This Happen?

According to the article, “[i]f you back up an extremely large number of files, or if your computer has used the Backblaze Computer Backup software for a very long time, the bzfileids.dat file may become too large to be fully loaded into RAM, prompting an error message”. I think I’ve been doing both, so there you go.

QNAP – Add Cache With SSDs

Posted on by

I recently added some cache to my QNAP NAS at home and thought I’d document the process here. It’s pretty straightforward, but there are nonetheless some things you need to take into consideration. Also, this is more about me satisfying my curiosity. I’m not suggesting you’ll real benefits (particularly the way I’m using it).

 

The Hardware

Firstly, my NAS is a TS-831X model, with 8 drive bays and the ability to add in a small form-factor PCIe card. As all of the bays are already occupied with spinning disks, I opted to use the PCIe solution. To that end, I purchased a QM2-2S-220A Dual M.2 22110/2280 SATA SSD expansion card from a local supplier. It’s relatively inexpensive, and you get some choice in terms of the storage you install. It also comes with low-profile and normal baseplates as well, so you could load this in a PC if you’re into that sort of thing. Since I’ve been using Apple gear for a while, I’m a little rusty with the screwdriver, but installation was straightforward. Remove the heatsink and fan, add some thermal pads to your M.2 drives, and you’re good to go.

The card slots into the single PCIe slot in the NAS.

You’ll notice I went with the cheapest 1TB M.2 drives I could find on Amazon. Time will tell (probably sooner rather than later) whether that was really a wise choice.

 

The Software

Once the NAS has booted up again (and hopefully with no errors), you’re good to start the software configuration. The QNAP documentation covering the process can be found here. While I’m running version 5.2 of QTS, note that other versions may look a little different. There are some things to consider with regard to the amount of RAM you have installed, the version of software you’re running, and the amount of cache you have installed.

To get started, click on Storage & Snapshots.

You’ll see that I have the main enclosure (with 8 drives) and a USB-C enclosure with 4 drives. I also now have two SSDs installed on the QM2 card.

Click on Cache Acceleration to get started. Then click on the big + sign.

You’re then presented with a wizard that will guide you through the configuration process.

The first thing you’ll need to do is select the disks you want to use for your cache. Note that you can’t select any disks that are already in use.

Once you’ve selected the two disks you want to use, you’ll see that the RAID Type changes to RAID 0. You can use this option if you’re only going to configure read cache.

In this example, however, I’m going to configure the cache as read-write.

As I’m configuring the cache as read-write, I need to set the RAID Type to RAID 1, as I don’t want to lose writes if one of my cache devices fails. Once that’s done, click on Next to continue.

On this screen you can configure SSD Over-provisioning, Cache Mode, and Advanced Settings. If you’re confused about the percentage you should set for over-provisioning, there is a tool available in QTS that you can use (the SSD Profiling Tool). Or you can just click click next like I did.

In this example, I’m changing the Cache Mode to All I/O, as this NAS is used primarily for streaming media. I’ll talk about whether it really makes a difference further down.

Under Advanced Settings, you can configure the Bypass Block Size setting (this only comes into play if your Cache Mode is set to Random I/O) and the Cache replacement policy. Click on Next to continue.

You can now select which volume can use the SSD cache. In this example, even though I have two volumes configured, only the internal one can leverage the cache as it’s configured as read-write. If I wanted to use cache with my external volume, I’d need to set the cache to read-only. Click on Next to continue.

You’ll then be presented with a summary of your configuration options. If you’re happy with how it looks, click on Create.

You’ll then be presented with some warnings regarding the cache creation and removal considerations.

Click on I Understand and OK to proceed.

Once you’ve finished you’ll see that the Cache is set to Initializing. It doesn’t take a terribly long time to complete.

And now you’re done and ready to enjoy that cache goodness.

 

Thoughts

Adding a Terabyte of SSD Cache to my home NAS was a fairly simple process, and relatively inexpensive to boot. I haven’t had much chance to test the performance yet, but I have noticed that the UI in QTS is reporting slightly higher CPU usage than I’d seen previously. While the process is easy, I’m not entirely convinced that I’ll see any real benefits, given that I’m serving up video files over SMB to a Plex server using a gigabit network. Maybe if I had a 10GbE running in the house I’d see it. And the workload has a bit to do with it too. If I was running Virtual Machines on this machine I think I’d see something good from a Random I/O profile, particularly with data coming off the SATA drives. Why didn’t I use Qtier? Because I have static volumes configured, not storage pools. And SSD Caching seemed the easiest thing to do without tearing it all down and starting from scratch. In any case, it was a fun experiment, and I’ll be curious to see whether I notice any real improvements and how long the drives last with normal use.

Random Short Take #98

Posted on by

Welcome to Random Short Take #98. Some of these links have been in my queue for a little while, so apologies if it’s not as fresh as it could be. Let’s get random.

  • Let’s start off with a brilliant article from JB “Going tharn“. We build all these systems to get stuff done, and when it goes wrong it can go ever so wrong. To quote the article, “[t]hat is the terrible power of modern systems, I think. They make us alone, atomise us, and disconnect us from each other. They turn us tharn. They replace those human connections with a User-Platform relationship that feels remarkably similar to the old master-servant relationship in its power imbalance.” I haven’t read something that resonated with me as much as that in quite some time.
  • My friend Damo is heading off on esCarpade later this week, raising money for Camp Quality. If you have some of the folding stuff lying around I urge you to support him.
  • Which disk format? Howard Oakley asks (and answers) the question so I don’t have to ask it anymore.
  • Here are some of the announcements from NetApp Insight 2024.
  • I enjoyed Tom’s article “Semper Gumby“. It’s important to try and be as flexible as possible, even when you get old and crotchety like I am.
  • Preston has started “Preston’s Assorted musings (Episode 1?)” and I believe it’s already a series. Worth checking out if only for the fact that he puts some thought into it rather than just spraying links to random articles everywhere like I do.
  • Paolo put together a handy article on how to configure the Object First VSA as a Veeam Repository if you’re into that kind of thing.
  • Over at El Reg there’s an article on how your Smart TV sucks. And it’s tracking you. No surprises there, but look into blocking yours with a pi-hole if you can (or however you like to restrict access to things that shouldn’t be talking to the big, wide world).

VMware Cloud on AWS – TMCHAM – Part 13 – Delete the SDDC

Posted on by

*Update* 2025/01/30 – The VMware docs site has been moved to Broadcom’s Techdocs site. I won’t be updating all of the VMware links on this weblog, but this post will get the treatment.

Following on from my article on host removal, in this edition of Things My Customers Have Asked Me (TMCHAM), I’m going to cover SDDC removal on the VMware-managed VMware Cloud on AWS platform. Don’t worry, I haven’t lost my mind in a post-acquisition world. Rather, this is some of the info you’ll find useful if you’ve been running a trial or a proof of concept (as opposed to a pilot) deployment of VMware Cloud Disaster Recovery (VCDR) and / or VMware Cloud on AWS and want to clean some stuff up when you’re all done.

 

Process

Firstly, if you’re using VCDR and want to deactivate the deployment, the steps to perform are outlined here, and I’ve copied the main bits from that page below.

  1. Remove all DRaaS Connectors from all protected sites. See Remove a DRaaS Connector from a Protected Site (new link here).
  2. Delete all recovery SDDCs. See Delete a Recovery SDDC (new link here).
  3. Deactivate the recovery region from the Global DR Console. (Do this step last.) See Deactivate a Recovery Region (new link here). Usage charges for VMware Cloud DR are not stopped until this step is completed.

Funnily enough, as I was writing this, someone zapped our lab for reasons. So this is what a Region deactivation looks like in the VCDR UI.

Note that it’s important you perform these steps in that order, or you’ll have more cleanup work to do to get everything looking nice and tidy. I have witnessed firsthand someone doing it the other way and it’s not pretty. Note also that if your Recovery SDDC had services such as HCX connected, you should hold off deleting the Recovery SDDC until you’ve cleaned that bit up.

Secondly, if you have other workloads deployed in a VMware Cloud on AWS SDDC and want to remove a PoC SDDC, there are a few steps that you will need to follow.

If you’ve been using HCX to test migrations or network extension, you’ll need to follow these steps (new link here) to remove it. Note that this should be initiated from the source side, and your HCX deployment should be in good order before you start (site pairings functioning, etc). You might also wish to remove a vCenter Cloud Gateway, and you can find information on that process here (new link here).

Finally, there are some AWS activities that you might want to undertake to clean everything up. These include:

  • Removing VIFs attached to your AWS VPC.
  • Deleting the VPC (this will likely be required if your organisation has a policy about how PoC deployments  are managed).
  • Tidy up and on-premises routing and firewall rules that may have been put in place for the PoC activity.

And that’s it. There’s not a lot to it, but tidying everything up after a PoC will ensure that you avoid any unexpected costs popping up in the future.

VMware Cloud on AWS – TMCHAM – Part 12 – Host Removal

Posted on by

In this edition of Things My Customers Have Asked Me (TMCHAM), I’m going to cover host removal on the VMware-managed VMware Cloud on AWS platform. This is a fairly brief post, as there’s not a lot to say about the process, but I’ve had enough questions about that I thought it was worth covering.

 

Background

I’ve written about Elastic DRS (EDRS) in VMware Cloud on AWS previously. It’s something you can’t turn off, and the Baseline policy does a good job of making sure you don’t get in hot water from a storage perspective. That said, there might be occasions where you want to remove a host or two to scale in your cluster manually. This might happen after a cluster conversion, or you may have had a rapid scale out event and you have now removed whatever workloads caused that scale out event to occur.

 

Process

The process to remove a host is documented here. Note that it is a sequential process, with one host being removed at a time. Depending on the number of hosts in your cluster, you may need to adjust your storage and fault tolerance policies as well. To start the process, go to your cloud console and select the SDDC you want to remove the hosts from. If there’s only one cluster, you can click on Remove Hosts under Actions. If there are multiple clusters in the SDDC, you’ll need to select the cluster you want to remove the host from.

You’ll then be advised that you need to understand what you’re doing (and acknowledge that), and you may be advised to change your default storage policies as well. More info on those policies is here.

Once you kick off the process, the cluster will be evaluated to ensure that removing hosts will not violate the applicable EDRS policies. VMs will be migrated off the host when it’s put into maintenance mode, and billing will be stopped for that host.

And that’s it. Pretty straightforward.

VMware Cloud Disaster Recovery – Using A Script VM

Posted on by

This is a quick post covering the steps required to configure a script VM for use in a recovery plan with VMware Cloud Disaster Recovery (VCDR). Why would you want to do this? You might be running a recovery for a Linux VM and you need to run a script to update the DNS settings of the VM once it’s powered on at another site. Or you might have a site-specific application that needs to be installed. Whatever. The point is that VCDR gives you that ability to do that via the Script VM. You can read the documentation on the feature here.

Firstly, you configure the Script VM as part of the Recovery Plan creation process. Specify the name of the VM and the vCenter it’s hosted on.

Under Recovery steps, click on Add Step to add a step to the recovery process.

When you add the step, you’ll want to add an action for the post-recovery phase.

You can then select “Run script on the Script VM”.

At this point you can specify the full path to the script file, keeping in mind that Windows looks different to Linux. You can also set a timeout for the script.

And that’s pretty much it. Remember that you’ll need working DNS, or, failing that, valid IP addresses for things to work.

VMware Cloud on AWS – Check TRIM/UNMAP

Posted on by

This a really quick follow up to one of my TMCHAM articles on TRIM/UNMAP on VMware Cloud on AWS. In short, a customer wanted to know whether TRIM/UNMAP had been enabled on one of their clusters, as they’d requested. The good news is it’s easy enough to find out. On your cluster, go to Configure. Under vSAN, you’ll see Services. Expand the Advanced Options section and you’ll see whether TRIM/UNMAP has been enabled for the cluster or not.

VMware Cloud Disaster Recovery – Ransomware Recovery Activation

Posted on by

One of the cool features of VMware Cloud Disaster Recovery (VCDR) is the Enhanced Ransomware Recovery capability. This is a quick post to talk through how to turn it on in your VCDR environment, and things you need to consider.

 

Organization Settings

The first step is to enable the ransomware services integration in your VCDR dashboard. You’ll need to be an Organisation owner to do this. Go to Settings, and click on Ransomware Recovery Services.

You’ll then have the option to select where the data analysis is performed.

You’ll also need to tick some boxes to ensure that you understand that an appliance will be deployed in each of your Recovery SDDCs, Windows VMs will get a sensor installed, and some preinstalled sensors may clash with Carbon Black.

Click on Activate and it will take a few moments. If it takes much longer than that, you’ll need to talk to someone in support.

Once the analysis integration is activated, you can then activate NSX Advanced Firewall. Page 245 of the PDF documentation covers this better than I can, but note that NSX Advanced Firewall is a chargeable service (if you don’t already have a subscription attached to your Recovery SDDC). There’s some great documentation here on what you do and don’t have access to if you allow the activation of NSX Advanced Firewall.

Like your favourite TV chef would say, here’s one I’ve prepared earlier.

Recovery Plan Configuration

Once the services integration is done, you can configure Ransomware Recovery on a per Recovery Plan basis.

Start by selecting Activate ransomware recovery. You’ll then need to acknowledge that this is a chargeable feature.

You can also choose whether you want to use integrated analysis (i.e. Carbon Black Cloud), and if you want to manually remove other security sensors when you recover. You can, also, choose to use your own tools if you need to.

And that’s it from a configuration perspective. The actual recovery bit? A story for another time.

VMware Cloud Disaster Recovery – Firewall Ports

Posted on by

I published an article a while ago on getting started with VMware Cloud Disaster Recovery (VCDR). One thing I didn’t cover in any real depth was the connectivity requirements between on-premises and the VCDR service. VMware has worked pretty hard to ensure this is streamlined for users, but it’s still something you need to pay attention to. I was helping a client work through this process for a proof of concept recently and thought I’d cover it off more clearly here. The diagram below highlights the main components you need to look at, being:

  • The Cloud File System (frequently referred to as the SCFS)
  • The VMware Cloud DR SaaS Orchestrator (the Orchestrator); and
  • VMware Cloud DR Auto-support.

It’s important to note that the first two services are assigned IP addresses when you enable the service in the Cloud Service Console, and the Auto-support service has three public IP addresses that you need to be able to communicate with. All of this happens outbound over TCP 443. The Auto-support service is not required, but it is strongly recommended, as it makes troubleshooting issues with the service much easier, and provides VMware with an opportunity to proactively resolve cases. Network connectivity requirements are documented here.

[image courtesy of VMware]

So how do I know my firewall rules are working? The first sign that there might be a problem is that the DRaaS Connector deployment will fail to communicate with the Orchestrator at some point (usually towards the end), and you’ll see a message similar to the following. “ERROR! VMware Cloud DR authentication is not configured. Contact support.”

How can you troubleshoot the issue? Fortunately, we have a tool called the DRaaS Connector Connectivity Check CLI that you can run to check what’s not working. In this instance, we suspected an issue with outbound communication, and ran the following command on the console of the DRaaS Connector to check:

drc network test --scope cloud

This returned a status of “reachable” for the Orchestrator and Auto-support services, but the SCFS was unreachable. Some negotiations with the firewall team, and we were up and running.

Note, also, that VMware supports the use of proxy servers for communicating with Auto-support services, but I don’t believe we support the use of a proxy for Orchestrator and SCFS communications. If you’re worried about VCDR using up all your bandwidth, you can throttle it. Details on how to do that can be found here. We recommend a minimum of 100Mbps, but you can go as low as 20Mbps if required.

QNAP – Expand Volume With Larger Drives

Posted on by

This is one of those articles I’ve been meaning to post for a while, simply because I forget every time I do it how to do it properly. One way to expand the capacity of your QNAP NAS (non-disruptively) is to replace the drives one at a time with larger capacity drives. It’s recommended that you follow this process, rather than just ripping the drives out one by one and waiting for the RAID Group to expand. It’s a simple enough process to follow, although the QNAP UI has always struck me as a little on the confusing side to navigate, so I took some pictures. Note that this was done on QNAP firmware 5.0.0.1986.

Firstly, go to Storage/Snapshots under Storage in the ControlPanel. Click on Manage.

Select the Storage Pool you want to expand, and click on Manage again.

This will give you a drop-down menu. Select Replace Disks One by One.

Now select the disk you want to replace and click on Change.

Once you’ve done this for all of the disks (and it will take some time to rebuild depending on a variety of factors), click on Expand Capacity. It will ask you if you’re sure and hopefully you’ll click OK.

It’ll take a while for the RAID Group to synchronise.

You’ll notice then that, while the Storage Pool has expanded, the Volume is still the original size. Select the Volume and click on Manage.

Now you can click on Resize Volume.

The Wizard will give you information on the Storage Pool capacity and give you the option to set the new capacity of the volume. I usually click on Set to Max.

It will warn you about that. Click on OK because you like to live on the edge.

It will take a little while, but eventually your Volume will have expanded to fill the space.