OT – Upgrading From macOS Mojave To Catalina (The Hard Way)

This post is really about the boring stuff I do when I have a day off and isn’t terribly exciting. TL;DR I had some problems upgrading to Catalina, and had to start from scratch.

 

Background

I’ve had an Apple Mac since around 2008. I upgraded from a 24″ iMac to a 27″ iMac and was super impressed with the process of migrating between machines, primarily because of Time Machine’s ability to recover settings, applications, and data in a fairly seamless fashion. I can’t remember what version of macOS I started with (maybe Leopard?), but I’ve moved steadily through the last few versions with a minimal amount of fuss. I was running Mojave on my iMac late last year when I purchased a refurbished 2018 Mac mini. At the time, I decided not to upgrade to Catalina, as I’d had a few issues with my work laptop and didn’t need the aggravation. So I migrated from the iMac to the Mac mini and kept on keeping on with Mojave.

Fast forward to a April this year, and the Mac mini gave up the ghost. With Apple shutting down its stores here in response to COVID-19, it was a 2 week turnaround at the local repair place to get the machine fixed. In the meantime, I was able to use Time Machine to load everything on a 2012 MacBook Pro that was being used sparingly. It was a bit clunky, but had an internal SSD and 16GB of RAM, so it could handle the basics pretty comfortably. When the Mac mini was repaired, I used Time Machine once again to move everything back. It’s important to note that this is everything (settings, applications, and data) that had been accumulated since 2008. So there’s a bit of cruft associated with this build. A bunch of 32-bit applications that I’d lost track of, widgets that were no longer really in use, and so on.

 

The Big Update

I took the day off on Friday last week. I’d been working a lot of hours since COVID-19 restrictions kicked in here, and I’d been filling my commuting time with day job work (sorry blog!). I thought it would be fun to upgrade the Mac mini to Catalina. I felt that things were in a reasonable enough state that I could work with what it had to offer, and I get twitchy when there’s an upgrade notification on the Settings icon. Just sitting there, taunting me.

I downloaded the installer and pressed on. No dice, my system volume wasn’t formatted with APFS. How could this be? Well, even though APFS has been around for a little while now, I’d been moving my installation across various machines. At the time when the APFS conversion was part of the macOS upgrade, I was running an iMac with a spinning disk as the system volume, and so it never prompted to do that upgrade. When I moved to the Mac mini, I didn’t do any macOS upgrade, so I guess it just kept working with the HFS+ volume. It seems a bit weird that Catalina doesn’t offer a workaround for this, but I may just have been looking in the wrong place. Now, there was a lot of chatter in the forums about rebooting into Recovery Mode and converting the drive to an APFS volume. No matter what I tried, I was unable to do this effectively (either using the Recovery Mode console with Mojave or with Catalina booting from USB). I followed articles like this one but just didn’t have the same experience. And when I erased the system drive and attempted to recover from Time Machine backups, it would re-erase the volume as HFS+. So, I don’t know, I guess I’m an idiot. The solution that finally worked for me was to erase the drive, format it as APFS, install Mojave from scratch, and recover from a Time Machine backup. Unfortunately, though, this seemed to only want to transfer around 800KB of settings data. The normal “wait a few hours while we copy your stuff” just didn’t happen. Sod knows why, but what I did know was that I was really wasting my day off with this stuff.

I also ran in to an issue trying to do the installation from USB. You can read about booting from external devices and the T2 security chip here, here, and here. I lost patience with the process and took a different approach.

 

Is That So Bad?

Not really. I have my Photos library and iTunes media on a separate volume. I have one email account that we have used POP with over the years, but I installed Thunderbird, recovered the profile from my Time Machine data, and modified profiles.ini to point to that profile (causing some flashbacks to my early days on a help desk supporting a Netscape user base). The other thing I had to do was recover my Plex database. You can read more on that here. It actually went reasonably well. I’d been storing my iPhone backups on a separate volume too, and had to follow this process to relocate those backup files. Otherwise, Microsoft, to their credit, has made the reinstallation process super simple with Microsoft 365. Once I had most everything setup again, I was able to perform the upgrade to Catalina.

 

Conclusion

If this process sounds like it was a bit of a pain, it was. I don’t know that Apple has necessarily dropped the ball in terms of usability in the last few years, but sometimes it feels like it. I think I just had really high expectations based on some good fortune I’d enjoyed over the past 12 years. I’m not sure what the term is exactly, but it’s possible that because I’ve invested this much money in a product, I’m more forgiving of the issues associated with the product. Apple has done a great job historically of masking the complexity of technology from the end user. Sometimes, though, you’re going to come across odd situations that potentially push you down an odd path. That’s what I tell myself anyway as I rue the time I lost on this upgrade. Was anyone else’s upgrade to Catalina this annoying?

Cisco IT Blog Awards

I’m very happy to announce that this blog is a finalist in the 2018 Cisco IT Blog Awards under the category of “Most Entertaining”. Voting is open until January 4th 2019, so if you’ve felt entertained at any point this year when reading my witty articles please go to http://cs.co/itblogawards and pop in a vote for “PenguinPunk”.

And if you are not entertained, check out some of the other entrants in any case – they’re pretty ace.

OT – I Voted. Now It’s Over To You

Eric Siebert has opened up voting for the Top vBlog 2018. I’m listed on the vLaunchpad and you can vote for me under storage and independent blog categories as well. There are a bunch of great blogs listed on Eric’s vLaunchpad, so if nothing else you may discover someone you haven’t heard of before, and chances are they’ll have something to say that’s worth checking out. If this stuff seems a bit needy, it is. But it’s also nice to have people actually acknowledging what you’re doing. I’m hoping that people find this blog useful, because it really is a labour of love (random vendor t-shirts notwithstanding).

OT – Digital Movie Consumption Still A Bin Fire – News At 11

This article’s a little different from my normal subject matter, but I felt the strong urge to have a bit of a rant, and explore some feelings, so buckle up. Digital content distribution (particularly for feature films) as it relates to consumers has been a mess for some time. It still is in my opinion. I wanted to work through some of my issues with it in this article. I don’t have a lot of answers, so if it’s resolution you’re after, you’re in the wrong place.

 

Background

It’s been a long time since video tape was the de facto mechanism for film consumption for the average punter. Unlike VCRs, DVDs (and Blu-ray) were readable on computers at around the same time they became available to the consumer to watch on standalone devices plugged into televisions. DVDs also came with a bunch of protection mechanisms that were pretty easily thwarted (if you were adept at searching the Internet). As a result you could take feature films and store them in a digital format relatively simply. So why not just distribute those files to consumers?

For some reason we’re okay to treat the storage and distribution of music in a way that’s different to movies. To wit, the iPod was massively successful in the market, but movie storage devices (even after we got past the capacity limitations of the time) have struggled to gain traction, commercially or legally. Even legitimate content delivery services like kaleidescape were, in my opinion, crippled by the licensing requirement to have the physical discs in the unit when they played files from their internal storage.

It took a long time for companies to get behind the idea of distributing movies in a digital format. Studios focused on using Digital Rights Management (DRM) to cripple consumption in a way that seemed positively hostile. In some instances it felt like they were not terribly interested in you actually consuming the film in a fashion that was simple or convenient. Movie studios to this day seem mighty afraid of putting content in the digital realm. This isn’t necessarily unwarranted, with tools like AnyDVD lasting a lot longer (and doing a lot more cool stuff) than anyone had imagined. I think some of this focus on making things difficult was the idea that consumers were merely accessing a license to consume the content, and the transport mechanism could be determined by the content owners. The problem with this is that people think of films in much the same way as they think of books. They have the idea that once they purchased the media, that should be sufficient to consume the film forever. Content owners (the studios, really) are pretty happy for you to think that, but they were also chuffed when we transitioned from VHS to Laserdisc to DVD to Blu-ray (and now, potentially, some new UHD variant). I have The Way of the Dragon on a variety of formats at home. I’m an edge case perhaps, but what about that copy of Throw Momma from the Train that you have on VHS? You probably don’t have a working deck anymore, but I’m sure you’d like to dip back into a cinematic masterpiece every now and then, wouldn’t you?

The other problem with digital content distribution was that, once the studios decided to go ahead with it, it was Apple versus the world in terms of distribution standards. In much the same way that Blu-ray was pitched against HD-DVD, Apple’s iTunes was promoted as a superior delivery mechanism. And it can be, as long as you’re all in with Apple, and happy with the content catalogue they have in place. Disney was also guilty of this approach. But if there’s stuff you want to watch that isn’t part of their ecosystem, you need to look at alternative methods of consumption. Like streaming, for example.

 

But Not Everyone is Streaming

Bandwidth is a problem in Australia. It’s a first world problem, to be sure, but it’s still a problem. And for a lot of people. A common connection type is ADSL1 or 2+, and fibre to the home was killed off in a political stoush that we should all be ashamed of. But I digress. In any case, things aren’t overly fast, and streaming content options are fairly limited (it’s a small market). Since its launch in Australia Netflix has been steadily improving its content catalogue, but it’s nowhere near as extensive as the one in the US.

It’s for that reason that I still buy movies on Blu-ray. And I get access to “Digital Copies” of movies along with these discs. In the olden days, these were often files I could import directly into iTunes off a separate DVD. Sometimes they were DRM-protected wmv files that I couldn’t really play anywhere except on a Windows PC. Nowadays they are primarily UltraViolet-based redemption codes. This makes sense, as a lot of computers don’t have optical drives any more. I don’t use UltraViolet services as my primary consumption mechanism, as I tend to watch movies on a big screen connected to an Apple TV running Plex. But from time to time (particularly when travelling on long-haul flights) I’ve found the ability to load up a reasonably sized file on an iPad or laptop to be very convenient, particularly when the in-flight entertainment system fails.

 

UltraViolet

The idea behind UltraViolet is / was pretty cool. People realized a few things about content distribution. Firstly, studios weren’t always going to agree on which service to use for distribution, or which device the content could be consumed on. And sometimes you wanted to change the way you consumed your media. So the narrative changed from media or streaming to licensing, and you were granted rights to consume the content you wanted, ostensibly on any platform you liked. Sounds like a great idea, and even in Australia, a number of content providers jumped on board. I found the redemption process to be fairly straightforward, although I didn’t like how some studios insisted on me handing over my details in order to gain access to the titles (after I’d already created accounts with UltraViolet and a provider of my choosing). I found the number of standalone devices that actually supported UltraViolet titles to be pretty small, despite what the FAQs were saying. I had the most success consuming content via the website of one of the providers, rather than using an app on an Apple TV or similar.

Is it still working? Sort of. If you read through the change notice of this FAQ you’ll notice a bunch of providers slowly disappearing from Australia and around the world. Again, I’m an edge case, consuming content in a small market. But it seems like just when every Blu-ray has a standardized electronic rights copy included we’ve slowly started to take away the ways to consume those copies. Well, that’s what I thought at first, but apparently there’s something else, potentially better, happening.

 

Movies Anywhere

A service recently launched called Movies Anywhere. It was originally launched in 2014 as Disney Movies Anywhere, and was rebranded and re-launched in the last month. The idea is that it ties together your content licenses from any number of providers and systems and allows you to consume them on a unified platform. That’s about all I can tell you, because it’s US-based and not available anywhere else. I’m not going to turn this into an ad for the service, because I can’t tell you how well it actually works, and whether it really does what I want it to do. But it does seem to tick a number of boxes in terms of linking a number of disparate services together.

 

So What’s the Problem?

I like the idea of being able to pay for content once and having access to it for a long time. I still have a Laserdisc player, but a lot of people don’t. So they’ve re-invested in media over and over again. This makes sense if you follow the progression of technology (and improvements in playback quality), but when we have better mechanisms to access content (such as digital storage) it makes less sense that we should continually pay for the same thing over and over.

The problem, as always, is that any time we do get close to having some cool tech available to do what we want, it gets restricted to a specific region. By the time this stuff gets to Australia, the rest of the world has moved on and we’re left with patchy support for what are considered legacy services. Or we get the service at launch but don’t get the full product. This is usually because of existing licensing agreements, differences in copyright law, and all kinds of other complicated reasons. Some of these reasons are even, well, reasonable. But it’s still annoying, and I think the Internet just serves to amplify this feeling of annoyance when it comes to things like this. I don’t really know how to solve the problem either. The studios will continue to do what they do until consumers stop consuming. And I think there are enough people out there going along with this that they won’t need to stop any time soon. I still think it’s a bin fire, and that’s a shame. Of course, my kids also think it’s weird that I still purchase content on media, so what do I know?

OT – You Vote Now

Eric Siebert has opened up voting for the Top vBlog 2017. I’m listed on the vLaunchpad under the top 100, and you can vote for me under storage and independent blog categories as well. I climbed the heady heights to number 78 last year. So thanks to my mother for voting for me. You can go directly to the voting survey here. There are a bunch of great blogs listed on Eric’s vLaunchpad, so if nothing else you may discover someone you haven’t heard of before, and chances are they’ll have something to say that’s worth hearing. Or reading. Look, you know what I mean. If this stuff seems a bit needy, it is. But it’s also nice to have people actually acknowledging what you’re doing. This all means nothing without your validation.

OT – Top 78

Eric Siebert recently published (okay, fine, it was three weeks ago) the full results of the Top vBlog voting. I was pleased to find I’d made a jump up from last year.

vBlog_2016_snip

I’ve previously changed my tune on asking for votes in this competition, not because I don’t think it’s a good bit of fun, but I think there’re a bunch of other bloggers you should be voting for. A few people like to huff and puff about it being a popularity contest, but if nothing else I’ve found these types of lists (and Eric’s site in general) to be extremely useful when tracking down links to things on the internet that I know I need but can’t remember how I googled them in the first place. A lot of work goes into the site, so thanks Eric, and please keep it up! Thanks also to anyone who did throw a vote my way, I do actually appreciate it.

OT – Top Virtualisation Blogs – Don’t Vote For Me

Eric Siebert has opened up voting for the 2015 top VMware & virtualization [sic] blogs. I’m listed on the vLaunchpad under storage blogs. Previously, I’ve been keen to get your vote. I even once made it in the top 10 (for storage bloggers). But this time around I think it would be a bit silly to vote for me when there’s all these other great bloggers you can vote for. People like Ray, Nigel and Chin-Fah are really, really, switched on people and you should be giving them some voting love. As are all those virtualisation types. So go and do that. And don’t vote for me. Unless you’re really into CX700 FLARE recovery and QNAP mdadm shenanigans.

OT – A vote for me is a vote, er, for me.

Eric has launched the voting for the 2014 top VMware and virtualisation blogs here. There’re also categories for storage, independent, podcasts and other things. Please head on over and check it out. I’m listed under the general category, storage and independent. Last year I made the top 109, let’s see if I can move up that list a little.