Even Apple can’t figure out AR

Big news from Bloomberg’s Mark Gurman: Apple has reportedly cancelled work on augmented reality glasses designed to pair with a Mac:

The company shuttered the program this week, according to people with knowledge of the move. The now-canceled product would have looked like normal glasses but include built-in displays and require a connection to a Mac, said the people, who asked not to be identified because the work wasn't public.

The cancellation of any project is never a great sign. But there are a few things about this one that paint a worrying picture about Apple’s future with augmented reality (AR).

The struggles of Vision Pro have been well documented, but three factors in particular stand out: it’s too expensive, it’s too heavy, and it doesn’t have a “killer app” — a singular feature, app or function that is just so compelling that it drives people to buy the product. By most accounts, the closest thing it does have to a killer app is Mac Virtual Display, which displays your Mac’s screen as a massive ultra wide display inside Vision Pro.

If Gurman’s reporting is accurate, these glasses would have addressed all three points. It’d have been cheaper, light enough to be worn without a cumbersome head strap and would have connected to a Mac to project the computer’s screen on to the glasses.

To put it another way: these glasses would have featured the Vision Pro’s most compelling feature without two of its biggest flaws. And yet, Apple’s leadership deemed the product not good enough for release.

I am not saying that was the wrong decision. Mac sales are a fraction of iPhone and iPad sales, and these glasses would have been an accessory to that; it’d effectively be a niche product for owners of a niche product. That’s probably not a big market.

Beyond the merits of the product itself, what stands out to me is an apparent lack of focus to Apple’s plans. The Vision Pro may act as an AR device, but it is technically a VR device: it merely projects the real world on to a screen. These glasses are, well, glasses. They would have been a pure AR device, a wholly different technical undertaking to the VR Vision Pro and an entirely different proposition to consumers.

Remember, the point of Vision Pro — the supposed reason for the high price and not-quite-mass-market features — is that it’s meant to be for developers. It’s supposed to give them a taste of what visionOS is, of how to build apps for AR, so that when a consumer-ready Vision device arrives, people will find a fully-stocked App Store.

But this project, and its cancellation, beg the question: what exactly are developers building apps for? The Vision Pro isn't selling, the company isn't talking about the next generation, and there is no consensus among Apple watchers about what the next Vision device even is. Will it be Vision Pro 2? A cheaper, consumer-focused Vision Air? Or will Apple try a different AR glasses project?

One thing that reporters and analysts covering Apple agree on is that whatever's next isn't coming out any time soon. And from Gurman’s reporting, that’s because even Apple isn't sure where to go next:

(Cancelling the Mac-connected glasses) marks at least the second time Apple has walked away from an AR project. Around two years ago, it shelved its first effort, a project code-named N421. Apple had originally aimed to bring those glasses to market around a year after the initial Vision Pro. Apple canceled N421, in part, to focus on a new mixed-reality headset code-named N109, which has been discussed as a cheaper variant of the Vision Pro's VR-based design.

To be clear, I do believe in AR. I do think that the base concept of layering the digital world on to the physical is a no-brainer. Having directional arrows appear to tell you where to go or nutritional information pop up when you pick up something at the supermarket makes more sense to me than having to glance down at your phone to look those things up. It removes a layer of abstraction to things that we’re already doing.

But while I believe in the theory, I am less confident about the execution. There are still seemingly too many technical hurdles, too many questions about form and function. It feels like there is a long, long way to go.

(In case you’re wondering why I haven’t mentioned Meta’s Orion AR glasses concept: that they admit that the technology they’ve used to make the prototype glasses won’t be used in the yet-to-be-seen consumer version means I remain skeptical that they are, for now, anything but a neat demo.)

The last three times Apple released a brand-new hardware platform, you didn’t have to wait long for the second generation. The original iPhone was followed by the iPhone 3G just over a year later. The iPad 2 arrived 11 months after the first iPad. And Apple Watch Series 2 was released less than a year and a half later.

It is now exactly one year since Vision Pro was released. Not only is there little indication of a second Vision device, we have no idea what form it will take: VR or AR? Pro or consumer? Standalone platform or accessory?

What’s most troubling is that Apple doesn’t seem to know either.