This was a big fanboy moment for me being such a huge fan of Marques’ work for so many years especially after spending a decade in the smartphone world.
Being around the studio and hanging out with the whole team was truly an experience because these guys run a really tight show. Besides Marques (whom I’d previously met in person 10+ years ago at some distant edition of Google I/O) and this episode’s co-host David, I also got to hang out with the rest of the Waveform crew: Andrew Manganelli, Adam Molina and Ellis Rovin. I had as much fun outside of the podcast recording as I did in it.
Friends and colleagues have been asking me to share my perspective on the Apple Vision Pro as a product. Inspired by my dear friend Matt Mullenweg’s 40th post, I decided to put pen to paper.
This started as a blog post and became an essay before too long, so I’ve structured my writing in multiple sections each with a clear lead to make it a bit easier to digest — peppered with my own ‘takes’. I’ve tried to stick to original thoughts for the most part and link to what others have said where applicable.
Some of the topics I touch on:
Why I believe Vision Pro may be an over-engineered “devkit”
The genius & audacity behind some of Apple’s hardware decisions
Gaze & pinch is an incredible UI superpower and major industry ah-ha moment
Why the Vision Pro software/content story is so dull and unimaginative
Why most people won’t use Vision Pro for watching TV/movies
Apple’s bet in immersive video is a total game-changer for Live Sports
Why I returned my Vision Pro… and my Top 10 wishlist to reconsider
Apple’s VR debut is the best thing that ever happened to Oculus/Meta
My unsolicited product advice to Meta for Quest Pro 2 and beyond
The Apple Vision Pro is the Northstar the VR industry needed, whether we admit it or not
I’ve been a VR enthusiast for most of my adult life, from working as an intern at Disney Quest VR in the 1990s, to being an early backer of the Oculus Rift DK1 on Kickstarter in 2013, to leading the Oculus VR/AR team at Meta from 2017 to 2020 (and getting to work alongside VR legends like John Carmack, Brendan Iribe and Jason Rubin), and always testing every VR product or experience I can get my hands on.
Back in my Oculus days, I used to semi-seriously joke with our team (and usually got a lot of heat for it!) that the best thing that could ever happen to us was having Apple enter the VR industry and become a direct competitor to Oculus. I’ve always believed that strong competition pushes a team to do their best work in any industry. This became clear to me especially after living for nearly 10 years at the center of the iOS/Android battle of ecosystems where each side made the other infinitely better by constantly raising the bar on UX, features, performance, developer APIs etc, and seeing each side respond by not only fast following but usually also improving on what the other had released. (And this definitely went both ways: iOS copied Android as much as Android copied iOS).
But in the case of VR at Oculus, we also never really felt like the world had a Northstar that could truly capture human hearts and minds, and without that it would be impossible to transition VR from being a niche gamer tech to the incredible spatial computing paradigm that we always thought it potentially represented (which I still very much believe in). Apple could really help us if they cared about VR.
The Vision Pro launch has more or less done exactly what I had always hoped for, which is to build a huge wave of awareness and curiosity that elevates the spatial computing ecosystem and could ultimately lead to mass-market consumer demand and a lot more developer interest than VR has ever had. Now it’s up to the industry to create enough user value and demonstrate whether this is in fact the future of computing.
The Vision Pro’s instant magic comes down to just: (1) an unprecedented new level of presence in VR, and (2) a new UI superpower using gaze & pinch
Using Vision Pro is an instantly magical and intuitive experience — whether or not you’ve used other VR headsets — purely because of Apple’s unrelenting focus on delivering two specific capabilities that speak to our humanity:
1) Feeling present and connected to your physical world: thanks to a high-fidelity passthrough (“mixed reality”) experience with very low latency, excellent distortion correction (much better than Quest 3), and sufficiently high resolution that allows you to even see your phone/computer screen through the passthrough cameras (i.e. without taking your headset off).
Even though there are major gaps left to be filled in future versions of the Vision Pro hardware (which I’ll get into later), this level of connection with the real world — or “presence” as VR folks like to call it — is something that no other VR headset has ever come even close to delivering and so far was only remotely possible with AR headsets (ex: HoloLens and Magic Leap) which feature physically transparent displays but have their own significant limitations in many other areas. Apple’s implementation of Optic ID as an overlay on top of live passthrough is a beautiful design decision that only enhances this sense of presence.
MY TAKE: The Vision Pro high-fidelity passthrough experience parallels Apple’s introduction of the iPhone’s original retina display, which set a new experience bar and gold standard in mobile display fidelity. While much remains to be improved in the Vision Pro passthrough experience, Apple is unquestionably setting a new standard for all future headsets (by any vendor) that VR passthrough must be good enough to closely resemble reality.
2) Having a new UI superpower with gaze & pinch, thanks to a very precise eye tracking system (with 2 dedicated cameras per eye) embedded into the lenses, coupled with a wide-field-of-view hand tracking system that can “see” a finger pinch even with your hands are down or resting on your lap. Because it works so effortlessly for the user, it really feels like having a new “laser vision” superpower.
The hardware needed to track eyes and hands in VR has been around for over a decade, and it’s Apple’s unique ability to bring everything together in a magical way that makes this UI superpower the most important achievement of the entire Vision Pro product, without a shadow of doubt.
MY TAKE: The Vision Pro’s new “gaze + pinch” input modality is the VR equivalent of the iPhone’s capacitive multi-touch gestures. Introduced by Apple with the first iPhone launch nearly 17 years ago, multi-touch instantly became a new standard that changed computing forever. “Gaze + pinch” is so groundbreaking that it’s an instant de facto standard for VR interaction that future VR headsets will be forced to adopt sooner or later. It’s also going to be a huge developer unlock that leads to gaze-based interaction ideas that will blow our minds.
Hardware
Vision Pro is a meticulously over-engineered “devkit” that is far too heavyto have product-market fit but good enough to seed curiosity into the world
The Oculus VR story began with the 2013 launch of Oculus Rift DK1 (short for “devkit v1” or “development kit v1”). This was a headset launched by the original Oculus startup team — years before it was acquired by Facebook — with the explicit goal of seeding developer interest well before a commercial release. Given that VR was a non-existing market then, releasing a devkit was the correct and necessary strategy there and then for a startup to start building a content library as well as momentum among enthusiasts ahead of launching a consumer product. The team released a DK2 about a year later in 2014, and finally launched the first Oculus Rift consumer headset in 2015.
Oculus co-founder Palmer Luckey wearing the original Oculus Rift DK1 released in 2013
When I joined Facebook to lead the Oculus team in 2017 after the acquisition, one of the many battles I found myself in the middle of almost immediately was the “devkit war”. The Oculus team’s DK1 and DK2 legacy was so strong that it was not uncommon to hear arguments in product meetings pushing for us to launch VR headsets still in prototype stage as “devkits” to end users. Since Oculus was no longer a startup — and had the resources to both extensively test prototypes without launching them as products and run extensive pre-launch developer programs — it no longer made sense for Oculus devkits to exist. This stance often didn’t make me very popular amongst some of my Oculus OG colleagues.
Fast forward to 2024. After the Vision Pro launch, the VR hardware enthusiast community (including Oculus OG folks I’m still in touch with) quickly arrived at the conclusion that Apple really played it safe in the design of this first VR product by over-engineering it. For starters, Vision Pro ships with more sensors than what’s likely necessary to deliver Apple’s intended experience. This is typical in a first-generation product that’s been under development for so many years. It makes Vision Pro start to feel like a devkit.
A sensor party: 6 tracking cameras, 2 passthrough cameras, 2 depth sensors (plus 4 eye-tracking cameras not shown)
Here’s a quick comparison with existing VR headsets:
Sensor
Vision Pro
Meta Quest 3
Meta Quest Pro
Environment passthrough cameras
2
2
1
World tracking cameras
6
4
6
Depth sensors
2
1
–
Eye tracking cameras
4
–
2
Total
14
7
9
Side-by-side comparison with the sensor stacks of other VR headsets
This over-spec’ing is unsurprising and characteristic of a v1 product where its creator wants to ensure it survives the hardest tests early users will no doubt want to put the product through. It’s also a way for Apple to see how far developers will push the product’s capabilities, as Apple is no doubt relying on that community to produce the majority of software/content magic for this new type of computer, as they’ve previously done with every other device class.
Apple’s decision to over-spec the Vision Pro does, however, lead to the inevitable consequence of a headset weighing above 600g — heavier than most other VR headsets in the market to date — that makes it difficult for most people to wear it for more than 30-45 minutes at a time without suffering a lot of discomfort. Most of the discomfort comes in the form of pressure against the user’s face and the back of the person’s head.
MY TAKE: Because of its heavy weight, Vision Pro has inevitably landed in the world as a high-quality “devkit” designed to capture everyone’s curiosity, hearts & minds with its magic (especially through the voice of enthusiastic tech influencers) while being realistically focused on developers as its primary audience. In other words, the Vision Pro is a devkit that helps prepare the world to receive a more mainstream Apple VR headset that could have product-market fit in 1 or 2 generations.
All things considered, I do believe Apple’s calculus was correct in prioritizing launching a first-generation product with fewer experience and design compromises at the expense of user comfort. And while many people have argued Apple could have avoided this major comfort issue by redistributing weight or using lighter materials, those attempts would have come at the expense of beauty and design. (I’ll come back to the weight issue shortly.)
With this in mind, it’s easy to understand two particularly important decisions Apple made for the Vision Pro launch:
Designing an incredible in-store Vision Pro demo experience, with the primary goal of getting as many people as possible to experience the magic of VR through Apple’s lenses — most of whom have no intention to even consider a $4,000 purchase. The demo is only secondarily focused on actually selling Vision Pro headsets.
Launching an iconic woven strap that photographs beautifully even though this strap simply isn’t comfortable enough for the vast majority of head shapes. It’s easy to conclude that this decision paid off because nearly every bit of media coverage (including and especially third-party reviews on YouTube) uses the woven strap despite the fact that it’s less comfortable than the dual loop strap that’s “hidden in the box”.
The existence of Vision Pro in 2024 is entirely a function of Apple managing to ship a first-of-its-kind ultra high-resolution display
One of our biggest product positioning struggles within the Oculus VR team from the very beginning — especially when trying to convince reviewers — was always related to having underwhelming displays. Every single Oculus headset that ever shipped (including the latest Quest 3) has suffered from resolution/pixelation issues varying from “terrible” to “pretty bad”. It’s like we’re living in the VR-equivalent world of VGA computer monitors.
MY TAKE: In order For Apple to make a huge splash entering the VR market — a category that’s been around the consumer world for nearly 10 years — they needed to launch a product that was unambiguously better than anything that had ever existed. The obvious way to do that was to attack the Achilles heel of all existing headsets and reinvent the VR display, and that’s exactly what Apple did with the Vision Pro.
Vision Pro is the first VR headset that offers good enough resolution and visual acuity with little semblance of a screen door effect or pixelation artifacts. This level of presence and fidelity could only be made possible with an ultra high-res display, and it’s 100% clear that achieving a first-of-its-kind level of display quality was the internal launch bar for Vision Pro at Apple.
Apple’s relentless and uncompromising hardware insanity is largely what made it possible for such a high-res display to exist in a VR headset, and it’s clear that this product couldn’t possibly have launched much sooner than 2024 for one simple limiting factor — the maturity of micro-OLED displays plus the existence of power-efficient chipsets that can deliver the heavy compute required to drive this kind of display (i.e. the M2).
Micro-OLED displays differ from any other previous consumer display technology because they are manufactured on top of a silicon substrate (similar to how semiconductor chips are made). To put the insanity of micro-OLED displays in perspective, the Vision Pro panel has a 7.4x higher pixel density than the latest iPhone and nearly 3x the Quest 3:
Feature
Vision Pro
Bigscreen Beyond
Quest 3
iPhone 15 Pro Max
Display Type
Micro-OLED
Micro-OLED
LCD
OLED
Resolution (pixels per eye)
3660 x 3200
2560 x 2560
2064 x 2208
2796 x 1290
Total Pixels
23 million
13 million
9 million
3.6 million
Pixels Per Inch (PPI)
3386
(unknown)
1218
460
Pixels Per Degree (PPD)
34
32
25
94 (at 1 foot distance)
(See the appendix of this essay for a quick explanation of Pixels Per Degree or PPD)
The folks at iFixit created this stunning GIF using a scientific microscope to compare the pixel size of the Vision Pro display — which measures 7.5 μm, the size of a human red blood cell — with the pixel size of the latest iPad and iPhone displays:
The Apple Vision Pro’s micro-OLED display has created a lot of chatter in my hardware supply chain world, with lots of companies — predominantly smartphone OEMs — quickly racing to try and build a product that can deliver a similar experience to Vision Pro. Apple has secured a 1-year exclusive with Sony Semiconductor Solutions Group and its second supplier SeeYA Technology. There are also rumors Apple is dropping Sony as a display supplier and replacing it with BOE (whose website says a panel equivalent to Vision Pro is at “sample” stage).
MY TAKE: I fully expect the recently announced Meta/LG partnership to be all about creating a supply chain advantage for Meta so they can race a Quest Pro 2 product into market that can compete with Vision Pro with LG putting some skin in the game to lower the street price of the headset.
(P.S. For anyone who wants to dig into more details of the Vision Pro display and pass-through system, I highly recommend this article from iFixit and this article from Karl Guttag, who’s an amazingly talented expert in display devices).
Apple made the Vision Pro display intentionally blurry in order to hide pixelation artifacts and make graphics appear smoother
There is a very good reason Apple has not used the word retina anywhere in their marketing materials for Vision Pro. It’s the simple fact that Vision Pro’s display does not pass the retina test — which is a resolution high enough that the human eye can no longer discern individual pixels. The Vision Pro display is nowhere near retina quality for a VR headset (see appendix for details) and yet our eyes cannot see individual pixels when looking at it. What gives?
During the first few days using Vision Pro, there was something that kept calling my attention but which I struggled to get my arms (or eyes) around. Everything my eyes saw in the headset felt a bit softer than I expected, and I initially attributed this to the seemingly refreshing absence of any screen door effect — a pixelation artifact that has essentially doomed all VR headsets created up until now.
Well, as it turns out, the incredible Karl Guttag ran a meticulous photographic analysis of the Vision Pro display and came to a curious and possibly disturbing conclusion: Apple intentionally calibrated the Vision Pro display slightly out of focus to make pixels a bit blurry and hide the screen door effect “in plain sight”.
This image from Karl’s blog explains this well by comparing Vision Pro and Quest 3 displays side by side at a close enough distance where it’s possible to see individual pixels and clearly see the intentional blur that was added to the Vision Pro display:
Extreme close-up comparison between Vision Pro (AVP) and Quest 3 (MQ3) displays (Source: KGOnTech)
What Karl concluded is that even though Quest 3 has a much lower display resolution than Vision Pro (1,218 PPI vs. 3,386 PPI), Quest 3 appears objectively crisper especially when showing high-contrast graphics. In other words, Quest 3 is squeezing the highest possible resolution out of its display at the expense of a “harsher look” while Apple is giving up some of the Vision Pro’s display resolution in order to achieve a “softer look”. Karl may disagree with my conclusion on this point:
MY TAKE: Intentionally making the Vision Pro optics blurry is a clever move by Apple because it results in way smoother graphics across the board by hiding the screen door effect (which in practice means that you won’t see pixelation artifacts). This is also where Apple’s “taste” comes in, essentially resulting in the Vision Pro display being tuned to have a unique, softer, and more refined aesthetic than Quest 3 (or any other VR headsets). This is certainly a refreshing approach to designing VR hardware.
With this design decision, Apple is no doubt giving up a bit of the Vision Pro display’s high pixel resolution in order to achieve overall smoother graphics. You are definitely losing some text crispness in order to gain a higher perception of quality for images, video and 3D animations. This is a big benefit of starting with an ultra high-resolution micro-OLED display — Apple had enough pixels to work with that they could afford to make this trade-off. This is the kind of thing that our hardcore VR engineers at Oculus would have fought against to the end of the world, and I doubt we could have ever shipped a “blurred headset”, LOL!
Sadly, the Vision Pro display suffers from significant motion blur & image quality issues that render passthrough mode unusable for longer periods
While Apple’s decision to make individual pixels blurry on the Vision Pro display was extremely clever, the headset unfortunately suffers from a completely different type of blur that’s extremely problematic for the overall experience.
From the very first time I put on my Vision Pro, I noticed a lot of motion blur in passthrough mode even in excellent ambient lighting conditions and a still noticeable amount even when viewing immersive content. While my immediate instinct was to think that all VR headsets have that kind of motion blur and it’s just more noticeable on Vision Pro, a side-by-side comparison with Quest 3 quickly proved it’s significantly more serious on Vision Pro. This is particularly surprising considering that the passthrough cameras and display are both running at 90 hertz.
Since none of the initial Vision Pro reviews pointed out this issue, I ended up even calling Apple support to find out if this might be a known problem or possibly even a hardware defect. But then more in-depth reviews began pointing out the same problem (I highly recommend this review by Snazzy Labs).
Motion blur in passthrough mode ended up being one of the many reasons why I decided to return my Vision Pro, because it’s just uncomfortable, leads to unnecessary eye strain, and really gets in the way of anyone using the headset for longer periods of time in passthrough mode.
There are other noticeable issues as well which affect passthrough mode, including very little dynamic range, incorrect white balance in most indoor use cases, and signs of edge distortion and chromatic aberration. Some of these might be addressed by software updates, but I expect most will not as they probably are limitations of the hardware stack.
The Vision Pro packs a lot more computing power than most people might realize — the M2 + R1 combination puts it at the level of a MacBook Pro
Any standalone VR headset is basically a 2-in-1 system: a regular “computing” computer and a spatial computer bundled together.
A regular computer in charge of running applications and performing general computation: this is everything that happens on your smartphone, tablet or notebook, including running the OS, executing applications across CPU/GPU loads, and doing computation work in the background.
A spatial computer in charge of the environment: it keeps track of the whole environment, tracks your hands & eyes, and ensures that everything — your surroundings, the OS system UI, and your apps — gets rendered in the right physical place in space and updated at 90 to 120 times per second while your head and body are moving around.
These two “computers” must operate together without missing a beat — any latency above 20 milliseconds becomes quickly noticeable in VR and will often translate very quickly into user perception of unresponsiveness or jankiness, which can cause discomfort, eye strain or even dizziness for many people.
Enter the Vision Pro dual-chip design:
“A unique dual‑chip design enables the spatial experiences on Apple Vision Pro. The powerful M2 chip simultaneously runs visionOS, executes advanced computer vision algorithms, and delivers stunning graphics, all with incredible efficiency. And the brand-new R1 chip is specifically dedicated to process input from the cameras, sensors, and microphones, streaming images to the displays within 12 milliseconds — for a virtually lag-free, real-time view of the world.“
Apple Vision Pro website
The Vision Pro ships with the same M2 chip as the 2022 iPad Pro (or 2022 MacBook Air) alongside the new R1 chip which handles the massive amount of data coming from the 20+ tracking cameras and depth sensors (sensor fusion). What’s interesting to note is that Vision Pro actually does perform largely like an iPad Pro in benchmark tests that push CPU and GPU to their limits in both single-core and multi-core scenarios (see chart below).
This is more impressive than it seems, and demonstrates that the R1 chip is doing a very significant amount of heavy lifting — essentially the vast majority of the spatial computing workload — leaving a lot of compute room for the M2 chip to deliver the same level of performance as if it was just running inside an iPad Pro.
By all accounts so far, the R1 chip appears to be of a fairly similar package size as the M2 chip (though built using a specialized architecture), which puts the Vision Pro well ahead of any current generation iPad or MacBook Air, and likely more on par with a MacBook Pro from a silicon performance perspective. Definitely an impressive achievement by the Apple Silicon team.
This also begs the question… what if you could completely offload the Vision Pro’s compute to another Apple device?
Apple’s decision to use a tethered pack willenable future Vision headsets to be much lighter by offloading compute to an iPhone, iPad or MacBook
One of the most controversial aspects of Vision Pro is the fact that it sports a tethered battery pack, differently from all other commercially available standalone VR headsets. Many people have heavily criticized Apple for this decision because of the inconvenience of the “hanging” external battery.
I agree with Palmer Luckey (from his recent interview with Peter Diamandis) that this was a necessary short-term decision on Apple’s part given the reality of the hardware shipping inside the Vision Pro, but more importantly it was a very intentional long-term decision, which I’ll explain.
As I mentioned earlier, the Vision is a meticulously over-engineered computer with a very significant collection of power-hungry components:
2x laptop-class processors (the R1 chip is almost the same size as the M2, which is the same processor shipping on MacBooks)
2x very bright micro-OLED displays with high pixel density
1x auxiliary EyeSight display
12x cameras and other sensors
2x blower fans
2x speakers
As Quinn Nelson from Snazzy Labs carefully explains in his excellent review, the Vision Pro likely draws as much as 40 watts of power, which is more than most MacBook laptops. This also means it has a power supply with the potential of generating a lot of heat. So, in addition to transferring the battery weight out of the headset, the decision to move to a tethered pack also keeps a huge heat source safely away from your head.
MY TAKE: All that said, the long-term strategic reason for having an external battery pack is to set expectations with Vision Pro users that there will always be an external box connected to the headset. In future Vision headsets, Apple should be able to comfortably start moving a lot of electronics off the headset, possibly shaving off as much half of the weight over a few generations and target around 300g. This also opens an extremely interesting path for Apple in a few years to use an iPhone, iPad or MacBook as the tethered computer driving the headset, which would dramatically simplify the headset.
Interestingly, there is a tethered VR headset in the market today that demonstrates this desirable end state. It’s the Bigscreen Beyond, the world’s smallest PC VR headset (i.e. needs to be tethered to a computer) that is lighter than even most ski goggles at 127 grams. Bigscreen’s ability to build this product is in many ways a bit of cheating since the headset was stripped of all sensors (no external cameras or eye tracking), but its existence nonetheless plays an important role in letting us experience what the future holds and where Apple’s sights are focused.
John Carmack wearing the Bigscreen Beyond VR headset, which weighs 127g (vs. the Vision Pro’s 600g)
Software
The Vision Pro software story is a bold antithesis of VR — and the lack of exciting AR apps at launch paints the product into an empty corner
“Welcome to the era of spatial computing” is Apple’s leading slogan for Vision Pro and, and as expected by everyone in the VR industry, Apple is going all-in on AR (augmented reality) to deliver on this proposition. The company has gone out of their way to actively ignore everything that VR has been known for over the last decade.
At the center of Apple’s marketing for Vision Pro is “keeping users connected to their surroundings and other people”. Reading in-between these lines, it’s not hard to see that Apple is taking an anti-VR stance that borderline accuses Meta’s approach to VR of promoting human isolation while positioning Vision Pro as the antithesis of that.
MY TAKE: Apple’s anti-VR stance is a risky move because it negates most of the traditional immersive content that has made the VR medium popular until now, and at least for now is painting Vision Pro into an empty corner. This reminds me of Apple’s broad stance on privacy — built to be in complete opposition to Meta/Google — which has put them in a tight spot by severely limiting their options and restricting innovation in the age of Gen AI.
There are no fully immersive games in the Vision Pro app store, whereas easily >90% of the Oculus Quest catalog is made of immersive VR games. Instead of leveraging the existing community of high-quality immersive VR content developers, Apple is focusing all of its energy exclusively on AR use cases that play to the company’s ecosystem strengths — iOS apps and MacOS productivity — which I’ll dive into over the next few sections.
The launch roster of 3D AR apps & games is a tremendous disappointment — in both quality and quantity — and mostly includes a few simple casual games, some of which are originally 2D games hastily converted into 3D art. The fact that ARKit has been available for so many years on iPad and iPhone (despite its limited success) should have made it possible for Apple to easily round up developers into building a sufficient number of exciting and impressive AR titles for Vision Pro. Instead, we’re seeing an initial lack of developer excitement for the category that should have been the most defining and inspiring category on Vision Pro.
Ironically, Meta made almost exactly the same mistake launching Quest Pro in 2022. That headset shipped with close to zero AR apps despite their emphasis in the launch messaging on “full-color mixed reality”.
MY TAKE: This may be the first device category where Apple’s “build it and they will come” approach to creating developer traction may simply not work as previously. It will be many years (and possibly even more than a decade) before there are tens of millions of active Vision Pro users willing to pay for spatial AR apps. Apple will need to take a page out of the Oculus playbook and actively motivate developers financially to develop for Vision Pro.
Vision Pro’s positioning as a productivity & movie watching “big screen”is dull & unimaginative but Apple is unashamedly owning it
With a weak and limited launch roster of AR apps that doesn’t include a single flagship 3D app or game, Apple had to focus the entire positioning for Vision Pro at launch almost entirely on how it plugs into the existing Apple ecosystem of 2D apps.
In the usual Apple-style product marketing, the launch messaging for Vision Pro is very explicitly codified in the product webpage and every single marketing asset is consistent with it. How Apple has chosen to sequence their product messaging matters just as much as the messages themselves. Vision Pro is 60% about 2D productivity and 40% about watching media/movies on a big screen:
Use case
Apple Slogans
Productivity
“Free your desktop. And your apps will follow.”
“How to work in all‑new ways.”
Media
“The ultimate theater. Wherever you are.
“An immersive way to experience entertainment.”
On a side note, FaceTime with Persona avatars and spatial photos & videos are also pushed as core pillars in the Vision Pro product messaging, but they’re clearly just ancillary use cases to support marketing. Though too small to matter for now, they may (and for Apple’s sake, hopefully will) end up playing a much bigger role in the future.
MY TAKE: The Vision Pro launch is a significant missed opportunity, with Apple “welcoming us into the era of spatial computing” with a software and services stack that is practically only focused on 2D use cases. Though the in-store demos paint an exciting future, the experience delivered by Apple at launch is dull and unimaginative at best.
Putting aside my criticism of Apple’s focus for the Vision Pro at launch, the next few sections will offer a deep dive into my thoughts and opinions on the software and experience enabling Productivity and Media use cases.
The Vision Pro desperately wants to be the “future of work” and pick up where Meta Quest Pro completely dropped the ball, but…
One of our strongest thesis from the early Oculus days was always about VR playing a defining role in the “future of work”, from running 2D apps in massive virtual displays to having native 3D apps that would make it a lot easier to work and collaborate with others on a project.
When Meta announced the Quest Pro in 2022, much of its marketing hype was in fact around the Workrooms app (at the time led by my incredibly talented friend Mike LeBeau). The app allows you to use your Mac from within VR, with a lot of attention to details needed to make it truly possible to work in VR for several hours, including support for up to 3 virtual monitors and the ability to see your physical keyboard in passthrough or replace it with a fully 3D rendered tracked twin.
Image from the Quest Pro launch marketing
Quest Pro was designed with the goal of being a lot more comfortable than other VR headsets so that people could wear it for longer periods of time. While this was a well-intended attempt, the product had a major flaw which made it less than a “minimum viable product” and simply did not justify a price tag well above $1,000. The display resolution — at 22 PPD (pixels per degree) — was too low and vastly insufficient to unlock “working in VR” because of poor text readability. This shortcoming (in addition to very poor quality passthrough) was so massive that it rendered the product practically irrelevant at launch — I ended up returning my unit within 24 hours of first use.
Can Vision Pro deliver where Quest Pro (and Quest 3) have failed?
In order to really put Vision Pro to the test in real-life scenarios, I spent well over 100 hours trying to deploy as many of my own productivity workflows as I could, including about 1/3 of the work in this essay. I’ll share my conclusions over the next couple of sections.
First off, before diving into the value proposition of Vision Pro as a work/productivity computer, I needed to clearly frame my “jobs to be done” as specifically as possible. When I’m in “work mode” — whether doing actual professional work or just life management stuff — I have three distinct workstations that I use back and forth (aside from my smartphone, which I won’t include here):
Office workstation| Mac Pro with 2x Apple XDR 6K displays: my highest productivity setup because it gives me access to everything I need in a single view and enables zero-hurdle multi-tasking; it’s basically my gold standard for any task or project no matter how complex with the highest speed & quality
Laptop | MacBook Pro 16-inch: medium-high productivity setup with a sufficiently large retina-quality display that still enables complex tasks with good enough multi-tasking though I do feel noticeably less productive; it requires a backpack to carry when going outside of home/office
Tablet | iPad Pro 11-inch with keyboard: a low-medium productivity setup good for focused single-app work with extremely limited multi-tasking (ex. email, writing that doesn’t require research, some life planning) but still better than using my phone; one great advantage is that I can carry this “mini computer” more easily than a laptop without really needing a backpack
Here’s a table summarizing these workstations:
Device
Number of Pixels
Number of simult. windows
Ideal for
Ergonomics
Portability
Cost
Mac Pro + 2x XDR displays
XXLarge (2×20 million)
8+
Any creative project with lots of multitasking
Ideal
–
>$10,000
MacBook Pro 16-inch
Large (7.7 million)
2-4
Most creative projects with limited multitasking
Good
Medium
$3,000
iPad Pro 11-inch + keyboard case
Medium (4 million)
1
Full email, simple editing
Not great
High
$1,200
I then asked myself: Could I see myself using Vision Pro as a productivity device instead of (or in conjunction with) any of my existing workstations?
These are the specific questions I set off to answer (from the lowest to highest bar):
Can Vision Pro be a complete alternative to my Tablet Workstation so that I could carry it around instead of an iPad Pro?
Can Vision Pro enhance my Laptop Workstation enough that it feels like having a “virtual XDR display” or two?
Could Vision Pro ever be better than ALL of my workstations at least for some productivity tasks? ⇒ This is what excites me the most!
Productivity Thesis #1: Vision Pro as an iPad Pro replacement
Status: ❌ NOT READY (but it’s promising!)
MY TAKE: The Vision Pro aspires to become your “spatial iPad Pro” with really good potential for much better multi-tasking (than an iPad) and the ability to do focused work anywhere, but there’s simply too much usability friction and too many important apps missing for that to be a reality today (or likely in the next 1-2 years).
The Vision Pro is conveniently designed by Apple to immediately fit right into the existing Apple ecosystem as a (rather expensive) alternative to an iPad Pro. The headset has identical compute (same M2 chip) to an iPad Pro and conveniently supports iPad apps running natively. In fact, it’s easy to claim the Vision Pro should in principle be better than an iPad Pro because you can run multiple iPad apps side-by-side in full screen mode, which would overcome one of the biggest productivity limitations of iPads — poor multitasking.
However, in reality this claim really doesn’t hold true at all (at least not yet) for a few important limitations of Vision Pro at launch:
Many iPad apps don’t work well (or at all) on Vision Pro despite Apple automatically opting in developers. There is substantial friction and instability navigating inside productivity apps given that they’re designed for a multi-touch UI (ex: some iPad gestures don’t exist in Vision Pro, and some touch targets are too small). A lot of apps will require some effort by their developers to work well enough.
Most productivity apps are still missing from the App Store (likely for the reason above), which would leave large holes in most people’s workflows. For example, the most important missing apps for my own workflows include Chrome, Gmail, GDocs/Sheets/Slides, Asana.
Text input is still quite buggy which adds more friction to any productivity workflow. Cursor placement, text selection and editing are super error prone. Dictation doesn’t stream results as you speak.
You must carry a keyboard and a trackpad (mice are not supported) for the vast majority of your iPad-class productivity workflows on Vision Pro, which could be an added inconvenience (compared to carrying an iPad with a keyboard case or even a laptop). Editing documents, spreadsheets or presentations without those is virtually impossible.
There is no reliable workspace persistency which adds even more friction — you are forced to re-open apps, and then re-size and reposition windows almost every time. The capabilities we all want (which Apple should be able to ship soon if they want to) are (i) persistent workspaces, (ii) location-specific workspaces, and (iii) a spatial computing equivalent of Mission Control.
All that said, these limitations can all be addressed by Apple and the potential of Vision Pro as an iPad Pro replacement really is there. Even though the iPad Pro has nearly twice the PPD (pixels per degree) as the Vision Pro, text readability of iPad apps on Vision Pro is good enough for you to run 3 or 4 side-by-side apps plus a number of ambient widgets.
I also really believe there’s a large enough white canvas for lots of Apple-style innovation and magic around letting users configure and manage their workspace with a combination of 2D panels and virtual 3D objects. The potential is really significant as long as Apple really empowers developers to innovate here (try Nicholas Jitkoff’s translucent.vision to see some great early examples — the NY Times front page widget is my favorite).
Call me crazy, but I personally could get quite excited by the idea of a “spatial iPad Pro” if I was able to actually get all of my iPad apps on the Vision Pro, and if Apple addresses all of the issues causing friction in my workflows. The reason why is simply that of focus – to be able to really “dial down reality” and tune into the work wherever I am, without carrying my laptop with me but still having some degree of multitasking available.
Productivity Thesis #2: Vision Pro as a MacBook virtual external monitor
Status: ✅ ALMOST READY (needs some bug fixing!)
MY TAKE: The Vision Pro is a few software bug fixes away from being a suitable virtual-equivalent to an external monitor similar to a 27-inch Apple Studio Display that makes it easy to work immersively in VR using all your existing MacOS apps and workflows on a huge screen (but don’t expect an Apple XDR 6K experience!).
One of Vision Pro’s best pieces of pure software/experience magic is the ability to seamlessly connect to a MacBook by simply looking at the computer while wearing the headset. This is a simple improvement to the traditional AirPlay UI that creates a profound sense of seamlessness which VR has always lacked.
Before diving into this thesis, I’ll establish that the Vision Pro will never become a suitable alternative for my office workstation with dual Apple XDR 6K displays. At 32 inches per-monitor and a total of 40 million pixels (with pixel density of 218 PPI and angular resolution >100 PPD) and without a weight around my head, it is simply not a bar I would hold a VR headset against today or at any point in the future.
The more interesting question to focus on is whether Vision Pro could even begin to look like a suitable replacement for one or more 27-inch Apple Studio Displays (or equivalent). The short answer today is that Vision Pro can indeed come close (or very close) to that, but there are some important limitations that Apple needs to address to make this a relatively frictionless use case:
Lack of dual (or triple) monitor support is a huge bummer despite the fact that there are decent reasons for it (largely related to requiring a lot of local Wi-Fi bandwidth). Even though Vision Pro has a relatively narrow field of view that still feels like looking through binoculars, if I could get two or three virtual monitors out of a MacBook Air for example, things would start to look more interesting.
Inconsistent keyboard and trackpad behaviors makes it very hard to switch back and forth between iPad/Vision apps and the Mac virtual display. I constantly find myself looking for my cursor, seeing the virtual keyboard pop up onscreen when clearly I don’t need it (if I’m using a physical keyboard), not to mention that I cannot use my beloved Logitech MX mouse.
There is no reliable workspace persistency which is exactly the same issue I discussed when talking about the iPad Pro use case in the previous section. This should be an easy fix.
Eye tracking doesn’t work in MacOS which not only leads to inconsistent input modalities as I said above but also just feels like a huge missed opportunity to offer a magical capability that MacOS has never seen before. This is not a low-hanging bug fix, but I don’t see it as a huge technical leap for Apple if the MacOS team wants to address it.
MacOS apps are “stuck” inside the virtual monitor instead of being allowed to move around the entire space. This is another missed opportunity to deliver a truly spatial/immersive experience with Vision Pro, though significantly more complicated for Apple to address and would require really careful engineering by both MacOS and visionOS teams.
Many of the issues I highlighted above are straightforward software challenges that Apple is well equipped to address and would make a world of difference. I suspect it’s a matter of dealing with the usual internal politics/collaboration challenges getting the MacOS team to dedicate the necessary resources to address bugs and feature requests from the visionOS team.
The bottom line for me is that we can see a relatively near future where carrying a MacBook Air and a Vision Pro in your backpack could give you a reasonably good workstation, one that delivers enough benefits in the form of productivity gains that you might be willing to wear a headset for a few hours in a café, on an airplane, or even on your couch at home. (This perspective is of course made in complete absence of value-for-money considerations).
Unsurprisingly, this is Apple’s strongest hand with the Vision Pro launch as it’s 100% controlled by them and uniquely leverages the existing Apple ecosystem. Yes, it’s a very uninspiring and unimaginative use case, but it might be powerful enough for Apple to move a lot of headsets.
Watching movies in Vision Pro is great at first but most people will stop doing it after the initial novelty excitement wears off
Watching TV/movies in virtual reality seemed like such an incredibly compelling idea that we (the Oculus team at Meta/Facebook) built an entire product around that idea — Oculus Go. Launched in 2018, Oculus Go was the biggest product failure I’ve ever been associated with for the simple reason that it had extremely low retention despite strong partnerships with Netflix and YouTube. Most users who bought Oculus Go completely abandoned the headset after a few weeks. The full story is much more nuanced (including the fact that the Oculus Go failure got us on the path to Oculus Quest very quickly), but it taught us an important lesson.
The lesson we learned is that watching traditional (rectilinear) TV or movies in VR feels incredibly compelling at first, but the novelty wears off for most people after a few weeks. The reasons are:
It’s just not physically comfortable compared to watching TV or movies on an TV, tablet, or laptop, primarily because of the pressure on your head and face, plus the fact that you can’t comfortably sit in any position or lie down while wearing with the headset
There’s a lot of friction to start watching a video in a VR headset if you’re not already in VR — frequently a lot more steps required (especially finding and putting on the headset) and a more cumbersome navigation UI compared to our other devices
It’s socially isolating and lonely to watch videos in VR, which will be a deal breaker for many people (although definitely not all)
Back in the Oculus Go days, we concluded rather quickly that media consumption in VR is simply not a core “daily driver” pillar but more an ancillary use case that adds some value to other core pillars (such as productivity or gaming).
Vision Pro does bring more to the table with a much better display than previous VR headsets which can create magical movie experiences on occasion. For instance, watching an animated Disney or Pixar movie in 3D is absolutely stunning. But the essential product-market fit challenge remains:
MY TAKE: VR is simply not a medium people will gravitate towards for watching 2D media on a regular basis. Adding to this all of the Vision Pro’s comfort and friction issues, most people who get excited about watching media in the headset will eventually find themselves going back to their TV, tablet or laptop as their primary devices for video.
Watching 3D movies on Vision Pro is a fun entertainment experience, but these videos are “boxed” and don’t feel anything like witnessing real life. With the Vision Pro, Apple launched its new Apple Immersive video format, which opens the door for a new class of entertainment.
Apple Immersive Video opens a new world of possibilities for media in VR — but its hyperrealism may bring an unexpected uncanny valley challenge
One of the big original bets we made at Facebook/Meta with the launch of Oculus Go in 2018 was that immersive 180-degree video would attract a massive amount of consumer interest and that this would somehow trigger a chain reaction in the world of entertainment. We were able to secure partnerships with a small number of media companies who had become specialized in capturing VR video early on, and we were off to the races.
Our initial excitement cooled off quickly. VR180 video quality on Oculus Go was decent but flat, washed out and far from amazing mostly due to low resolution. These videos didn’t create a true sense of presence, of feeling transported to another reality. And most of the content was of one-off nature, with no real franchises that would have people coming back for more (with the exception of sports, which failed initially for other reasons that I’ll come back to later).
Within a year, the Oculus team pivoted to VR gaming and stopped investing in immersive video almost completely.
In 2020, Apple acquired NextVR, one of the small but highly respected companies we had been working with as they were edging into bankruptcy (we passed on the acquisition at Meta/Oculus). NextVR had spent over a decade building and perfecting VR 180 camera technology and production pipelines for broadcast-quality video. The NextVR YouTube channel is still live and provides amazing examples of what became possible with their technology (make sure to pan around using your mouse/finger while watching videos in their YT channel).
The latest publicly displayed NextVR 180-3D camera in 2018 (Source: The Verge)
The NextVR acquisition is what led to the incredible Apple Immersive video format, which enables capture of 3D video in 180 degrees in 8K resolution at 90 frames per second, an absolute juggernaut format with 8 times the number of pixels of a regular 4K video. The best way to think of the new Apple Immersive video format is kind of like a new IMAX-3D, but the real magic is the fact that it’s projected inside an imaginary 180-degree sphere (horizontally and vertically) that takes over your entire field of view.
Vision Pro is the first VR headset that enables playback of 180-degree 3D video at what feels to the eyes like 4K quality. At launch, there are four Apple TV short films on Vision Pro shot in Apple Immersive video format. My absolute favorite of these films — Adventure — is a jaw-dropping cinematic piece that is likely to win its share of movie awards. Experiencing the Norwegian fjords with this level of immersion is completely breathtaking, so much so that it might be my favorite experience so far in Vision Pro. I have never felt transported to another place in this manner in any experience I’ve ever had, anywhere, period.
Adventure is one of the short films shot in Apple Immersive format released with the Vision Pro launch
My second favorite Apple Immersive video — Alicia Keys: Rehearsal Room — is a super fun and intimate concert that really makes you feel what presence in VR could be like with another human. While it’ll be fun for nearly everyone to see Alicia Keys in a close-up VR performance, it may not be nearly as happy and inspiring to see a human in close proximity in other situations.
MY TAKE: The super high-fidelity Apple Immersive video format will run into an unexpected and significant “uncanny valley” challenge as a consequence of its hyperrealism. Seeing someone right in such close proximity to you and in such high fidelity may feel cool to one person but will feel uncomfortable or overwhelming to others. Less so in a scene like this intimate music concert or sports game, but probably a lot more so in dramatic storytelling and other types of more realistic films.
Back in the Oculus days, we used to run experiments to try and really understand which lines could not be crossed in VR content to avoid people feeling overwhelmed or even unsafe. One of the findings in these experiments was that too much realism and fidelity could be one of the things that crosses a line. In other words, hyperrealism could quickly drag people into the uncanny valley, one of two places we always want to avoid in VR (the other place is motion sickness).
Many people will feel themselves crossing the uncanny valley while watching Alicia Keys: Rehearsal Room on Vision Pro
Navigating this creative challenge will take time and a lot of experimentation on Apple’s side, and they’re the one company in the world we can trust to have the level of sensitivity and artistry for this journey, not to mention the ability to hire the best of the best talent. Practically, it probably means we can expect to see beautiful experiential films in Apple Immersive format exploring topics such as beautiful landscapes, wildlife, travel and music, but are less likely to see deep human storytelling with people in close proximity to the camera (which is typical of nearly all traditional filmmaking).
Luckily for Apple, there is one category where hyperrealism is much less likely to be an issue especially for hardcore fans — Live Sports.
Live sports will be Apple’s secret weapon to sell a huge number of Vision Pro headsets to hardcore fans — but it’s going to be a long & expensive journey
One of the original Oculus Go 30-second TV commercials featured an NBA courtside banter between Adam Levine and Jonah Hill wearing the Oculus headset while watching a live game together in VR (each sitting in their own physical living room):
Screenshot from an original Oculus Go TV commercial (full video)
This TV commercial did extremely well, drove a significant amount of Oculus Go sales (after all, that headset only cost $199) and definitely showed that we were on to something potentially quite powerful with hardcore sports fans. But as I explained in the previous section, we did not manage to bring it to reality in a way that would meet expectations.
In the end, our team at Oculus completely failed to realize the opportunity of redefining the sports audience experience through VR for a number of reasons, but primarily because we just didn’t have the patience to develop that market. We were unable to build the necessary industry support with sports leagues and broadcast rights holders initially, so we stopped trying and the VR sports segment nearly died. There are small efforts on Quest today such as Xtadium and Meta Horizons, but the quality of the experience and the limited live content make it all too insignificant to matter. To date, nobody ever really tried hard enough to create this market.
Apple has the opportunity to completely change this, for a few reasons:
Apple Immersive on Vision Pro is a transformative experience in terms of video quality and its ability to deliver a real sense of presence. Watching a game in high-resolution VR has the potential to be legitimately better than a regular 4K TV broadcast by enabling hardcore fans to feel much closer to the action.
Apple has VR broadcast expertise with its acquisition of NextVR, and could have been painstakingly building a robust production pipeline for live 8K video, which is a tall technical challenge requiring non-trivial investment and specialized talent.
Apple is already active in the sports broadcast rights world through their existing MLS license and several other rumored conversations that may lead to Apple buying a lot more broadcast rights to continue to strengthen Apple TV (ex. English Premier League, Formula 1).
The first place where Apple will likely explore using Apple Immersive and Vision Pro for a live broadcast is Major League Soccer in the US. Their recent announcement is a strong indicator this is likely coming in late 2024 or early 2025 (to continue building momentum for Vision Pro):
Coming soon, all Apple Vision Pro users can experience the best of the 2023 MLS Cup Playoffs with the first-ever sports film captured in Apple Immersive Video. Viewers will feel every heart-pounding moment in 8K 3D with a 180-degree field of view and Spatial Audio that transports them to each match.
Apple will likely use MLS as a testing ground for developing Apple Immersive live broadcasts
Beyond MLS (where Apple already has a long-term agreement and the ability to basically do anything), it will take a significant amount of time and money for Apple to strike the necessary agreements with the main sports leagues (NBA, NFL, MLB, Premier League etc) to enable this kind of immersive broadcast experience. That said, this is likely only a matter of time, as the opportunity to rethink audience sports is large enough that it would matter a lot even to a multi-trillion dollar company like Apple.
MY TAKE: Just to put things in perspective, prices of tickets for watching live sports (in the actual venue) have been going steadily up and are now in the $100s even for average to bad seats, with premium tickets easily going into the $1000s (the cheapest SuperBowl ticket in 2024 was around $2,000 at face value). The business case for a high-quality immersive “courtside” experience on Vision Pro is almost unquestionably very strong.
There are two major aspects Apple will have to nail in order to successfully monetize this opportunity, both of which will require a lot of design, engineering and experimentation:
Live sports are very social, which means Apple will have to invest heavily in delivering a co-watching experience that works equally well for people who are physically in the same room or virtually co-located, and which feels as natural as casually watching a game sitting on the couch with your family or at a bar with your friends.
The experience bar will be very high, which means Apple will have to really customize every aspect of the experience to the nature of each sport to make it better than watching a game on a large 4K television — including camera angles, special replays, birds-eye visualizations, analysis overlays, game stats etc.
This is a massive canvas for innovation, and it will take several generations of Vision Pro to get there. I’m optimistic and, speaking from the position of having been part of a team that really tried to go after this opportunity, I really believe this is one of those things where “it takes an Apple” to change the game (pun very much intended!).
Watching live tennis on Xtadium app on Quest: multiple cameras to choose from plus simultaneous TV broadcast on giant virtual floating screenPGA app on Vision Pro: birds-eye view of a 3D model of the course and ability to track shots of a recorded prior matchConcept of a Formula 1 mixed reality broadcast by viz artist John LePore (Source: YouTube): birds-eye track view, main broadcast on giant floating screen, multi-camera access, live telemetry
Conclusions
Why I returned my Vision Pro, and my wish list for what Apple could do to fix & improve the product
As a “product guy”, I usually force myself to behave like a real consumer making real trade-offs as much as I possibly can. I believe that always putting myself in the user’s shoes is an important part of what I do not just for my own products but also for products built by other people. I admit Vision Pro is the ultimate tech toy, but since I’m not an active developer I can’t justify the $4,049.78 price tag (512GB model + California sales tax) simply for keeping up with the VR market, so I returned my Vision Pro for a full refund inside the 14-day return window.
In Apple’s journey of product-market fit in VR, the Vision Pro has a long way to be able to deliver true retention. Apple’s high-risk decision to completely exclude immersive VR games from the Vision Pro app store — plus their inexplicable failure to create exciting momentum by not having high-quality AR apps at launch — don’t leave them with many options to deliver user value in the near term to non-developers.
The only low-hanging fruit is to make productivity really good, which despite being incredibly unimaginative and dull, should be one of Apple’s biggest focus in the next iterations of visionOS. I don’t discard the possibility of once again owning a 1st-gen Vision Pro in the future once Apple addresses all of the friction issues I uncovered and shared above.
During the 2 weeks of my Vision Pro experience, I accumulated a very long list of bug fixes and feature requests based on 2 weeks of Vision Pro usage. I’m going to share my Top 10 here:
Make productivity use cases frictionless first of all by closing the gap with developers to bring essential iPad apps to Vision Pro // fix text input & editing and make it seamless // add support for 2 (and ideally 3) MacOS remote displays // add workspace window persistency // build “spatial Mission Control” and enforce a minimal recommended focal distance
Have developers build amazing AR games and do everything possible to set a really high quality bar & reward their creativity // add SharePlay support with Personas and really push for multi-player support, enabling people to be and play together.
Improve passthrough mode to the extent that the hardware sensor stack allows, ideally reducing motion blur, improving white balance, and making seeing your hands more seamless (when viewing immersive content)
Create workspace spatial persistency and allow me to configure different rooms in my home or office in such a way that Vision Pro always remembers my room-specific configurations
Make 3D widgets & objects first-class citizens in visionOS and enable people to decorate their homes & offices persistently
Let people bring their iPhone into VR by simply looking at the device (like the MacOS virtual display feature) and then getting a floating panel that they can place anywhere in their space — this will work wonders in reducing FOMO while in VR
Add a Guest mode so anyone can give the Apple in-store demo and make it possible for Vision Pro users to “spread the love” — there’s nothing more magical than giving someone their first VR demo
Add Persona support to SharePlay for watching video so people can actually feel like they’re together — VR has a bad reputation of loneliness & isolation, so making VR social must be a priority even though few people will use social features at first. There aren’t enough people with Vision Pro for this to be a practical use case today, but it’s important for Apple to set the right tone. Their plans shared during this WWDC session are a good start, so it’s just a matter of rolling this out.
Launch tons of beautiful environments ideally with a steady frequency, taking a page out of the Apple TV screensaver playbook — and include beautiful indoor environments as well (not just landscapes) that people can walk around a bit more in.
(Lastly…) Find a way to let people play immersive VR games by implementing OpenXR support, forming a partnership with SteamVR or simply opening up visionOS a bit to allow VR developers and enthusiasts to build compatibility themselves
ONE MORE THING: (1) Why Meta’s Android moment is finally here, (2) My unsolicited product advice for Quest Pro 2 and beyond
As I said at the beginning of this essay, while working at Meta/Oculus I used to semi-seriously joke that the best thing that could ever happen to us was having Apple enter the VR industry. One of the main reasons for me to say this was that I knew Apple would do the best job of any company making people really want VR through its unparalleled brand, design and marketing. Oculus co-founder Palmer Luckey puts it best:
“VR will become something everyone wants before it becomes something everyone can afford.”
For Meta, the Vision Pro launch is the best marketing tool for Quest VR that the company could have dreamed of but could have never achieved on its own, for a few reasons:
It elevates VR to a level of mainstream consumer curiosity and breaks away from gamer and VR enthusiast niches; in media coverage alone the Vision Pro probably had 1,000x more reach than any Oculus/Quest launch in history
It sets a new experience gold standard for VR especially by pushing the existing boundaries in display resolution and creating a new paradigm of “UI magic” with gaze & pinch which may be an instant de facto standard
It establishes a pricing envelope that enables Meta to break away from the $500 price point that Quest has been stuck in, and specifically allows them to ship a Quest Pro 2 headset priced at a $1,000 to $1,500 (but likely not higher) without being completely rejected by consumers
It creates a formidable competitor for Meta teams to maniacally chase and will almost certainly force the company to move with a much greater sense of urgency internally (which would be a great outcome as friends on the inside are constantly complaining Meta Reality Labs moves too slowly)
What should Meta do in response to the Vision Pro launch?
In order to really seize this moment and opportunity created by the Vision Pro launch, Meta would need to ship a VR headset by mid-2025 that both builds on the new experience gold standard created by the Vision Pro and is objectively a better product across as many dimensions as possible.
I put together my own Top 10wish list for Quest Pro 2:
Double down investment in micro-OLED as it’s likely the only way to achieve display resolution at or near Vision Pro; I suspect this may be exactly what the recently announced LG partnership is about
Build an ergonomic headsetthat can be worn for 2-4 hours without causing any major discomfort issues; ideally offering two battery options: (1) a head-strap with a built-in battery in the back of the head, and (2) a wired pack (like the Vision Pro) that moves the battery off the head and reduces the headset weight to below 500 grams while increasing energy capacity.
Deliver better passthrough than Vision Pro by dramatically improving Quest 3’s latency and distortion correction and improving upon all of the Vision Pro passthrough issues — ensure no perceivable motion blur, high dynamic range, accurate white balance
Take Apple’s gaze+pinch UI to the next level by productizing all of the amazing research on hand tracking done at Meta (ex: Rob Wang’s super talented group) to enable fine-grained gestures such as scrolling and D-pad selection by detecting small finger movements solely via camera input (this is not the CTRL Labs stack… that’s for the future)
Partner with Microsoft to make Windows computers 1st class citizens in Quest Pro 2 and enable advanced desktop productivity use cases that go well beyond virtual monitors (ex: make it possible to take any window and place it in space)
Launch Android 2D tablet apps natively on Quest to match the Vision Pro iPad compatibility library either by partnering with Google to license Play Store (which seems unlikely these days though I still believe Ash Jhaveri and Hiroshi could work together to pull it off) or just build a curated tablet app store directly (which we had considered in the past at Oculus but passed on)
Launch high-definition room scanning and unlock teleportation using technology that has existed within Oculus Research for several years now; it is time for Meta to make this future a reality where people can be remote but feel truly present by visiting each other’s home, office or favorite place
Appendix: Other Fun Things
Despite all of its hardware insanity the Vision Pro display is still a far cry from a VR retina display (and may never get there)
As noted above the Vision Pro display delivers insane pixel density at over 3,000 PPI (compared to 500 PPI for the highest resolution smartphones), but because the panel is so close to our eyes, it still doesn’t come even close to the resolution it would need to have in order to qualify as a retina display.
A retina display, by Apple’s definition, is a display with a high enough resolution that the human eye cannot resolve individual pixels. Because different devices are used at varying distances from the eye, there is no single PPI (pixels per inch) standard for retina across all device categories. Instead, it’s useful to look at PPD (pixels per degree), which is a measure of angular resolution independent of viewing distance, or specifically the number of horizontal pixels per degree of viewing angle. See this article from SimulaVR for an excellent explanation, and here’s an image from that article:
A human eye with 20/20 vision has a resolution of 60 PPD. This means very specifically that we can resolve 60 pixels per each 1 degree of viewing angle (or 1 pixel per arc minute, which is 1/60th of a degree). The Vision Pro has an angular resolution of 34, which is 1/3 more than the Meta Quest 3 but still very far away from the 60 PPD we’d need for a retina-quality display.
Achieving angular resolution anywhere near 60 PPD in a VR headset is likely not possible with any technology in the mid-term horizon.
Hugo's Blog 