The highlight of Apple WWDC 2023 was the anticipated announcement of Apple Vision Pro, a spatial computer that combined both virtual and physical worlds.
It looks like a traditional VR headset, but you can actually view apps, web pages, content, and more overtop your actual background, seeing both at the same time. Fully controlled by subtle hand gestures, your eyes, and your voice, it is powered by Apple’s visionOS spatial operating system.
What is the Vision Pro?
The ultra-high-resolution display system packs 23 million pixels across two displays, and custom Apple silicon in a dual-chip design. Using Vision Pro, you can interact with favourite apps, capture and relive memories, enjoy TV shows and movies, and connect with others in FaceTime. The three-dimensional interface can display apps side by side at any scale. With support for Magic Keyboard and Magic Trackpad, you can set up a workspace or bring the capabilities of Mac into Vision Pro wirelessly, creating a private and portable 4K display.
With two ultra-high-resolution displays, Apple Vision Pro can transform any space into a personal movie theatre with a screen that feels 100 feet wide and an advanced Spatial Audio system. Apple Immersive Video offers 180-degree high-resolution recordings with Spatial Audio, and you can access a line of immersive videos.
Spatial computing makes new types of games possible as well, including over 100 Apple Arcade games on a screen as large as you want, with immersive audio and support for popular game controllers.
With Environments, you can change your own background to one with beautiful landscapes to help focus or reduce clutter in busy spaces. A twist of the Digital Crown lets you control how present or immersed you are in an environment.
Featuring Apple’s first three-dimensional camera, Apple Vision Pro lets you capture and relive favourite memories with Spatial Audio. Access your entire photo library on iCloud, and view photos and videos at a life-size scale. Every Panorama shot on iPhone expands and wraps around you as well, creating the sensation that you are standing right where it was taken.
With Apple Vision Pro, FaceTime calls take advantage of the room around you, with everyone on the call reflected in life-size tiles, as well as Spatial Audio, so it sounds as if participants are speaking right from where they are positioned. Users wearing Vision Pro during a FaceTime call are reflected as a Persona, a digital representation of themselves created using Apple’s most advanced machine learning techniques, which reflects face and hand movements in real time. You can do things together like watch a movie, browse photos, or collaborate on a presentation.
Apple Vision Pro has its own App Store where you can discover apps and content from developers, and access hundreds of thousands of familiar iPhone and iPad apps that run great and automatically work with the new input system for Vision Pro. Apple’s developer community can take advantage of the capabilities of Vision Pro and visionOS to design new app experiences, or reimagine existing ones for spatial computing.
visionOS features a new three-dimensional interface that makes digital content look and feel present in you physical world. By responding dynamically to natural light and casting shadows, it helps you understand scale and distance. To enable navigation and interaction with spatial content, browse through apps by simply looking at them, tapping your fingers to select, flicking a wrist to scroll, or using voice to dictate.
Apple Vision Pro also features EyeSight: when a person approaches someone wearing Vision Pro, the device feels transparent, letting you see them while also displaying your eyes to them. When you are immersed in an environment or using an app, EyeSight gives visual cues to others about what you are focused on.
The Build of Apple Vision Pro
The headwear is built using a singular piece of three-dimensionally formed and laminated glass is polished to create an optical surface that acts as a lens for the wide array of cameras and sensors needed to blend the physical world with digital content. The glass flows into the custom aluminum alloy frame that gently curves around your face, while the modular system allows for a tailored fit to accommodate a wide range of people.
The Light Seal is made of a soft textile, and comes in a range of shapes and sizes, flexing to conform to different faces for a precise fit. Flexible straps ensure audio remains close to your ears, while a Head Band, available in multiple sizes, is three-dimensionally knitted as a single piece to provide cushioning, breathability, and stretch. The band is secured with a simple mechanism, making it easy to change to another size or style of band.
Vision Pro uses micro-OLED technology to pack 23 million pixels into two displays, each the size of a postage stamp, with wide colour and high dynamic range. This is combined with custom catadioptric lenses that enable sharpness and clarity. If you wear eyeglasses, you can use Zeiss Optical Inserts to ensure visual fidelity and eye tracking accuracy.
An advanced Spatial Audio system is core to the Apple Vision Pro experience, creating the feeling that sounds are coming from the environment around you and matching the sound to the space. Two individually amplified drivers inside each audio pod deliver Personalized Spatial Audio based on the wearer’s own head and ear geometry.
Apple Vision Pro uses high-speed cameras and a ring of LEDs that project invisible light patterns onto your eyes for responsive, intuitive input.
New R1 chip processes input from 12 cameras, five sensors, and six microphones to ensure that content feels like it is appearing right in front of your eyes, in real time. R1 streams new images to the displays within 12 milliseconds, eight times faster than the blink of an eye. Apple Vision Pro is designed for all-day use when plugged in, and up to two hours of use with its external, high-performance battery.
Optic ID is a new secure authentication system that analyzes your iris under various invisible LED light exposures, and then compares it to the enrolled Optic ID data that is protected by the Secure Enclave to instantly unlock Apple Vision Pro. A user’s Optic ID data is fully encrypted, is not accessible to apps, and never leaves their device, meaning it is not stored on Apple servers.
Where you look stays private while navigating Apple Vision Pro, and eye tracking information is not shared with Apple, third-party apps, or websites. Additionally, data from the camera and other sensors is processed at the system level, so individual apps do not need to see your surroundings to enable spatial experiences. EyeSight also includes a visual indicator that makes it clear to others when a user is capturing a spatial photo or video.
Apple Vision Pro starts at US$3,499 and will be available early next year in the U.S. with more countries coming later in the year. You can personalize the fit for Vision Pro at an Apple Store location once available.
Will You Use Apple Vision Pro?
The jury is out on Apple Vision Pro and how well it will do in the market. Aside from the astronomical price tag and the long wait time, there are a few things to consider. First is the only two-hour battery life when it isn’t actively plugged in. This makes on the go and “view anywhere” capabilities limited: you’d have to be tethered, for example, to watch a single movie.
Second is usability: when and how would you want to use this? It’s great for travel or while watching alone. Just like VR, it’s good for solitary viewing, playing games, and even work. But what advantage does it provide versus using a big screen you already have at the ready? There are times when it could come in handy, however. I was recently in a hotel, for example, equipped with a smart TV that didn’t allow us to connect to streaming apps. We were able to use an Amazon Fire Stick though still had to connect to a phone’s Personal Hotspot to watch a show on Netflix because the Wi-Fi refused to connect. A headset like this one would have been wonderful…for one person, that is.
This leads to the next downside: it’s a personally fitted device, which means sharing with others is less attractive when it’s designed to fit perfectly to your head and with your eyes. Parents are certainly not going to buy two of these for teenage siblings, and considering it’s so personalized, it isn’t a device roommates, friends, or partners can share either. This is a solitary device designed for use by the owner and only the owner, which limits its usefulness.
With that said, there’s no denying that the concept is incredibly cool. The ability to meld both your real and virtual worlds is great for multitasking: you don’t have to completely shut out the world to enjoy an immersive experience. Nor do you have to remove the glasses to talk to someone. But is this really what the future will look like?
Perhaps when the cost comes down, the Apple Vision Pro will become the next evolution in virtual and augmented reality. It might be a tool business owners have, schools use, and families own at home for gaming and entertainment. Right now, it’s a step above pie in the sky with an official launch date that’s actually here before we know it. Once the reviews are in, we’ll have a better idea of how Apple’s vision for Vision Pro plays into real, and virtual life.
See our Apple WWDC 2023 highlights for more of what Apple introduced for this year.