Apple Vision Pro: The Future of Mixed Reality, Great what Apple creates for us

Up to five people can watch movies and TV shows, play AR games, and collaborate on work together even when they’re across the planet.

Apple Vision Pro is above and beyond other mixed reality headsets, but even with the highest quality passthrough camera tech to help you stay grounded in your real surroundings, there’s no way to escape the fact that donning one is a single-user experience. That changes starting today with the rollout of spatial Persona.

At Vision Pro’s launch in February, Personas — the 3D recreation of a person’s head and hands in visionOS — arrived in beta, and it showed. Personas looked video game-like with facial expressions and hair reproductions that appeared more strange than lifelike or natural. Personas still look like the uncanny valley, but the recent visionOS 1.1 software update did improve the 3D avatars with more natural mouth movements and better facial textures for attributes like makeup and hair.

Spatial Persona, which was announced in a developer session post-WWDC keynote last June, is Vision Pro’s answer to its isolating, single-person experience. With spatial Persona, up to five people using their own Vision Pros can participate together within compatible apps without being in the same physical space. In the VR/Meta Quest world, this virtual collaboration is commonly called “presence,” or the ability to feel like a bunch of people are all hanging out together in a virtual space.

Yesterday, Apple invited me to try out a demo of spatial Persona — from the comfort of my own home — with an Apple PR person and a member from Product Marketing. The addition to visionOS is part of the still-in-beta Persona experience, meaning it’s still a work in progress. But even with a few connection issues during my demo, it was a glimpse of how Vision Pro could bring people together to watch movies, play games, or collaborate on work.

After accepting a FaceTime call from Apple’s PR team, I was greeted by two Persona faces inside of their respective 2D FaceTime square “tile” windows. A quick tap in Control Center, however, pulled them out of their tiles and into my living room. The two people weren’t actually standing in my living room, and they couldn’t see my messy apartment either, but their heads, hands, and the upper part of their torso floated in my room as if they were standing in front of me on the left and right. They could move around in their physical space to position themselves closer or farther away from me or each other, and I could too. If you try to walk through another Persona, they turn into a circle-shaped contact card (and they or anyone else will see you doing it). You could be that person and annoy everyone, but it goes without saying that you should respect each other’s personal space, even if it’s virtual.

The three of us could sit down or stand up, and just like in real life, our eye levels would match accordingly. On a phone, tablet, or laptop/desktop, we’re used to not making eye contact on a video call because the webcam is often positioned above the screen. (iOS and iPadOS do have a nifty hidden “Eye Contact” feature that remaps your eyes to appear as if you’re looking into the camera as opposed to at the screen.) So it was more natural to look directly into their eyes and maintain contact; it felt more real, more intimate.

In addition to realistic eye contact, spatial Personas are also directional. That means if several Personas are hanging out together in visionOS, you can see where they are relative to your position. For example, if I point to a Persona on my left with my left hand, everyone else will see that, and the person I’m pointing at will see that I am pointing at them from their right side. It’s basic presence 101 stuff and Apple nails it, even in beta.

Similarly, just like when you’re making a FaceTime call with people (either in 2D or a Persona) in Vision Pro in their tiles, audio is directional (or spatial Apple calls it) as well. Everyone will hear each other’s voices relative to where they are positioned in virtual space.

Once I had picked my mouth off the floor, we got to demoing some content together. First, the three of us watched a movie trailer together using the Apple TV app. There we were, watching a large, theater-sized screen — together. I could look over at them at any time and see how they were reacting, something I actually do when I’m watching a movie with my friends or family in a real theater. (I’m a weirdo!) It was pretty neat!

The cooler part is that everyone’s spatial Persona could get the “best seat,” which is dead center in front of the screen. Changing your seat to direct center of the virtual screen doesn’t affect other people’s Personas; you aren’t all sitting on top of each other in virtual space. In the virtual theater, other Personas appear to you as if they’re positioned elsewhere while you’re watching from the best seat. I noted to Apple PR how this would have been really great when we all experienced lockdown at home a few years ago.

Watching movies together in a virtual theater isn’t new. VR apps like Bigscreen and VRChat have offered this shared-watching experience for Quest, Vive, and other supported VR headsets for years. Apple Vision Pro supports up to five Personas for watching movies and TV shows together, which is a far cry from the dozens of virtual users that other VR co-watching apps offer. That being said, spatial Personas at least look real compared to the cartoony or video game-resembling avatars that populate virtual theaters in VRChat.

Next, we did some virtual work. We opened JigSpace and examined a 3D model of a jet turbine together. Each of us could walk around and through the turbine, enlarge or shrink it, and even detach its parts for closer inspection. I had already seen the jet turbine by myself, but it was still fun to examine it with two others. The one thing I noticed is that if you remove a piece of the 3D model and create a gap or space, you can see a Persona standing on the other side. It’s just as if a person was looking at the turbine from the other side.

After that, we checked out Apple’s Freeform app. It took a few seconds to load up the virtual whiteboard Apple created for the demo. We could walk up to the board and add circle things with the pen tool, rearrange notes, or add to the whiteboard in other ways. In the middle of the whiteboard was a 3D model of a Mars rover that I could manipulate with a gesture or get a closer look from all directions. I’m told five Personas can simultaneously work on the whiteboard or a few can hang back and watch as others work. I doubt this will be replacing Google Docs anytime soon for me, but I could see it being a more immersive way to collaborate for some people.

The last thing in my demo was playing the classic card game, Hearts, in the Game Room app. As I said in my Apple Vision Pro review, the augmented reality game app is one of the best demonstrations of how spatial computing can bring people together. A new update adds spatial Persona support, which means you can play with up to five people (if a board game supports it) or have Personas spectate on the sides. It was here that one of the Apple reps (and another who was invited to join for a four-player experience) had connectivity issues, and so I ended up playing Hearts with one other Persona and two AIs. Connection problems aside, the idea of playing board games together with multiple people and/or having spectators wasn’t lost on me. When I played several games with a friend in Vision Pro, we both wished the app supported spectators, and now it does.

VISION PRO IS A CONTINUOUS WORK IN PROGRESS
Spatial Persona, launching today, is a much-needed addition to Vision Pro to make the headset more than just a single-user experience. When you’re dropping $3,500 on a headset, shared entertainment and collaborative work experiences are expected to be staple features.

Like any first-generation product release, Vision Pro is a continuous work in progress. The fundamentals of spatial computing — responsive eye and hand tracking and high-resolution screens and passthrough cameras — are here, but visionOS has a long way to go. Every software update is going to improve Vision Pro from what it was on launch day. The same was true for the iPhone, iPad, Mac, Apple Watch, etc.

Like any first-generation product release, Vision Pro is a continuous work in progress.
Maybe all this makes the Vision Pro feel like a developer kit to some people. The good thing is, if that’s how you feel, you can just wait. Early adopters, however, will keep getting a taste of the future — today.