Good points from reader Martin K., riffing off Skarredghost's design thoughts for creating powerful mixed reality experiences for the Apple Vision Pro:
What each app needs to do is to establish what is known in the research on "Computer-Supported Cooperative Work" (CSCW) as a "shared workspace", e.g. a shared virtual 3D model, a shared virtual movie screen, a shared virtual projection screen for a shared slide show, a shared whiteboard, a shared virtual tabletop game, etc. It's not a new topic; some people have spent decades researching it in CSCW.
If an app allows to align the shared virtual screen with a real wall in your room or the shared virtual tabletop game with the surface of a real table in your room, then the app gets extra points for immersion, but that's not necessary. (And if the app requires this alignment, it might introduce additional friction for users.)
This scenario is what Skarredghost labels "You want to have a social AR experience about an element in particular"; according to them, it "was the theme of 80% of the social videos of the Vision Pro on X." I'm not surprised and would assume that an even higher percentage of successful MR apps will have some kind of "shared workspace" - at least that's what you would expect from CSCW research.
Where Martin disagrees is the idea that a good social mixed reality experience should model the real life space of a select member in it:
If 10 people meet this way, for 9 of them it will be basically a VR experience of a virtual reconstruction of someone's room. Thus, for the majority of users it won't be mixed reality.
The real magic would start when you could turn your whole room into a shared workspace, i.e. you would not only share a 3D reconstruction of your room, but track the dynamic position and rotation of interesting objects in your room and you would share the dynamic contents of interesting screens in your room. (One fun and cheap way of tracking an object is to attach a VR controller to it.) If you don't do that, it's somewhat lame.
It's basically as lame as a shared 360 photo (or video) of your room. It's like a Zoom call where someone rotates their device around to show their environment. It might be useful, but not very exciting.
After running my post, I chatted a bit with Skarredghost, who pointed to the mixed reality demo presented by Jesse Schell (above at around 22:19) as an exciting direction.
"Four users play together in mixed reality, and every player has his room joined with the rooms of the other players to create a big hybrid room," he explains in a recent tweet. "Thanks to the scene understanding of Quest, you can also have a rough representation of the furniture the other users have. It's like you remove the distances and you play together in the same space, in this case by throwing each other a hot potato."
For myself, I'm mostly agnostic on the best mixed reality experience, beyond saying it should definitely not 100% recreate our real world office experience, which misses the point of virtual world immersion. But the best MR social experience is the one that attracts massive, continued engagement of users -- i.e. an app that's at least as sticky as Zoom but hopefully much more whimsical and productive. Far as I know, none of the mixed reality apps have yet attracted that kind of success.
Thanks for the link to Jesse Schell's talk! I guess I cannot see how the hot potato scenario is going to be super useful - what good is a mixed-reality multiplayer game where you cannot even high-five other players? ;)
One interesting direction this could take is that of more immersive video calls, see Google Project Starline (e.g. https://www.youtube.com/watch?v=J1oEWiUsKgU ) - even if it is just to re-enact this scene from 2001: https://www.youtube.com/watch?v=TDUxxuNQwEA
Posted by: Martin K. | Tuesday, May 07, 2024 at 01:53 AM