"Why Sci-Fi Is Bad for Design" is a new Modus essay (reg. req.) by my colleague Amber Case (a fellow at the Institute for the Future) delving into some classic science fiction movie and TV tropes -- such as Jarvis from Iron Man and Star Trek-esque voice control AI -- that have infected the tech industry for the worse. Another trope she discusses is seriously significant for AR and VR developers: Minority Report-esque gesture control. Here's the problem with that, as she sees it:
Since its debut in 2002, Steven Spielberg’s Minority Report has been an archetypal reference point for technologists enthralled by the way Tom Cruise could manipulate complex data with dramatic swipes of his hands.
But what looks amazing for a few minutes of screentime doesn’t necessarily perform as well offscreen. Can you imagine how sore your muscles would be after using this system for eight hours a day?
And as it happens, she can:
On a visit to the MIT Media Lab in 2008, I tried out a gesture control system just like the one from Minority Report. At first, I loved how it looked and felt, but I quickly noticed my arms aching. While gesturing with my hands above my heart, less blood pumped into my arms, causing strain. I couldn’t possibly imagine using a gesture control setup like that for over an hour. Picking up and dragging files is extremely tedious. Tom Cruise even shows us how frustrating the user interaction is in the actual film: There’s a scene where he reaches out to shake the hand of a visitor, and everything on his screen drops! Nevertheless, Minority Report has inspired businesspeople to dream up systems that would require people to use controls like this all day.
What’s more, I realized that not everyone has the fine motor control to learn precise gestures and perform them with consistent accuracy. The system would have to filter out quite a few false positives, which require far more computing resources than a basic button or multi-touch system. Anyone familiar with Fitts’s Law, which suggests that user response time greatly depends on the size of a target and the user’s distance from it, can see the problems inherent in the Minority Report system: It requires large, sweeping motions for even the smallest of interactions, while similar actions aren’t clustered together. I would never want a system like this to play Mario Brothers when a simple wired video game controller is precisely responsive 100% of the time.
Tour of Dynamicland from Amber Case on Vimeo.
As a feasible alternative, she suggests this system (above) from a non-profit tech lab. For VR/AR developers, they probably want to keep the Minority Report style hand flailing to a minimum, for occasional but dramatic moments in the app/game/world they're making.
Comments