DeepMind Tech Uses Smartphone Camera to Capture User's Expression & Movement, Then Replicates It on Phone's Avatar in Real Time

From DeepMotion, the AI company which taught NPC avatars how to argue, and how to learn how to dribble basketballs, new avatar technology coming for the Samsung Galaxy S10:

Led by industry experts from Blizzard, Disney, Pixar, Ubisoft, and Microsoft, DeepMotion leverages the latest in AI technology and robotic control theory, allowing users to instantaneously animate their emojis simply by moving in front of the phone’s camera. With DeepMotion’s Digital Avatar simulation, users will now be able to use their body to express themselves more fully. The new technology will bring body language to emoji communications, which to date have been limited to facial expressions and simple head movements. Users can now capture body movement using the rear camera, or upper body movement using the front-facing camera.

More here, another video below. The demo video leaves me a bit confused as to how the avatar interacts on the phone itself -- in an app, or on the default screen, or what -- but if the real time motion capture works as well as represented, I'm impressed. And plan to check this out in person next week at GDC.

Comments

[Bavid Dailey]

Just 2 points to make on this "advance"

1) I don't want to drive my avatar in this way. I ma sitting on a chair for typing this comment and I will sit in the same chair to "pilot" my second life avatar. Not stand, not walk about, jump, throw arms about , etc. Nor do I want my real life facial expression mirrored by my avatar, because ... it's an avatar!

2) if I am physically unable to do these motions, I'd be quite unhappy. Second Life has a strong membership from people who cannot. Not to be written off lightly.

My geek side does appreciate the technical wizardry (with some reservations). Might be a nice way to create animations.