Avatar Kinect Conundrum: Do You Want an Avatar Which Only Expresses What You Want, or Expresses What You Really Feel?

Here's a cool tech video explaining how Avatar Kinect, which I think promises a revolution in virtual world technology and adoption, is able to map your genuine facial expressions onto your cartoon avatar. It also raises a very interesting question, because Avatar Kinect is a model for 3D avatar representation that's quite different from what we're accustomed to so far. Watch first:

Two observations: The use of cartoon avatars (as opposed to realistic ones) is probably a technical necessity, because it's likely impossible to capture and simulate all the facial activity which goes on when a person smiles and frowns and so on. Also, with a cartoon avatar whose facial expressions are depicted in broad, exaggerated animations, it's easier for other viewers to see what emotions are being expressed. (Especially since they're probably sitting on a couch 8 feet from the screen.)

If that's right, I doubt it would be practical to use this facial recognition technology with hyperrealistic 3D avatars we're used to in Second Life, OpenSim, and Blue Mars. Typically, avatars in those worlds express a lot about the user's personality (via fashion, custom animations and poses, and so on), but they rarely express what the user is feeling right at that very moment.  (Unless, for example, a sad user takes the time and effort to launch a "crying" animation.) Doubtless the facial recognition and depiction technology will improve, but for the near future, it's probably a case of going with one model of avatar representation, or the other. So that's the question: All else being equal, would you prefer an avatar which only embodies expressiveness that is controlled by you, or would an avatar which expresses what you're really feeling in real time (often, whether you want to or not)?

Hat tip: Kotaku.

Comments

[Niko Donburi]

"All else being equal, do you want an avatar which only embodies expressiveness that's totally controlled by you, or would you want an avatar that expresses what you're really feeling in real time?"

While reading this, I was reminded of the Japanese concepts of "honne" and "tatemae." Roughly translated, honne refers to your true (inner) feelings while tatemae is the mask you wear for others, which necessarily may be different than how you feel.

Of course, in your example it would be:

inner feeling (honne) --> face (tatemae) --> avatar reflection (in-world observation)

I would prefer the first option, totally controlled expression, to the second. That would allow me, and not some software algorithm, to determine how I come across; much like an emoticon is used in local or IM. Otherwise I fear there would be a lot of mistake, misunderstanding and miscommunication.

[Nalates Urriah]

The cartoon faces don't work for me.

There is a measure of playing with dolls in the realistic avatars. People explore clothes and makeup. All of that is lost with a cartoon avatar.

I suspect the general casual conversation Facebook crowd moving from email/texting to cartoon avatars would find it a step up.

I'm not sure if the cartoon avatar is manipulating a mesh or textures. Avatar expression in SL is mesh manipulation. That suggests to me the infrastructure for Kinect to connect to and control is already there. So, I don't see it as a huge technical hurdle.

[Aeonix Aeon]

I believe Kinect and 3D Camera technology in general is a very good step forward for allowing avatars to show in real time facial expressions that are normally subtle cues in conversation.

As I've said quite often, text (and often times just voice) is a poor carrier of true emotion and intention.

[Adeon Writer]

I'd love it if a webcam could map my expressions in real-time. I'd probably make a human avatar for myself just to be able to use it. (Something I haven't quite done yet.)

[Senban Babii]

Notwithstanding the many problems I'd have with such technology anyway, given that Distance Lab and its projects have been wound up, I doubt there's really that much of a market for "emotion at a distance" beyond initial gimmick value.

http://www.bbc.co.uk/news/uk-scotland-highlands-islands-12160829

[Arcadia Codesmith]

"it's likely impossible to capture and simulate all the facial activity which goes on when a person smiles and frowns and so on."

I would expect if Xbox and Sony were truly interested, they could produce a dual-camera system (or dual lenses with alternating shuttering to the same plate) that captures a wide angle for body motion and posture and a zoom that can not only capture the face, but also track it as the user moves.

With such a system, you could have both options - hi-rez tracking of realtime facial expressions, but with a manual override for playing poker or running for political office.

[Angèke]

This is amazing ! Now to transfer that onto our avatars and not some cartoon image, that would be truly something! This is pretty awesome, any way you look at it and a step up from what's available today. Thanks for sharing this, Wagner!

[JeanRicard.broek]

In regards to amazingly detailed facial recognition of non cartoon characters see the Dec 22nd post: http://jeanricardbroek-architect.blogspot.com/2010/12/facial-recognition.html

[Maria Korolov]

I can see this as the logical evolution of lip-synching. I'd love to be able to make my avatar smile, or share her head, without having to remember how to do facial expressions -- by the time I remember the commands, I'm probably no longer in a smiley mood, anyway.

Being able to do hand gestures would be great, too.

It will help machinima quite a bit as well -- instead of having to hire an actor to control the avatar and an animator to handle the gestures and behaviors, you could just have the actor act out the gestures, making machinima faster and less expensive.

-- Maria Korolov
Editor, Hypergrid Business