Good Lip Sync Technique For Second Life Machinima

Among my biggest personal irks against most Second Life machinima is excessive use of lip sync, which really got out of hand when VoIP-driven voice animations were introduced last year. In my opinion, we're still several years away from convincingly matching dialog to an avatar's mouth and facial movements; for now, it's distracting and almost always weird. (It's all too often an easy crutch for relating narrative, too.) This isn't to impugn just SL machinima, by the way; even Half-Life 2's vocal animations, still the benchmark, are far from perfect. (If you don't believe me, watch this HL2 version of A Few Good Men.)

Which brings me to this slickly produced machinima advertisement created by Skribe Forti; product aside, Forti employs a clever technique with lip sync, quickly tracking/zooming toward or away from the avatar as she begins to speak. That way, you see enough mouth movement to establish who is speaking, but not enough to feel too niggled by the audio/visual mismatch, when lip sync lacks sync. What other methods works as well to you?

Comments

[Adric Antfarm]

This is kind of dumb, but instead of a microphone, I feed my mp3 player into the sound card since i can control the lip movement using the pause button very quickly. Any dubbing is later and i can time better with the lips done (or the avatar is made to say more or less as needed).

[Elrik Merlin]

My personal suspicion is that the problem with SL Voice lipsync is that for the last couple of Viewer releases the sensitivity has been set too high. If the lipsync triggered only on peaks it would be a lot more realistic. As it stands, the sensitivity is so high that even leakage from headphones triggers the green "waves" and also lipsync, which I presume are driven by the same data.

We (Designing Worlds, Treet.tv) actually use Skype for audio as SL voice is so idiosyncratic, and I've found that turning the SL Voice input level all the way down can help (as we aren't using the audio). But it's still waaaay too sensitive and I'm still not happy with it.

We have the additional consideration of being live so offline techniques don't do us a lot of good.

[skribe]

Thank you for the mention. It's appreciated.

Machinima and especially SL machinima is an evolving artform. There's constant experimentation involved because the tools of our trade are constantly changing and improving. I would have given my left proverbial for even bad in-world lip-sync 2 years ago. Same thing with flycam. Now I would give it up for better avatars capable of decent facial expressions and fine motor control (ie finger movements). In the meantime we'll just have to keep using tricks to hide the problems =).

Once again, thanks

skribe

[Robustus Hax]

A new version of Crazytalk has been released, and looks to have some nice features. If you really are handy with video editing and machinima you can combine Crazytalk and SL for some nice visual