Georgia Tech Grad Adds Augmented Reality Feature to Second Life's Open Source Client

How about a first-person shooter that you can play in the real world, or a filmmaker's tool for displaying space ships and flying dragons out of thin air on a small budget?  That's just two immediate applications that spring to mind with Augemented Reality Second Life, a project from a grad student at Georgia Tech (known in SL as Tobey McElroy, supervised by his professor, known as Blair Potluck.)  As the name suggests, it's a technology that lets you export and merge SL video with real world video in such a way that the image appears in proper perspective, and proportion-- in other words, to make Second Life elements convincingly look like they're part of the real world.  This is achieved, Tobey tells me, with a "fiducial marker", a pattern printed on a sheet of paper that enables their program to properly match the SL feed to the RL video.  (See the YouTube video above, where a moving SL car seems to ride on the surface of a coffee table, or the image to the left.)

But that's only the first part of their Augmented Reality project, because they've also taken a headset display and mounted it with a video camera and a tracking sensor.  The sensor monitors the wearer's motion and position, while the display feeds Second Life video into the headset.

The result?  Life-size SL avatars that seem to be standing in front of the headset wearer.  (This is how you could make a first-person, in-person shooter, among other numerous applications.)

There's already been numerous projects dealing with this kind of mixed reality, one I even mentioned earlier today.  However, says Tobey, "Past attempts and software tools to create Augmented Reality experiences require sophisticated technical knowledge and experience in programming."  So a major goal was to create a program that's merged to the open source version of SL, thus making it a standard feature readily available in the client, one that even novices can use.  "This novel design concept sees an AR experience less like an 'application' that is designed, loaded and executed," as he puts it, "and more like an 'installation' that is created, evolved and performed in a defined area in the virtual world."  For that reason, they call these mergers "AR Stages", like a theatrical space.  "Residents with the AR enabled Second Life client can create and visit such an AR Stage in Second Life and perceive the augmentation in their own physical environment."

The AR Second Life project is actually part of Tobey McElroy's Master's thesis at Georgia Tech. "Depending on public interest and further research projects," he says modestly, "we plan to release our modified client in the near future."

Images courtesy of Tobey McElroy.

Update, 1/16: Corrected status of Blair Potluck per his request.

Comments

[csven]

Nice.

[Blair MacIntyre (aka Blair Potluck)]

Thanks for the great article. My only comment would be that only one of us is a grad student (Toby, who you talked to); I'm an associate professor at Georgia Tech, who supervised Toby's work in my lab. :)

We're really having fun with the AR SL viewer, too; we've teamed up professors and students in our digital media program (http://www.lcc.gatech.edu) and undergraduates in our computational media program (http://www.cm.gatech.edu) to work on AR machinima and AR performance, for example, and are constantly coming up with new things this technology enables.

Looking forward to many more fun times mixing SL with the real world!

[Extropia DaSilva]

On October 16th 2006, Gwyneth Llewelyn kindly printed an essay of mine called 'The Metaverse: Reloaded' on her great blog.

In this essay, I wrote: At the moment, friends in SL feel somewhat separate to friends in RL. That’s not to say that one set of friends is more or less important than the other, but rather that it is very hard for the two to mix. You can either sit with a computer and be with your SL friends, or away from one and be with RL companions. But now imagine being at a party with some RL friends and, upon donning your retinal displays, seeing your SL friends as if they were standing amongst your guests, interacting with them. Or imagine a fancy dress party where a child dressed as Snow White has cartoonish dwarfs dancing around her feet, or a real fireworks display that incorporates SL particle effects for a show worthy of Gandalf.'

It really is impressive to see such ideas being brought to life:) Wonderful stuff!

[semiotimatic]

i can't help but think of practical world-mashing applications like this: http://www.mvs.net/

[Hamlet Au]

Thanks for the correction, Blair, aptly amended.

[Bret Treasure]

You mean my virtual girlfriend will be able to dance naked around my bedroom? And she can sit on my lap in the office taking notes? Nah, that'll never catch on...

[Fire Centaur]

Fantastic work! I'd love to have office hours where I could see avatars come into my RL space through a pair of SL glasses. Can't wait to experiment with with it!