Along with announcing his return to Linden Lab leadership on Wednesday, Philip Rosedale and I also had a wide-ranging conversation on multiple topics around the future of Second Life, AI, and beyond. The company is hosting an Ask Me Anything appearance with him in the virtual world tomorrow morning (details here), but I got a chance to chat with him about:
- Deploying AI applications in Second Life -- and why the virtual world is a better or a more promising test bed for AI than the real world.
- Spatial audio upgrades to SL.
- The future of FairShare, his digital currency project launched last year.
- Linden Lab's plans for the Second Life mobile app -- including gamification features.
Read all that and more below! But first, to help support my writing feature articles like this (and to get special benefits) please consider joining/subscribing to my Patreon.
Deploying AI Applications in Second Life vs. In the Real World
Philip Rosedale: The Optimus robot in your house, that's still a ways away. We got problems with batteries and stuff. AI is not going to invade our day-to-day social lives in the real world.
WJA: It's just creepy too. Who the hell wants a man-sized robot staring at them?
PR: Did you see that [video] with the guy doing the “robot left in my house for a day”, and it's remote tele-op? Talk about creepy.
WJA: A guy in the Philippines piloting it, right?
PR: It's like, okay, I get it, but I'm not sure I'm happy with that. I don't know whether that's better or worse. [As] the philosopher and the futurist in me, I don't even really know how to respond -- you have a remote-operated servant robot that somebody is running, I didn't even know where that goes.
But anyway, I think that, on the other hand, a lot of the experiments that we've actually done, both at the Lab and at my lab, when you're talking to AI and avatars, that actually do look kind of interesting. It feels like there's going to be some great breakthroughs there.. So that's an area of focus for us.
WJA: I covered the ConvAI integration from earlier this year, and it didn't seem to work that well, to be honest. And then there's been a bunch of user-made ChatGPT integrations. And it seems like it's fun for a bit, but it sort of wears out quickly, and it kind of feels like, ultimately, it's just bots. And you know that we've had traffic bots and sitting bots, all kinds of bots, sex bots, of course… [But] a virtual world is all about people. So how do we balance the desire for actual human beings with having bots that are cool?
PR: I'd say a couple things about that. First of all, I agree with you. Like most of the experiments -- actually this applies not just to Second Life AI, but applies to AI in Silicon Valley -- a lot of this stuff is fun for two or three times, then you stop using it.
Like copywriting, right? Can AI write copy? Well, sure it can, but not very good copy. So really, how is that useful to you? I guess if you're just a SPAM advertiser. But other than that, you still have to call Hamlet.
So I would say with Second Life experiments, yeah, I agree. Stuff like ConvAI, it's still very early, there's too much latency. There's not enough awareness in the world. That's the other thing about talking to a real person in Second Life, is they're obviously a real person who's perceiving Second Life with you in a way that is complete and rich, so you can do things together.
That said, though, there are a few things that I think will happen that you haven't seen yet with bots. One of the things that's really interesting about Second Life, and actually it's exceptional, people have reached out to me and to us about this:
There's so many millions of user-made, meta-tagged objects in Second Life, someone once turned their massive inventory into a performance art piece
The [virtual] world, as you know, is the size of Los Angeles and it has a lot of content in it, right? Like tens of millions of things on the ground -- but all of those things are labeled, which is pretty cool.
When you drive around a Tesla and it takes pictures of everything, or a Waymo, none of that stuff is labeled. It's just video that's coming in from LIDAR and video. In Second Life, if you walk around as an NPC and you're looking at the world, you can see the world, and you can also, if you’re an AI, at that same moment, right-click on all that metadata. So you're associating the visual image with all the metadata, like chair, post, wall, whatever -- all of that stuff is available to you as AI, and as you know, that's what AI thrives on: Multimodal metadata. That's kind of the whole thing.
The problem with Sora right now, for video AI, is that it doesn't semantically understand the scene yet. That's why everybody's like, “This is kind of stupid.” Somebody walks behind a post in [a Sora video] and a different person comes out behind the post.
But labeling in Second Life is incredible, because everything was built by individuals who were putting labels on it so they could sell it. So that's just one kind of nerdy thing about it. But to your point, it's going to be a lot of interesting work to actually get to something that's compelling.
There's also different categories of use, right? What if the AI could help you get dressed? That actually would be pretty cool… [An] AI might be able, for example, to help you put clothes on or change your clothes. Imagine the AI getting a permission and saying, “Hey, Hamlet can you click on this dialog box?”, and then, “I'm going to put another shirt on you.”
WJA: So do all the annoying fiddly bits in the background. That's good. I think pets too [for] AI, because it's already very popular. But actual, really intelligent ones that are, you know, as smart as your actual dog.
PR: The other thing I wanted to say about AI -- aside from, I agree with you, it's hard, I don't think we've seen the breakthrough stuff yet, we'll have to invent it again, that sounds fun to me because that’s what I do -- but the other thing is, the ethical quandary is so important.
Looking back on 20 years now, I really do feel like Second Life has actually been an exceptionally positive experience for the most part for a lot of people. And I think we've got to apply the same ethical compass to stuff like AI.
For example, I don't think we should build companions that replace other people, period. You know, I've said this before, you've heard me say it on Twitter -- what are the ethical rules of AI? I have a few that I throw out almost every couple of weeks on Twitter because I'm like, my God, listen to me before it all ends. People have to increase, not decrease, the frequency and intimacy or depth of communication they have with other people.
So when you think about AI and Second Life, I want that AI to be a matchmaker between real people. One of the fantasies I’ve had is, imagine you have an AI that’s like a bot that says, “Come with me, James, and then it takes you, and it introduces you to somebody you might like. “I’m going to take you to this island where this guy built a lot of stuff I want to show you.”
That feels to me like a good use of AI. Having the AI be a sex bot, but you fall in love with it forever, does not feel like a good idea to me.
The good thing, unlike, I guess, politics, is there's a pretty bright line around some of these things. So I think we can define it, and then stay on the right side of it. But I think there's a lot of questions around the impact of AI on us as humans and whether it makes us friendlier to each other.. That's part of the reason why I'm excited to come back and work on stuff.
On FairShare, Rosedale’s New Digital Currency Startup
WJA: Where do you see FairShare? Do you think that might be integrated?
PR: On pause... I am 125% or so a full time [Linden CTO] right now… I may well find someone new to take on the development [of FairShare], because we're pretty close to moving to that next phase of FairShare which is the mobile app. So if I can find somebody to help me do it, I'll be totally stoked. But my first priority without question is working here [at Linden Lab].
On Spatial Audio, Speech-to-Text (And Vice Versa)
PR: I don't know if you remember, but some of the people who came from High Fidelity to Second Life are audio experts, and so we were battle testing… our first 100 person pile-on to a totally new spatial audio server for Second Life that we were testing just an hour ago.
My hope with that would be, imagine Second Life, imagine text and audio, everything is converted to the other. So if you type text, it says Audio [as a UI option]. If you want to hear it, you just hear it. If you say something, it captions it and puts it into text. And so imagine Second Life being so accessible, because you can just use text or voice, you can turn the audio off, you can see the captions, we can start translating stuff to people in different languages. There's all these opportunities that I think are examples of how that's a really nice thing.
WJA: People have been clamoring for that. Also, speaking of spatial audio, do you think we'd be able to improve live music with that? Like you can have more people?
PR: Yep, stereo transmission, more people, lower latency. So yeah, yell at the band, they can yell right back at you. Well, if you remember High Fidelity, the same architecture we're building here will allow stereo streams and the whole thing. So you can do spatial and then if somebody wants to broadcast stereo, you can kind of set up the speakers, or whatever you want to do. So we'll have all those same capabilities built into Second Life as we modernize that system..
So that's an example of just cleaning up, modernizing, making more effective that system. I'd like to see voice and text turned on everywhere, exactly the same. I don’t know about you but if you're gonna modernize Second Life, you gotta look at the fact that voice in gaming is pretty expected now, right? You got moderation tools around it; we're working on that too.
On the future of Second Life’s mobile app
WJA: So with the mobile, are you driving just the back end stuff? Are you on the product, or where do you see yourself?
PR: Oh I’m on everything. As CTO, I'm looking at all of it…
My first hope with mobile is that it’s a new way to see the world, to stay in touch with people that is lightweight but doesn't replace the desktop. But it makes the expansiveness and the reach of the experience [grow].
Imagine being able to randomly teleport to places very, very, very easily on a mobile device, and then bookmark them for later, right? So then you get home and you're like, “Oh, I got a bunch of weird new stuff I want to go see with the right outfit on or whatever.” But I can do that once I'm at home.
I think the way that we'll approach it is to start without the presumption that we can just magically make a mobile app that, because you can play Fortnite on mobile, means you should be able to play Second Life on mobile. That feels like a big assertion. But a small one is a huge subset of things that you already do, if you get into Second Life is, you know, get notifications about events, talk to your friends with chat, see a new place, see a rendering of a new place, and be like, “Wow, I want to go see that later.”
Stuff like that can be done. Whether you turn your phone sideways and go to a club in Second Life and hang out, I don't know yet. I hope we can do that. We certainly have the rendering engine to do it, the initial work on it. The initial work on the renderer is very good -- it's Adam’s work or some of his work, it’s strong. So we're definitely going to lean into that. We've got a pretty good renderer, so we don't have to make it just a text companion or something like that.
WJA: The other thing that jumped out with me talking to Brad is we’re going to have a gamified element to encourage logging in and so on, getting rewards. So… we're a bit of a game!
PR: The thing that made me really laugh, it made me really smile [and say in a meeting] “I love this place.”
We had some gaming loops we're playing with on mobile, but we actually put in the settings you could turn them off if you don't want any gaming.
That's the conversation that the world needs right now -- do I want my dopamine or do I not want my dopamine? I think it's pretty funny to say, “We'll just give you a choice.” But yes, we are playing with gamification elements as well.
Please support posts like these by buying Making a Metaverse That Matters and joining my Patreon!
Now if only AI and sort my massive inventory, i'm all in!
Posted by: Imp Trollop | Thursday, October 31, 2024 at 04:27 PM
Voice to text and back sounds great! SL, like almost no other platform, has both types, the chat people and people that only like voice, so to have a way to mix them will help many people. Gamification is very much needed. There needs to be incentives for people to make games also. The AI stuff, I've always thought a virtual world would be the best place to train them. Like Philip says, everything is labeled. I would not think of AI has something separate from us, but an extension of us. Imagine you have an AI pet. If you are in a game, they are there to protect you, or help you in a fight. In the real world, they would do all the things you don't want to do. Again tho, as an extension of you.
Posted by: Medhue | Saturday, November 02, 2024 at 01:12 PM