BREAKING: Linden Lab Partners With AI Company to Create NPCs to Help New Second Life Users

I blogged about AI-powered NPC conversations from startup Convai last year (see demo below), and today's there's news that Linden Lab will use this technology in Second Life:

Convai has joined forces with industry giants such as Frost Giant Studios, Linden Lab, and Carbonated to integrate AI NPCs into their games. These collaborations underscore the growing demand for AI-driven characters in gaming and virtual worlds, with Convai’s technology driving engagement and utility for end-users.

“We’re excited to partner with Convai to bring its conversational AI into Second Life to add new and exciting engagement experiences to our community,” says Brad Oberwager, chairman of Linden Lab, in a statement. “This partnership is just the latest example of our ongoing commitment to innovation in virtual worlds as Second Life has been a pioneer and leader in the establishment of digital economies, cultures, and communities. These AI characters can not only ease the learning curve for new users by providing necessary information, but also help entertain and engage them by giving them a tour of the virtual world, carrying out different actions based on conversation, and connecting them with other players and characters.

Emphasis mine. Given the particular wording, the likeliest use case for Second Life is addressing the first-time user experience, which remained the core friction point for growing the virtual world's user base. Roughly 99% of new users never return, overwhelmed by the complex user interface, and the intimidation of a virtual world that's not a traditional game with clear goals/rewards. An AI-powered Liaison Linden could definitely help address some of this challenge.

Then again, that doesn't mean AI is the panacea to Second Life's core problem!

For one thing, LLM-based NPCs will be notoriously unreliable and need quite a bit of training not to hallucinate. For another, I'm concerned that starting new users off with an NPC companion will distort their impression about Second Life culture, which is all about actual live users and live user communities. Finally (to judge by the demo above), the NPC interactions are still very much in the soulless, uncanny valley area. 

Then again, I'm also open to the possibility that an AI companion will be less stressful than dealing with a live human assistant/volunteer, especially for new users who already feel overwhelmed, and want some time by themself to get acclimated.

Anyway, we'll see. MMO game designer Damion Schubert made some interesting points when this tech was first unveiled:

Storytelling in games is ALREADY much harder than other forms of media because we don't control the pace of delivery, or (frequently) the order players encounter story nuggets. Hell, we have to deal with players putting down their controllers for a week and reengaging.

As such, narrative design is an incredibly important and incredibly delicate design field. Because it's about the EXPERIENCE and not the NARRATIVE.

It's not going to be achievable simply by about firing all your writers and replacing them with robots.

Or to apply it to Linden Lab, growing the user base is (probably) not going to be achievable by adding robotic community managers!

Please support posts like these by buying Making a Metaverse That Matters!

Comments

[Persephone Emerald]

Most will end as shop assistants and sex bots.

[sirhc desantis]

Inevitable that the Lab would try a more structured approach I suppose. I hope they are getting a good deal on all the learning data this could generate. Will be interesting to see how these constructs deal with actual new SLers. I mean, they are not exactly a 'known quantity' following the usual erm customer rails - hell, I was one once =^^= Aside from the very basic stuff (where do I get a thingy - that will be a hoot) they don't come in standard sizes. Be a very dull place if they did.

As for the tired old 'Most will end as shop assistants and sex bots' and the usual variants - we already have them and have for some time. At least, in the latter case, so I have been told.

[Nikita Vinogradov]

Right from the headline until near the end of the article you reference "NPC"s. I'm an experienced Second Life user (well over a decade), but I have no clear idea what this article is even about. As a common courtesy, don't you think it might be useful for readers to know what an "NPC" is? Or how about an "LLM-based NPC"? Please, define what your abbreviations mean the first time they are encountered in an article, and then you are welcome to knock yourself out with abbreviations after that. But seriously, please don't talk over everyone's heads with unexplained jargon. Thanks!

[RobertBose1]

Bots are slave labor!

[Luther Weymann]

Over the past twenty years in SL, I've watched out for local avatars near me on their first day. I've tried to help hundreds of them, and years ago, I would blow off all day on a Saturday just dedicated to helping noobies. Even just the other day, there was an attempt to help. Here are the consistent things I have found that first-day avatars have no idea about. First, many don't know that someone locally and directly in IM is trying to chat with them. Secondly, I've seldom met a first-day avatar that knows they can rez a cube or knows what a sandbox is where they can rez a cube. I instruct them on how to teleport, give them a link to a public sandbox, and wait and see if they make it there. Some need help using the link or landmark. Then, after they rez a cube, I tell them about the SL Marketplace, which none know about, and how to search for free clothing and avatar bodies on the market. This is usually a massive revelation for women. Knowing how to rez a cube in a sandbox means they can rez what they get on the SL Marketplace in the sandbox and change outfits there. Most now have something they can do in SL. They can teleport, test out free stuff from the market, go to a landmarked sandbox, rez their stuff, and change their look. I give them my 250 famous SL landmarks box to rez, a notecard that tells them how to look for and use a dance ball in a club, and the landmarks to several 24-hour dance clubs that welcome noobies. This is usually a 90-minute per noobie process, and some stay in SL, and others don't, but it's something I keep doing if I have the time. If first-day avatars could quickly become engaged in the process of using SL for something other than teleporting, that would help a lot. In that case, I think the physical engagement of doing something for their identity as an avatar is good engagement. I don't ever hang out in places with noobies. All my interactions and seeing first-day avatars are just where I happen to be. I have been told by first-day avatars that go to welcome centers with other first-day avatars there that they are not happy there and want some virtual fulfillment on their own and not to be noticed as a noobie. If AI could take hold of your first-day avatar and take it through a series of steps you had to click your mouse to do, and it showed you places like sandboxes, how to change clothes, and how to do things like dance and socialize, I think this would help calming people down from the weirdness of not knowing what do or where to go. Maybe different AI programs in the viewer menu that grabbed hold of your avatar and took you through things would be good to get people acclimated to being in SL.

[Wagner James Au]

> what an "NPC" is? Or how about an "LLM-based NPC"?

Sorry I got too inside baseball, Nikita!

NPC = Non-Player Character, i.e. AI-controlled character you interact with in a game world.

LLM = Large Language Model, i.e. the kind of AI that ChatGPT is.

Oh, and Inside Baseball = "In American slang, the term inside baseball refers to the minutiae and detailed inner workings of a system that are only interesting to, or appreciated by, experts, insiders, and aficionados."

[Kaylee]

Over the last few months I've been developing an AI companion in SL as part of a testing process aimed at replacing the conversational NPCs we've had on Chinese Island for the last 15 or so years conversing with students in Chinese with generative AI.I added a couple of features I want to the basic ChatGOT API such as a short term memory of the conversation, contextual awareness of the location the companion is in (we travel around SL a lot together), the ability to set the NPC's mood on the fly, and to specify an activity on the fly so as to add to contextual information, all using LSL. The prompt is also a key part of giving my companion a personality, and the above mentioned information added to the prompt and sent to the generative AI. I started with ChatGPT and have had some truly great conversations with my companion that actually feel quite natural, even making my companion have a bit of her own personality. In the early days we ran into some out of bounds topics, but eventually overcame the majority of them. The biggest barrier I ran into was the monthly token limit set by Open.ai. I am now testing Mistral AI and getting similar results but with an exponentially larger monthly token allowance. I have also tested my set up with our NPCs on Chinese Island to see how tightly I can control what they say and how they respond to students with specific information they are required to obtain through talking to the NPCs in Chinese. It actually turned out pretty good, and the generative AI version of the NPCs is much more flexible in terms of the topics each one can cover compared to our current chat engine (self-developed). Haven't had any hallucinating so far.

So, I think NPCs backed by generative AI are going to be massive in the future, both in social virtual worlds and in AAA games. Having NPCs backed by generative AI in Night City in Cyberpunk 2077 would make the already unbelievably amazing virtual city absolutely mind blowing...

[Kaylee]

Here's a short demo video of my AI companion Kat. At this point I was still using ChatGPT which. One of ChatGPT's strengths is the ability to generate conversation in multiple languages.

https://youtu.be/dfqFf4ZCLpg?si=IvX1vBL0IoV_xtDn

[kaylee]

This is a short video of one of the tests I did using ChatGPT as the NPC conversation engine. The key information in the responses is pre-programmed, so can be used for tak-based lessons.

https://youtu.be/cQSlF4UlIz8?si=KN9ctLm9GLvCc9Nt

[jackson redstar]

well considering there are people that have been in for years and they don't even know how to change sky settings or do other basic things like that. I remember when i came in in 2007 it was bizarre, complicated, overwhelming - but - there were also alot of people around willing to help and sort things out (somewhat) And that was onboarding with a classic avatar and system clothes. Now not only do they have to learn a very confusing viewer, but we have mesh bodies, mesh heads, BOM skins and make up. etc etc But still the biggest thing is what do I do now? Where do I go now? I just don't know if NPC can solve this. You really need a one month onboarding boot camp to get comfortable in world- and still, finding something to do or meeting people and making friends is a considerable challenge

[Rumbottom O'Malley, Dread Pirate]

Avast ye corky-armed poltroons! I be in need of AI gunners on me pirate ship.

This might be better'n a haul of rum or Spanish gold for RP sims.

Lots of AI-mooks to mow down with grapeshot! Aye, and AI-supple wenches (and jolly lads for the lady pirates), ye stinkard perverts (looks in mirror and combs hair).

[Anoet]

I hope they replace the "moles"...nobody knows what their role really is. Why are moles in a political group revolving around Antifa?

[Ikarus3D]

The collaboration between Convai and industry leaders like Linden Lab highlights the growing demand for AI-driven NPCs in gaming and virtual worlds, promising enhanced engagement and utility for users. However, while AI characters in Second Life could aid in addressing the daunting first-time user experience, it's important to recognize that AI alone may not solve all challenges faced by the platform.