New World Notes

Metaverse Interoperability Across Multiple Platforms Was Tried By IBM & Linden Lab But Didn't Catch On -- Here's Why.

Why the Metaverse is Beating Gen AI https://t.co/0CegcGDiSk

— Open Metaverse Interoperability Group (@open_metaverse) September 6, 2024

Thanks to everyone from the Open Metaverse Interoperability Group especially host @jimmy6DOF for the fun and spicy conversation last Friday -- you can listen to the whole thing above (starting around 4 minutes in), with some particular spiciness around 36:30, where I mention that metaverse interoperability of virtual objects has been tried before and fairly failed, a topic I go into at length in my book. So below is a somewhat edited excerpt on that subject:

 

WATCH ChatGPT Generate Working Terrain in a Virtual World When Prompted

When I recently wrote about all the impressive innovation happening on Wolf Grid, an OpenSim-based virtual world developed by small team, I didn't even mention one of its coolest breakthroughs:

Wolf Grid has an option which uses ChatGPT to generate usable terrain with a prompt.

Watch above, with the money shot happening around 6:30 in. In the first demo, requesting a "cat" turns the land into a mountain range that's a picture of a cat (kinda sorta), while requesting a "maze" actually generates a working maze that springs up on the land. 

"Basically," lead developer Lone Wolf tells me, "we have some software between the AI and the grid that deals with any issues."

From the user perspective, they communicate with Bobby, the Wolf Grid's AI assistant, which is integrated with ChatGPT.

"[We] use our software to decide what to do? Do we need to generate an image? Do we need a terrain? Do we need to answer a query? Then it works out which 'bit' of ChatGPT to talk with, meanwhile recording information so it's able to know who said what to him, and then interpreting the information back from the AI and translating to a useable thing."

Wolf tells me Bobby will eventually be able to make all this work on the grid live. Unless I'm mistaken, this is the first instance of a virtual world actually using ChatGPT to generate physics-enabled terrain, as opposed to non-interactive, diorama-type backgrounds. (Correct me if I'm wrong, readers!)

Getting generative AI to create working virtual world terrain is more challenging than it might seem, but Mr. Wolf tells me they've done that:

 

Readers Enthuse Over Wolf Grid's Many Virtual World Innovations

I had worried my story on OpenSim-based world Wolf Grid might be too niche for readers, but to judge by reader comments, many virtual world veterans are giving it and other OpenSimulator worlds a closer look.  

Soda Sullivan argues that more innovation is happening across OpenSim worlds for the very reason they're often developed by small teams:

I decided to give OpenSim a visit and was pleasantly surprised. It is not just Wolf Grids that is innovating. Many regions are pushing hard and fast at adding new option to their grids. I would guess this is a benefit of having a small group (or even a single person) focused on their grid and looking at and implementing innovation. Wolf Grids is certainly a perfect example of this. Once once the new advancements are realized, they move very fast across the entire OS system. Weeks, not months.

One example is Wolf Grid's voice-to-text/text-to-voice function, which many hope Linden Lab would add to Second Life one day -- watch below:

Exploring Wolf Grid, OpenSim World With AI Integration & Other Innovations. Only 3 Years Old, Its Landmass is Already Larger Than Second Life.

I haven’t heard much in the way of innovation in OpenSim virtual worlds lately (though maybe I haven’t been asking in the right places), and somewhat assumed the open source version of Second Life had reached a sunset point. But someone I trust recently told me about the Wolf Grid, and my jaw steadily dropped.

For one thing, it’s already larger than Second Life in terms of virtual land mass, but that’s just the start. 

“We have a lot of features other grids don’t have,” founder Lone Wolf tells me. “Text to speech/speech to text. We can stream our computer screen in-world and share it with others including audio.” (Very handy for group movie nights.) “Our assets are stored on 10.2 petabytes of cloud storage with Cloudflare rather than standard server storage, so we have our own custom asset servers.” (More on its “hyperconverged infrastructure” below.)

But Wolf Grid is even more notable for the latest implementations of generative AI in virtual worlds:

“We have our own AI terrain generator that can randomly generate terrains for your region - so if someone buys a region, within 5 minutes they can have it terraformed with mountains, valleys etc.”

Speaking of which, landowners get a copy of Bobby, a bubbly bot integrated with ChatGPT. (Watch above, hosted by Wolf.)

“You can come and meet him on the Wolf mountain region,” says Lone, but warns: “He has been getting a bit full of himself lately.”

How did all this happen over the course of its 3 year history? Mr. Wolf told me the story, which inevitably begins 8 years ago in Second Life, with a desire to expand into a new frontier:

Bored With Zoom, Professor Teaches Computer Programming in a Virtual World, And Streams It To Her Twitch -- Here's Her Advice for Other Educators

Starting at 12:30pm PT today, you can watch this Twitch channel to see yet another clever solution to teaching a large group of students in quarantine times: UC Irvine computer scientist Crista Lopes, being "sick and tired of teaching in Zoom", moved her class of 150 students to OpenSimulator, an open source virtual world. And not just to stream a video lecture to an assembly of avatars, but using OpenSim's scripting functionality so students could actually learn programming by using the virtual world's language, and building applications within it:

"The course is ICS10, a General Education course in the spirit of CS Principles," she recently reported. "There's 150 students enrolled. So far, the students have programmed in assembly on a simulated computer, used various tile walls with assorted encodings, and are now deploying and configuring a simulated Internet inside the simulated world.

"So far, everyone seems to be turning in their projects on time, and engaging. And a curious side effect has emerged of a student creating YouTube video tutorials showing how to do the projects." (See below.)

As regular readers know, Professor Lopes isn't just an OpenSim user, but one of its early pioneering lead developers. So I was curious how she customized OpenSim for her class, and how challenging it was for students to use. OpenSimulator is basically a reverse engineered version of Second Life, with a notoriously difficult UI and old graphics; Fortnite or ROBLOX it is not. (In fact, her class uses Firestorm, a viewer for accessing both Second Life and OpenSim.)

"I was expecting it to be problematic," she tells me, "and I was ready to spend many hours doing tech support, but it was surprisingly easy. There was no major hurdle, shockingly!" However, having no hurdles required a fair amount of preparation on her part, which you can read about below.

Challenges aside, she recommends OpenSim as a teaching platform to other teachers, if their course work is enhanced by a 3D immersive space and simulations --  my conversation with her below:

Virtual World Simulation of COVID Infection Significantly Improves Student Awareness of Social Distancing

Among the many problems the COVID pandemic has created is the challenge of opening up elementary schools while also teaching kids the importance of maintaining social distance.  But how do you teach social distancing to kids who are learning remotely? 

Enter the SORBET project (for Socially Responsible Behaviours through Embodied Thinking), created in OpenSim, the open source virtual world platform, launched while Singapore was under quarantine. Lead developed by Kenneth YT Lim, a researcher with Nanyang Technological University in Singapore, the SORBET simulation gives kids treasure hunts and other mini-games to play -- with the likelihood of infection, if students allow their avatars to stray too close to each other. (See above.)

Could Second Life Slowly Migrate to Unity? Maybe, Says Veteran Developer Adam Frisby, But Here's Why That'd Be a Massive Undertaking

 

Last week in our conversation about whether Second Life should upgrade to a modern 3D engine, I suggested that Linden Lab could slowly shift toward Unity, the world's most used 3D game engine. But there's what might work in theory, and what is actually feasible. Developer Adam Frisby has been working on a new virtual world platform based on Unity for years, and has a long history developing with Second Life -- he's also a co-founder of OpenSim --  so I asked him to explain if my idea was even worthwhile.

"SL is a very unique beast," Adam tells. "And frankly, they'd gain very little switching to Unity, and lose a lot. It's already a mature product, and SL relies on doing a ridiculous number of very simple things, very fast. Whereas most games rely on doing a lot of very complex things, but at a reasonable pace.

"I think SL has something like a total of 5? very simple GPU shaders. The average game probably has no less than 100-200 complex ones on screen at any one time. The moment you open the potential for any kind of complexity, and SL's performance would tank even harder. Because no one has ever optimized anything, and the only reason it could ever even think of running at all, is because the rendering complexity is somewhere between 'jack squat' and 'nothing'."

But my specific proposal is to add Unity inside the Second Life viewer -- and only switch from SL's legacy OpenGL-based graphics to Unity, whenever a user teleports to a new, Unity-compatible island or continent: 

Text-Based Viewer for Second Life & OpenSim? There's an App for That

Linden Lab may be finally ready to release an iOS "companion" app for Second Life, but reader "Saga" points out there's already a third party SL app in the App Store: "There is another app for SL chat/inventory for iOS. I use it now and then, it’s called MetaChat!" And so there is:

MetaChat is a text-only viewer for all Virtual Worlds based on the OpenSimulator software for 3D worlds.

iOS apps CANNOT RUN IN THE BACKGROUND !

There is no 3D view, this is a text-only viewer.

Add accounts from different worlds with different start locations.,
See your friendlist, read messages or write
Chat with groups, see group infos or write group notices
Chat in open world 
Move around a mini-map, teleport to landmarks
See your inventory/worn, preview textures or pictures, send items
Attach/Wear/Detach/TakeOff Clothes
Enter/leave groups
Add friends/invite/pay/remove

So about the same feature list as the late-lamented Pocket Metaverse, but updated more often (check the developer blog here). But caveat emptor, my dude: MetaChat's user reviewers are pretty middling. However, even most of the critical reviews still suggest it's worth considering as a purchase:

Open Source VR Viewer for OpenSim in Development -- Low Cost Alternative for Social VR Projects?

Interesting project now in development within the community of OpenSim, the open source/reverse engineered version of Second Life:

OpenSimulator core developer Melanie Thielker — also known as Melanie Milland in-world — announced that she is releasing her virtual reality OpenSim viewer to the open source community, available here via GitHub. The new viewer uses the Unreal Engine to display OpenSim regions, such as areas from the grid Thielker founded, Avination.

While OpenSim never took off as a consumer-facing project, it counts thousands of developers and dedicated members, many within the education community, and counts thousands of regions/3D experiences. So this could potentially be a good, low budget/low frills alternative for indie social VR projects. Though the technical effort to make this possible seem fairly daunting -- porting OpenSim content into the Unreal engine:

What OpenSim Can Teach Us About Building Successful Virtual Worlds

On HyperGrid Business, my colleague Adam Frisby shares some fascinating thoughts on co-founding OpenSim, the open source spinoff of Second Life, and what they can teach about building new virtual worlds that can scale to a broader audience. Couple big surprises to me: OpenSim "was downloaded hundreds of thousands of times" during its first few years, and had incredible buy-in from Silicon Valley:

For a while, there were some big names adopting the project in droves. Nearly every major tech company had some involvement — or at least one employee contributing — to OpenSim at some point. IBM had an entire team of OpenSim developers and was running internal conferences using the project.

OpenSim, sad to say, couldn't sustain that momentum, and Adam has many insights for why. Here's one worth discussing as a group -- "IP protection needs to be baked into the virtual world from day one":