New World Notes

Philip Rosedale on Clubhouse Bringing High Fidelity's Metaverse-Driven Spatial Audio to the Massive Audio Chat App

Hear ye, hear ye 🔊 spAAaAaAatial audio on Clubhouse!

It's like surround sound, but w/ your own headphones. A more vibrant, human experience! Plus makes it much easier to tell who's talking.

thanks to @juberti for this one 👏 rolling out now on iOS, Android coming soon! pic.twitter.com/Zit6F9ijRK

— Clubhouse (@Clubhouse) August 29, 2021

Here's some Metaverse news that requires your headphones for the full effect (click the video above):

Clubhouse, the seriously popular audio chat app  -- it counts at least 10 million weekly active users -- just integrated Philip Rosedale's High Fidelity spatial audio:

High Fidelity’s HRTF technology, which stands for “Head Related Transfer Function,” maps speech to different virtual locations by subtly adding a time delay between stereo channels and replicating the way that high and low frequencies would sound entering the ear depending on a sound’s origin. The result, long used in social VR, gives virtual social experiences a sense of physical presence that good records have been pulling off for ages. Think listening to Pink Floyd’s Dark Side of the Moon in stereo with good headphones but instead of sound effects and instruments playing around your head, you’re hearing the people you’re hanging out with arrayed in virtual space.

"If people are going to talk in groups online, they are going to need to use spatial audio to understand each other," Philip Rosedale tells me, shortly after the announcement dropped. "We've put in 8 years (so far) working on doing spatial audio right, and are super happy to have people in the Clubhouse community now able to take advantage of this experience!"

This is the latest, fascinating plot twist in Philip's Metaverse ambitions, originally borne with the founding of Second Life, then re-emerging in his latest start-up, High Fidelity, which was originally founded as a Metaverse company. (Before, that is, pivoting in 2019 to focus on this spatial audio feature).

But where Second Life and the original High Fidelity struggled for mass adoption, it's finally been achieved, if only through spatial audio:

Watch: I Interview Philip Rosedale & Adam Frisby About the Metaverse in Breakroom

Update, July 2: Bumped up for viewing during the long holiday! Discuss in Comments below, and I'll nudge Adam and Philip to participate if they have time.

Appearing in his original Philip Linden avatar with a notably sparkly upgrade, here's a lightly edited cut of my fireside chat with Philip Rosedale and Adam Frisby on "The Metaverse as a Shared Imagination", hosted in Breakroom from Sine Wave Entertainment (a media partner to this blog). Yes, Sine Wave CPO Adam is the rabbit.

I'm highly biased but I think this was one of the most interesting metaverse conversations in quite awhile, covering the alchemy of user creativity, VR adoption, NFTs, and lessons that Second Life can teach the current generation of metaverse platforms.

Here's an annotated breakdown of questions:

Tomorrow: Join Philip Rosedale, Adam Frisby & Me for a Fireside Chat About the Metaverse & Imagination Inside the Metaverse of Sinespace's Breakroom

Click here to RSVP for a literal fireside chat with Philip Rosedale and Adam Frisby tomorrow at Noon PT inside Breakroom from Sinespace (a proud sponsoring partner to this blog), which I am thrilled to moderate. (And yes, that's Philip's classic SL avatar, updated for today with a glittering codpiece.) Event info:

Since its inception in the early 90s, the Metaverse has inspired our wildest imagination -- even if the technology to fully realize that dream has seemed just out of our grasp. But now, with advances in edge computing, VR, and 3D graphics/audio, nothing seems impossible. Given such powerful tools at our fingertips - what could this look like?

The fireside will also showcase Philip's High Fidelity spatial audio, which was just integrated into Breakroom, while the world itself will be streamed onto Chrome browsers (no install needed). If you can't make it, no worries: The Breakroom team is recording the event and will be broadcasting in coming days.

Breakroom Brings Philip Rosedale's High Fidelity 3D Spatial Audio to its Highly Immersive Virtual Worlds Streamed on the Browser

As VentureBeat just reported, NWN media partner Sine Wave has teamed up with Philip Rosedale to bring High Fidelity's 3D spatial audio to Breakroom, its browser-based platform for virtual world events and remote teams, and it's a metaverse reunion of sorts:

The deal is a convergence of pioneers who have made their mark on the development of virtual life. Philip Rosedale is the CEO of High Fidelity and launched Second Life in 2003. Sine Wave Entertainment, the creator of Breakroom, got its start as a content brand in Second Life before it spun out to create its own virtual meeting spaces for real-world events.

Adam Frisby, chief product officer and cofounder of Sine Wave, said in an interview conducted inside Breakroom that the High Fidelity spatial audio will help Breakroom create a triple-A quality experience in a virtual world.

Sine Wave’s Breakroom draws heavily on lessons learned in Second Life to create a frictionless, mass market, user-friendly virtual world.

“You can hear everything better with High Fidelity,” Rosedale said in our interview in Breakroom. “Breakroom combines low-latency server-side video and spatial audio in a way that lets you hold an event like it’s in the real world.”

Listen above for a demo, more here. I'm highly biased of course, but I believe this represents one major next step to bringing metaverse technology to the mass market.

Speaking of which, I'm hosting a literal fireside event in Breakroom with Philip and Adam this Thursday at Noon PM, and there are a few seats available: Click here to reserve a spot in "The Metaverse as a Shared Imagination" chat.

Watch, Listen: Minecraft Integrated With High Fidelity 3D Spatial Audio

Earphones on to hear a whole new way of experiencing Minecraft -- integrated with High Fidelity's 3D audio API. This not only makes it possible to talk with players around you on the same Minecraft server, but also experience the world itself in immersive audio, so when Creepers are groaning behind you, it will sound as if they are actually behind you, and when you're walking along a river, it'll sound like the river is literally right beside your computer, and so on: 

Watch: Skittish Adds Philip Rosedale's High Fidelity Spatial Audio & Other Virtual World-ish Features to Upcoming Playful Space for Online Events

Skittish, Andy Baio's upcoming "playful virtual space for online events" (go here to register for early access) recently added High Fidelity's 3D spatial audio to the platform, so naturally, I had to get a hands-on (ears-on?) demo from Andy -- watch above, but maybe turn the sound down a bit due to my overly booming voice.

So far Skittish is an incredibly charming virtual world-type space for live meetings of all kinds, especially if they lean toward the whimsical -- think Animal Crossing meets Zoom, except your cartoon animal avatar is the one on camera, not you.

Integrating High Fidelity, Andy tells me, "dramatically increases the cross-browser compatibility, audio quality/latency, and scale for concurrent users." This also makes Skittish the first virtual world-type experience using the API to the Philip Rosedale-led spatial audio technology. (At least the first one that I'm aware of.)

Other new features you'll see in my demo with Andy:

Tomorrow, Noon-2pm PT: Talk 3D Audio With Philip Rosedale, Brianna Wu, Jane McGonigal & Other Game Industry Luminaries in Both Club House & High Fidelity

Two related events to set your schedule for tomorrow via High Fidelity's e-mail list:

 [Discussing 3D audio with] High Fidelity's CEO Philip Rosedale, Brianna Wu, Jane McGonigal, Dean Takahashi, Nicole Lazzaro, James Powell, and others in the gaming industry.. We'll be live in Clubhouse from 12-1p Pacific, and then head over to High Fidelity's Spatial Audio demo space [pictured above] from 1-2p Pacific. Come and share your POV, bring questions, or just drop-in to listen.

High Fidelity's Ashleigh Harris tells me their 3D space has a maximum capacity of 90 people, so try and get there early. I'm hoping to attend as well! 

Street Meet: 3D Chat With People in Real Locations Around the World Via App Combining High Fidelity Spatial Audio & Google Street View

Pictured: Chatting in New York's Guggenheim art gallery via Google Street View & High Fidelity

Want to instantly teleport anywhere in the world to chat with friends or friendly strangers? Of course you would. And now you can:

[High Fidelity has] released a whole host of sample applications that we built during a week-long internal team hackathon, and all of them are available for you to use in your own work.

Street Meet . This experimental sample app uses Google's Street View so you can have a meeting almost anywhere in the world!

As that suggests, this is one of many demo apps using High Fidelity's 3D spatial audio. Personally I think Street Meet has the most potential for immediate and multiple applications. Just off the top of my head: teaching geography to remote students, giving real estate customers a "walking" tour of the neighborhood they're interested in, or simply having a drink with remote friends while enjoying various views of the world.

Speaking of High Fidelity, CEO and founder Philip Rosedale is talking social audio on Clubhouse right now (if you have an account):

High Fidelity Launches API for Its Spatial 3D Audio -- But Where's the World?

High Fidelity just launched the Alpha program for its API to deliver 3D spatial real time audio to many people in the same virtual space, with an SDK for Unity and iOS coming soon. (Speaking of which, the company's hiring an iOS developer.) The roadmap is pretty exciting:

High Fidelity Provides Spatial Audio for Live CES Show Featuring Billie Eilish

I mentioned just yesterday that Philip Rosedale and his team at High Fidelity were working on improving the live music experience using their 3D spatial audio-based virtual world, but at the time, I didn't know they'd be going so big.

As in: How about creating the audience experience for a live concert at this year's (now-virtual) Consumer Electronic Show, the biggest convention of its kind, featuring a live performance by Billie Eilish, among the most acclaimed pop artists of her generation? Yes they did, and incorporated a new 3D-ish visual overlay to enhance the social aspect:

"A partner of High Fidelity called SpatialWeb built the experience on top of High Fidelity," Philip tells me, "adding the 3D, social features, and an ability to do P2P video as well.  HF was used for the spatial audio between participants."

I could be wrong, but this might be the breakthrough High Fidelity needs to go massive; after all, Second Life's key breakthrough back in 2006, arguably, happened with Suzanne Vega's live in-world performance. In any case, expect more projects like this to come: