New World Notes

Apple Hiring Junior Engineer to Help Develop "a 3D Mixed-Reality World"

Intriguing new job opening from Apple for a AR/VR Network Engineer stating "In this role you will work closely with other developers and build tools and frameworks to enable connected experiences in a 3D mixed-reality world."

I found the job listing via this Bloomberg story which interprets it in the broadest way possible:

The most interesting job listing is one that specifically calls out the development of a 3D mixed-reality world, suggesting that Apple is working on a virtual environment that is similar to the metaverse—though don’t expect Apple to embrace that term. Its marketing chief said at a recent event that metaverse is “a word I’ll never use.”

It's definitely interesting, but as with previous rumors of Apple about to go big on a mixed reality Metaverse and XR headsets, I'm still curbing my enthusiasm:

How to Play With Second Life's Puppetry Project -- If You're Hardcore!

Linden Lab only recently announced its Puppetry Project, enabling real time control of Second Life avatars through webcams and other movement tracking devices. But unsurprisingly, while the project is very much in Alpha, hardcore SLers are already unpacking and setting it up. That includes Judas Shuffle (love the avatar name, that's him above), who recently posted some very detailed instructions for doing so yourself. Word of warning before clicking, this is for an unreleased project and requires some tech savvy (or extreme patience) -- so click here and read with caution.

"My favorite experience in virtual worlds so far was smiling at someone and having them smile back," Judas explains. "If it all works as well as it should it will bring people closer together." 

Hilariously, he got it going and posted these instructions only days after it was announced, which suggests some strong desire to see it in Second Life. ("A user in the beta grid got it working with my instructions.") To be sure, the final version should be relatively plug-and-play, but, "The [current] release as-is is about letting coders see what they can do with it. The puppetry build I would think means if you have the know how you could connect it to anything."

Yes, anything, not just webcams. In fact, Judas sent me this video of SLer Chaser Zaks running the Puppetry viewer through Blender:

Watch: Shopify Experimenting With Mixed Reality, AI-Powered Fashion

AI is enabling some really magical experiences. 🪄

Originally planned to share this tomorrow, @sergeyglkn scooped us! 😜

Keep on eye on this space. It moves fast! 💨 pic.twitter.com/Of8PgJWZMM

— Russ Maschmeyer (@StrangeNative) September 19, 2022

This is a fun and impressive demo: Online e-commerce site Shopify prototyping AI image generation used in fashion. (Russ Maschmeyer is AR/VR head at the tech company.) It was created in collaboration with Sergey Galkin, who explains how it was made on his own Twitter thread: "The prototype was made in Snapchat Lens Studio + post-processed in Adobe After Effects. Images were generated by Dalle 2." 

They're demonstrating this for a real world clothing context, but the virtual world applications are even more obvious. (I mean how many wacky T-shirts does the average consumer really buy, versus new avatar outfits?) Being able to adjust the style, fit, and composition of the clothing item itself seems like the next obvious next step.

XR Headset Controlling Giant Humanoid Robot to Construct a Railway is Peak Japan Moment of 2022

This is the mixed reality application of the month and is probably the Peak Japan moment of the year: A giant construction robot.. controlled through an XR headset. According to this Japanese news site (and Google's so-so translation), it is very much in production:

[Railway company] JR West announced on the 15th that it is developing a multifunctional railway heavy equipment that combines a humanoid heavy equipment robot and a railway construction vehicle in collaboration with the human-machine integrated and Nippon Signal. The company aims to improve productivity and safety by developing highly versatile heavy railway equipment that can be used for various equipment installed in high places and mechanizing work that previously required human intervention.

According to VR expert Adeon (who tipped me on this ), the HMD we see pictured here is an HP Reverb G2. ("Very good headset, better than Index.") New World Notes' Japanese translator Sanny found this video of the robot in action, watch below:

Code to Connect Real Life Devices to VRChat Published to GitHub - With a Small Catch

Remember "Dream Drive" (aka "Micchy" in VRChat), the Japanese inventor who is jaw-droppingly creating a robot in the real world that is controllable from within VRChat? Of course you do! We don't fully have a Metaverse along the strictest definition until there's a way to bridge the virtual world with real world devices. So here's some good news:

He just published to his Github the code enabling serial communication between VRChat and the real world. Now other inventors can take his code and start building other mixed reality projects.

Slight drawback: Micchy's Github code is only in Japanese for now, so the next step may be getting a tech-fluent translator to bridge Japanese with English and other languages. If you're interested, connect with him on Twitter directly, or post in Comments below, and I'll nudge him to reply. (Also see his Japanese language tutorial below the break.)

Watch: Inventor Creating 8 Foot Real Robot Controllable from Within VRChat!

I've seen some insanely ambitious VRChat projects but good god just watch this video already. It's from "Dream Drive" , a Japanese inventor who is creating an 8 foot tall Transformers-type robot in the real world... that is controllable from within VRChat.

That's possible, he tells me, by virtual serial communication between VRChat and Raspberry Pi. (His VRChat username is "micchy".) And while he's creating the software connecting the virtual world to the real one in order to control his robot, he plans to make the code generally available for people to create other mixed reality applications:

FOX 's Alter Ego: A Mixed Reality Competition Show from Producers Who Don't Understand Avatars or Why Virtual Worlds Make Them Real

Above: Pop star Grimes, who believes reality is a virtual world created by an omniscient AI, explains how Alter Ego works

Months after its producers began scouting among the Second Life community for avatar-based performers, the avatar-based reality competition show Alter Ego is here: Watch on Fox TV's site. 

I just saw the first episode and I have mixed feelings about its mixed reality presentation. Fundamentally, the whole concept seems to be directed at an audience who don't quite understand how virtual world avatars work, created by producers who definitely don't know how they work. Which is strange, because the vast majority of the show's target Gen Y/Z audience (I'm tempted to say all of them) have an avatar in Fortnite and other virtual worlds.

Speaking of which, as I suspected, the show uses Unreal, the graphics engine that also runs Fortnite:

Watch: The Teaser for Alter Ego is Here, and I Have Questions About FOX's Upcoming Avatars-Meet-Masked Singer Show

Back in May, a FOX producer was reaching out to members of the Second Life community (and presumably other virtual world communities) about Alter Ego, an upcoming avatars-meet-Masked Singer reality competition show. Now set to premiere on September 22nd, here's the teaser.

... and here's my questions about it:

TV Producers Searching for Virtual World Performers for Masked Singer-Meets-Avatars Reality Competition

Anytime I watch clips from FOX's The Masked Singer, my first thought is, "Why are all these celebrities costumed up to look like furries from Second Life and VRChat?" So I was utterly unsurprised to read about Alter Ego, an upcoming reality competition show from the same TV network:

ALTER EGO is an all-new original singing competition where lost dreams and second chances are reignited when singers from all walks of life become the stars they’ve always wanted to be. However, these contestants won’t perform as themselves. Rather, they’ll be given the chance to show how they’ve always wanted to be seen, creating their dream avatar ALTER EGO to reinvent themselves and perform like never before.

So yes, pretty much Masked Singer meets singing avatars. The producers are directly reaching out to U.S.-based performers who sing in Second Life, and I imagine singers from other platforms are also welcome.

Go here to apply, and send to performers you know who should. To judge by the kind of questions the application asks -- "What personal struggles have you had to overcome?", "Has music/singing helped you overcome a challenge?" -- the producers are just as interested in the real life story of each applicant, as their actual voice.

Next question is, which great singers who perform in a virtual world should apply to be on this show?

Watch: Epic BattleBots Duel, Fire-Breathing Slapbox vs. SL Fave Sharko!

In case you missed it last week, here's the Discovery Channel's BattleBots fight between SlapBox and Sharko, prototyped in Second Life by Artorius Constantine, owner of SL's Nano Station Technologies store.  Sent to us via BattleBots super-fan Devin Vaughn, it is almost more brutal and merciless as a major fashion drop in SL, which is saying something.

"Overall that was one of the best episodes of BattleBots I have seen in a long time," Devin reports. "Good even matches, much spark and explosions! Sharko had a good match against a rookie bot SlapBox, and a crazy moment: Shark bites down good, but then SlapBox lifted the entire shark, all 250 pounds was lifted into the air, totally inverted and shot flames down the mouth of poor Sharko who refused to let go. Wow."

Watch above to see what happens next! As to why it happened, Constantine and his crew have a good post-mortem in Reddit's /BattleBots community: