New World Notes

Apple Is Hiring Dozens of AR/VR Developers (Which Actually Isn't Much)

Apple has posted over 40 job AR/VR openings since last August, which more than any recent rumor swirling about, convinces me that Apple is moving into the AR/VR space.

But! 40 or so employees is actually not that much amid a company with nearly 150,000 total staff. (And translates to Apple paying these new hires roughly $10 million a year, a pittance for a firm of such size to pay.) More key, competitor Oculus has 1,300 employees listed on LinkedIn, while a significant portion of Sony PlayStation's 8000 employees are dedicated on some level to developing for the VR market leader, PSVR.

My take: Apple is still in very early, prototype stages of any VR/AR headset launch it has planned, and don't expect an Apple HMD to hit the market a year or two before you see them start hiring AR/VR developers by the hundreds and hundreds.

Here's Jeri Ellsworth's Vision for AR That Helped Tilt 5 Raise $7.5M

In case you missed it last week, AR startup Tilt 5 raised $7.5 million in venture funding, a hopeful sign amid a rather bearish market for AR/VR startups. I've been following the underlying technology for Tilt 5's headset since it was spun out of Valve under the name Cast AR seven years ago. Founded by CEO Jeri Ellsworth, one reason her startup won backers is likely her focused approach to augmented reality -- not attempting to make it the Next Great Platform For Everything, but starting first with a very specific use case. As she put it to me last year:

New iPad App Uses Machine Learning to Capture Real World Objects & Convert Them Into Standard 3D Files

Caveat emptor and all that, but the iPad LIDAR Scanner just published in Beta seems seriously impressive, able to "capture" real world objects and convert them into imperfect but pretty usable 3D models. It works, says developer Laan Labs, by automatically extracting objects via machine learning (i.e. extrapolating data for unseen surfaces).

More info from iOS app page: 

Founder Hubris Led to Magic Leap's Fall -- But Valley Hubris Enabled Its Rise

Bloomberg News has a great long feature on the rise and fall of Magic Leap, which raised $3.5 billion on the prospect of transformative AR technology, only to lose nearly all of that money in the fires of hubris. The company is still operating, but mainly now as an also-ran competitor to Microsoft's HoloLens/AR for enterprise use business. The hubris of Magic Leap's founders is not surprising -- it's pretty much par for the course for any startup that attracts that much funding so fast -- but the story doesn't cover another variation of hubris. 

Specifically: Why did so many major tech companies and VCs invest so much money on a product -- i.e., a head-mounted display -- with no successful track record whatsoever? Especially when one of the Internet's largest companies (Google) recently demonstrated the social disasters inherent in AR glasses -- i.e. "Glassholes"?

But invest they did, and took great pains to do so:

Watch: Half-Life Alyx Running in an Open Source AR Headset (Kinda Sorta)

Half-life Alyx running on #ProjectNorthstar,

Lots of improvements to do with the steamVR driver itself, but it's so cool seeing it progress this far. pic.twitter.com/BP9QtFxLSq

— Bryan Chris Brown 🚀 (@BryanChrisBrown) September 12, 2020

One moment you're in your apartment, and then the next, boom, your reality gets merged with City 17. This clever hack from Bryan Chris Brown, community manager at North Star, an open source AR HMD from Leap Motion.

"Lots of improvements to do with the SteamVR driver itself," he reports, "but it's so cool seeing it progress this far."

And yes, you can even control Alyx with hand gestures.

Watch: Light Show in Virtual Club Synced to Light Show in Real Life Club

This is a crazy cool project that links up the lighting system in a real life venue (San Francisco's venerable DNA Lounge, in this case) to the lighting system in a virtual world. As creator James Baicoianu explains:

Controlling lights in a WebXR night club using Chamsys MagicQ, a professional lighting control suite. This lets us configure a full digital twin of the real-life night club, where the light show running in the virtual world is driven by the same controller as [the real world club], and thus synced.

It's the brainchild of James Baicoianu, formerly lead engineer for the late-lamented WebXR metaverse startup called Janus VR, who tells me the system can probably work with non-WebXR worlds. As James explains to me:

Rosie's Rift Rips a Rift Into Reality

Rosie Summers recently wandered into a neighborhood park, and casually dropped this black hole into the grass.

"Soon as I saw its colossal size in an area of familiarity," she wryly notes, "the fear set in."

This rip of existence was made possible through several pieces of technology, and her own artistry:

Watch: Mesmerizing Procedural Art Sculpture by Michael Kozlowski

Mid-afternoon AR visual treat: Procedural artist Michael Kozlowski created this utterly hypnotic video sculpture in Unity with some kind of 3D projection display. I'm reaching out to hopefully know more.

Hat tip: William Gibson, who appropriately says just "Whoa" and AR expert Dr. Helen Papagiannis (last on NWN here), who properly pointed us to Kozlowski's Instagram to credit the creator.

Hit AR Game Company Niantic Partners With Hit Mixed Reality Sleep No More Play Producers

I have not seen many AR applications recently which struck me as exciting let alone likely to be financially successful, but a recent deal definitely caught my eye:

Pokémon Go creator Niantic Labs is partnering with Punchdrunk, the UK-based production company behind seminal immersive theater piece Sleep No More. The partnership will see both companies work toward “developing multiple projects that will reinvent storytelling for a 21st century audience and further expand the horizon of interactive entertainment.”

Sleep No More, launched in its current iteration in New York City back in 2011, is a landmark immersive theater production that lets audiences walk through a series of rooms and experience various storylines and other elements of the production at their own pace. Influenced in part by haunted houses and other nonlinear live productions, immersive theater sounds like a prime fit for augmented reality because it could allow for Niantic and companies like Punchdrunk to create fantastical virtual worlds that are then overlaid over real spaces.

I never got a chance to experience Sleep (trailer above) but the many colleagues who did consistently described it as a kind of virtual world layered on top the real one. So creatively, Punchdrunk makes a lot of sense as a partner. Interestingly, it's a partnership that might better financial sense for Niantic too:  

Watch: Awesome AR App Copies Real World Objects Then Pastes Them as 3D Files Directly Into Photoshop

4/10 - Cut & paste your surroundings to Photoshop

Code: https://t.co/cVddH3u3ik

Book: @HOLOmagazine
Garment: SS17 by @thekarentopacio
Type: Sainte Colombe by @MinetYoann @ProductionType
Technical Insights: ↓#ML #AR #AI #AIUX #Adobe #Photoshop pic.twitter.com/LkTBe0t0rF

— Cyril Diagne (@cyrildiagne) May 3, 2020

Cyril Diagne, an artist, designer and programmer currently in residency at Google Arts, created this utterly awesome augmented reality app which seamlessly captures objects in the real world... and then "pastes" them as 3D files directly into Photoshop. (Watch the video above, you'll immediately see what I mean.) I've seen similar AR apps before, but nothing quite as seamless or as magical.

Perhaps as amazing: Cyril also made the code publicly available on GitHub here.

"The code was given away because it's a research prototype," Cyril tells me. "I don't think it giving that code away necessarily take away commercial potential. There is much more that can be done with that approach."

His goal is to make AR reality capture as intuitive on a smartphone as it is in Magic Leap -- except without needing an expensive HMD.

"Also the phone has such a perfect form factor it's hard to beat," he adds. "[T]here is no UI, there's no need to move the thumb or care about how we're holding the phone, which happens to be quite pleasant and intuitive... Because we're able to delegate some of the cognitive tasks (like deciding which object is the one being pointed at in an image, selecting and removing the background, .etc) we can reduce the number of steps required to perform a task."

As for how he did it, he goes into great technical detail on Twitter, which I've excerpted for the technically minded below: