New World Notes

Excellent XR Video Effect Turns Real Life Face Into Avatar - Here's How

Hi! I’m Heather. I’ve been an interactive designer/illustrator/animator/developer (flash, web, app, & mobile games) for 15 yrs. Now, I’m using those game dev lessons in AR/VR: creating immersive art experiences that blur the line between art & audience. #WhatAGameDevLooksLike pic.twitter.com/NHyXiGvSdd

— Heather Dunaway Smith 🇺🇦 (@dunaway_smith) March 24, 2022

XR game developer Heather Dunaway Smith recently shared this seriously cool AR effect she created, where her real life face temporarily "pops" into her avatar persona. (See her video above to see what I mean.) It was created, she explains on Twitter, through a combination of Procreate, After Effects, and Spark AR. (The latter being a Facebook/Meta app, so caveat avatar.) "Yeah, you can export animations as PNG sequences (in After Effects), then bring them into SparkAR as textures, using an animation controller."

Heather even created a tutorial for creating this effect on her TikTok -- watch below:

Apple XR Headset Not Likely to Come Out in 2022 (Like I Said In 2021)

Just posted my 2022 predictions just last week, and this news already dropped this week:

Apple was reportedly planning to put the headset on sale sometime after the [June 2022] WWDC announcement.

Now, however, Apple is said to be exploring the possibility of delaying the announcement by at least a few months, or potentially pushing it back to 2023. If the iPhone maker goes forward with the plan, the device would reportedly start shipping to customers later than planned as well.

According to today’s report, Apple’s hesitance to debut the headset at WWDC in June is the result of ongoing development challenges related to the device’s software and cameras. Additionally, there are reportedly overheating issues caused by the onboard processor.

I don't doubt the hardware issues are an issue, but as I said, "I still fail to see why Apple would risk its brand on a still-niche product as yet." (If Meta announces much greater Quest 2 sales than expected, I would expect Apple's hardware challenges to get solved right quick.)

But even beyond sales, there's the other virtual shoe that hasn't dropped. If Apple has an XR headset... who is making the content for it? Because like I wrote:

Upland Now Attracting Over 100K High Active Unique Players for Its Blockchain/Tilia-Backed Virtual Real Estate MMO

Last month I estimated that Upland, the blockchain-backed mirror world real estate trading MMO, probably had about low-to-mid six figures of active users; now here's a pretty good data point of users who actually play Upland long enough to buy virtual real estate in it. A very active Upland user sent me the data above,  drawn from up2land.com, a player-driven analytics site.

"[T]hese numbers reflect only the UNIQUE players making transactions each month," my source tells me, "Either Sales, Accepted Offers, or Minting properties. It does not reflect the total number of players -- there are many [who] might go months without making a transaction - and some players may be counted each month. But it does give an indication of the trend of the game."

 

So unique players defined on those terms have ranged from 54,000 to 102,000 in recent months. My source suspects the dip in May is attributable to less advertising during that month. But these numbers likely do not include a lot of new users just playing around with the free-to-play game's starting money:

Niantic Throws Down Gauntlet Against the Virtual World Metaverse -- But Making Its "Real-World Metaverse" Will Be a Challenge

With all the hype around THE METAVERSE lately, this was bound to happen -- though I'm somewhat surprised it's coming from the CEO of AR-based virtual world creator Niantic:

A lot of people these days seem very interested in bringing this near-future vision of a virtual world to life, including some of the biggest names in technology and gaming. But in fact these novels served as warnings about a dystopian future of technology gone wrong. As a society, we can hope that the world doesn’t devolve into the kind of place that drives sci-fi heroes to escape into a virtual one — or we can work to make sure that doesn’t happen. At Niantic, we choose the latter. We believe we can use technology to lean into the ‘reality’ of augmented reality — encouraging everyone, ourselves included, to stand up, walk outside, and connect with people and the world around us. This is what we humans are born to do, the result of two millions years of human evolution, and as a result those are the things that make us the happiest. Technology should be used to make these core human experiences better — not to replace them.

Once again, based on what Neal Stephenson himself told me, Snow Crash's Metaverse is not exactly a dystopian warning. (We are starting to get into "Let me tell you what Fahrenheit 451 is really about, Ray Bradbury" territory with this novel!)

In any case, I'm glad to see this debate met -- I admire Niantic's technology and design team quite a bit, and would really love to see a mixed reality virtual world become a viable model. That said, the jury's out on whether Niantic itself is the one to pull what it has in mind off -- again from the post:

Apple Is Hiring Dozens of AR/VR Developers (Which Actually Isn't Much)

Apple has posted over 40 job AR/VR openings since last August, which more than any recent rumor swirling about, convinces me that Apple is moving into the AR/VR space.

But! 40 or so employees is actually not that much amid a company with nearly 150,000 total staff. (And translates to Apple paying these new hires roughly $10 million a year, a pittance for a firm of such size to pay.) More key, competitor Oculus has 1,300 employees listed on LinkedIn, while a significant portion of Sony PlayStation's 8000 employees are dedicated on some level to developing for the VR market leader, PSVR.

My take: Apple is still in very early, prototype stages of any VR/AR headset launch it has planned, and don't expect an Apple HMD to hit the market a year or two before you see them start hiring AR/VR developers by the hundreds and hundreds.

Here's Jeri Ellsworth's Vision for AR That Helped Tilt 5 Raise $7.5M

In case you missed it last week, AR startup Tilt 5 raised $7.5 million in venture funding, a hopeful sign amid a rather bearish market for AR/VR startups. I've been following the underlying technology for Tilt 5's headset since it was spun out of Valve under the name Cast AR seven years ago. Founded by CEO Jeri Ellsworth, one reason her startup won backers is likely her focused approach to augmented reality -- not attempting to make it the Next Great Platform For Everything, but starting first with a very specific use case. As she put it to me last year:

New iPad App Uses Machine Learning to Capture Real World Objects & Convert Them Into Standard 3D Files

Caveat emptor and all that, but the iPad LIDAR Scanner just published in Beta seems seriously impressive, able to "capture" real world objects and convert them into imperfect but pretty usable 3D models. It works, says developer Laan Labs, by automatically extracting objects via machine learning (i.e. extrapolating data for unseen surfaces).

More info from iOS app page: 

Founder Hubris Led to Magic Leap's Fall -- But Valley Hubris Enabled Its Rise

Bloomberg News has a great long feature on the rise and fall of Magic Leap, which raised $3.5 billion on the prospect of transformative AR technology, only to lose nearly all of that money in the fires of hubris. The company is still operating, but mainly now as an also-ran competitor to Microsoft's HoloLens/AR for enterprise use business. The hubris of Magic Leap's founders is not surprising -- it's pretty much par for the course for any startup that attracts that much funding so fast -- but the story doesn't cover another variation of hubris. 

Specifically: Why did so many major tech companies and VCs invest so much money on a product -- i.e., a head-mounted display -- with no successful track record whatsoever? Especially when one of the Internet's largest companies (Google) recently demonstrated the social disasters inherent in AR glasses -- i.e. "Glassholes"?

But invest they did, and took great pains to do so:

Watch: Half-Life Alyx Running in an Open Source AR Headset (Kinda Sorta)

Half-life Alyx running on #ProjectNorthstar,

Lots of improvements to do with the steamVR driver itself, but it's so cool seeing it progress this far. pic.twitter.com/BP9QtFxLSq

— Bryan Chris Brown 🚀 (@BryanChrisBrown) September 12, 2020

One moment you're in your apartment, and then the next, boom, your reality gets merged with City 17. This clever hack from Bryan Chris Brown, community manager at North Star, an open source AR HMD from Leap Motion.

"Lots of improvements to do with the SteamVR driver itself," he reports, "but it's so cool seeing it progress this far."

And yes, you can even control Alyx with hand gestures.

Watch: Light Show in Virtual Club Synced to Light Show in Real Life Club

This is a crazy cool project that links up the lighting system in a real life venue (San Francisco's venerable DNA Lounge, in this case) to the lighting system in a virtual world. As creator James Baicoianu explains:

Controlling lights in a WebXR night club using Chamsys MagicQ, a professional lighting control suite. This lets us configure a full digital twin of the real-life night club, where the light show running in the virtual world is driven by the same controller as [the real world club], and thus synced.

It's the brainchild of James Baicoianu, formerly lead engineer for the late-lamented WebXR metaverse startup called Janus VR, who tells me the system can probably work with non-WebXR worlds. As James explains to me: