New World Notes

Apple's AR Plan Per The Information: Headset in 2022, Glasses in 2023

Insider tech industry news site The Information has a great scoop on Apple's plans for augmented reality (subscription required), and given The Information's consistently reliable track record, I'm inclined to believe it:

Apple is aiming to release an augmented-reality headset in 2022 and a sleeker pair of AR glasses by 2023, The Information has learned... Apple’s headset, code-named N301, will offer a hybrid of AR and VR capabilities, according to people familiar with the device. It resembles the Oculus Quest, a Facebook virtual reality headset released earlier this year, but with a sleeker design, these people said. Cameras will be mounted on the outside of the device, allowing people to see and interact with their physical surroundings, they said. Apple wants to make heavy use of fabrics and lightweight materials to ensure the device is comfortable to wear for extended periods of time, executives said in the presentation in October.

... In contrast, Apple’s AR glasses, code-named N421, present bigger technical challenges than the headset and are further from release. They are meant to be worn all day, and current prototypes look like high-priced sunglasses with thick frames that house the battery and chips, according to a person who has seen them.

This report contradicts earlier rumors that Apple's AR glasses were coming next year and would be third party (which seemed believable to me at the time). Then again, Apple pushing an AR/VR headset back to 2022 is a good idea, especially if Oculus Quest sales continue to disappoint in the short term; that gives Apple time to reschedule the release of their own headset, as needed.

Another tidbit from The Information addresses AR's "Glasshole" problem -- i.e., how do you project information onto glasses in a social situation without the user appearing to be a disconnected jerk ignoring everyone around them? Apple's apparent solution:

Jeri Ellsworth's Tilt 5 Kickstarter Sells Over 6000 Augmented Reality HMD Units, Raising Nearly 4X Target Goal

With 8 hours left to go, Jeri Ellsworth's Tilt 5 Kickstarter for a new augmented reality system is already a runaway success, raising at least $1,660,865 -- nearly 4X its original $450,000 goal target goal. Even more impressive, she just announced the Kickstarter has sold over 6,000 HMD units.

To put that 6,000+ in perspective, that's nearly as many units sold in Oculus Rift's original Kickstarter (about 6,900), which launched this new generation of VR. And by some industry insider estimates, the extremely well-funded Magic Leap has only sold about 6000-7000 of its own first edition HMD. To be sure, Magic Leap's "Creator Edition" sells for $2,295 and up, as opposed to Tilt 5's $299 entry level price. However, if that estimate is fairly accurate, that still means Tilt 5's install base is competitive with Magic Leap's.

For Jeri, in any case, it's a validation of Tilt 5's vision for augmented reality.

"We're thrilled that the market is hungry for a social AR experiences," as she puts it to me.

To judge the Kickstarter, the social aspect seems important to Tilt 5's success:

Watch: Capture Real World Locations With Your Smartphone & Turn Them Into LEGO-Like Virtual Worlds With New SDK

Some cool videos from our creative teams building with https://t.co/Ob4co4nbsh. Who doesn't love turning a normal sidewalk into a lego land? #arcloud #mirrorworld #everythingisawesome pic.twitter.com/4Yhx6mtbt1

— 6D.ai (@6d_ai) October 14, 2019

Remember 6D.AI, the startup that's creating a 3D mirror world of real life from the input of standard smartphones? The company recently released its SDK, and it makes what you see above possible: Scan a real world location into a 1:1 scale 3D model... then turn into a LEGO-like virtual world.

"It’s all built on our SDK," 6D CEO Matt Miesnieks tells me. "Instead of mesh triangles we draw LEGO blocks." These blocks are the only texture available in the SDK, he adds, but users can create and add their own textures -- opening up incredible opportunities.

Why Apple's Rumored Partnership Third Party AR Glasses Makes Sense

Rumors of Apple's jump into augmented reality glasses keep cropping up, and here's the latest (relatively credible) one:

Apple's augmented reality headset will enter mass production as soon as the fourth quarter of this year in time for an early 2020 launch, according to a new report out today from Apple analyst Ming-Chi Kuo and seen by MacRumors. In his latest research note with TF International Securities, the analyst says Apple will cooperate with third-party brands to launch its first head-mounted AR product in 2020, with Changying Precision tipped to be the main chassis supplier.

I tend to believe this rumor more than most, because it answers the most obvious question: Why would Apple spend billions to manufacture a new hardware device with no proven mass market, but a track history of notorious failure (cough Google cough Glass)?

Answer:

AR/VR Pioneer Co-Creating Piano Playing Instructional App for Magic Leap

HDPiano for augmented reality on the Magic Leap is one of the more noteworthy AR applications I've come across so far. For one thing, it's immediately understandable ("Oh, learn how to play piano in AR!"), and potentially cost effective. (What's the cost of IRL piano lessons, $50-75/hour?) For another, much like Jeri Ellsworth's approach to AR, it doesn't assume there's a market that's inherently interested in wearing augmented reality glasses, but goes after an existing market.

The project is the brainchild of Dan Collins, whose YouTube tutorials for playing piano has earned over 1 million subscribers. (Like I said, there's that existing market.)

"[When w]e think about the future of learning we see AR as a mighty force," Collins tells me. "This in mind, we sought out to 'see what's possible' today and plan for a future where every consumer has an AR wearable that might become a part of their time at the piano."

Fortunately for him, Collins (who's based in Wisconsin) Googled around for a local AR developer to help create this app, and found the perfect person nearby -- pioneering AR/VR developer Jon Brouchoud of Arch Virtual, who's been helping create music applications in virtual world/VR technology since at least back during his Second Life heyday: 

Jeri Ellsworth's Tilt 5 Kickstarter Sells 1500+ AR Units in Under a Week

Jeri demo-ing Tilt 5 at GenCon

Less than a week after launching a Kickstarter for Tilt 5, a totally new approach to augmented reality, it's attracted over 1700 backers and pre-orders of over 1500 Tilt 5 AR kits, totally blowing past the original $450,000 pledge goal, and fast approaching $1 million total. At this rate, interest in Tilt 5 could reach or surpass the $2.4 million raised for the original Oculus Rift Kickstarter which launched the current generation of VR.

CEO and co-founder Jeri Ellsworth attributes this success to her company's precise targeting of a specific market (board gamers/gamers who also play board games) which she discussed with me last week:

Watch: Mirror World Capture Technology That's So Accurate, You Can Create a Virtual Space That's Accurate to Its Real Life Location to Within One Inch

A @6d_ai experiment, shows 6D visual scans are *also* slam maps where coords = real-world coords. Phone is tracked & rendered in the scan! Content placed in a scan aligns with the real-world. Critical for authoring of AR content when you can’t be at the physical site & mixed ARVR pic.twitter.com/5npt7MBQyW

— Matt Miesnieks (@mattmiesnieks) September 24, 2019

Remember this demo from from 6D.AI, a startup that's creating a 3D mirror world of real life from the input of standard smartphones? Here's the latest demo from CEO Matt Miesnieks, showing that the virtual world (on the laptop) has a near-exact overlay with its real world counterpart, and is updated/mirrored in real time. So much so, the demo suggestions, that you can walk through the real life location simply by navigating your position in the virtual version.

"It’s accurate to about an inch, plus or minus," Miesnieks tells me, which he attributes to "[an] error in ARKit tracking." (I.E. from the attached iPhone he's using.) 

Does this mean that this technology could turn a whole real life environment into a one-to-one 3D game/virtual world space?

Jeri Ellsworth's Vision for Her Tilt 5 Gaming System -- and For Growing the Augmented Reality Market

Update, September 30: A week after launching its Kickstarter, Tilt Five has already raised nearly double its fundraising target.

Tilt Five: Holographic Tabletop Gaming is a new Kickstarter that is, on the face of it, an augmented reality-based gaming system. But as Tilt Five co-founder, CEO, and renowned hacker Jeri Ellsworth suggested to me last week, it’s also a new vision for augmented reality that runs totally counter to the “boil the ocean” approach (as she puts it) currently being taken by Microsoft, Magic Leap, and other major AR/VR players. (Including, presumably, Apple.)

Rather than create an AR headset for a wide variety of use cases, Tilt Five’s vision is first to start with a specific target market for augmented reality: 

“Our focus to take what you love about video games and board games and marry them together,” as she put it to me. (Tabletop gaming is a $7 billion market, she notes, while videogames are 100x that number.) So Tilt Five’s AR experience is tightly integrated with its game board, and integrates well with many existing tabletop gaming systems -- including Fantasy Grounds, which holds the license to miniatures for Dungeons & Dragons and other major franchises.

Her company's HMD has a wider field of view than any other AR glasses on the market (110 degrees, versus, say, Magic Leaps’s maximum 50 degrees), and unlike other AR headsets, can do full object occlusion, in which virtual objects seem to block out real objects behind them. She says Tilt Five can also maintain focus at variable distances, i.e. you can lean back or move forward, and the virtual objects will remain clear -- another thing other AR headsets can't accomplish.

Tilt Five is based on technology that Jeri first developed when she was at Valve Software*. (She negotiated to keep the IP rights amidst a layoff.) It was also at Valve where she had a view of the future in which AR/VR would dominate. But it was then, she tells me, that she also saw how long it would take for the industry and mainstream consumers to catch up to that future.

So Tilt Five is her vision for augmented reality that’s immediately appealing to an existing market right now: a social gaming system that can be played with friends around the same board, or with two or more boards around the world, connected online. But that’s just the start:

Why Apple Owns the AR Industry Even Without Owning AR Glasses (Comment of the Week)

"Joey1058" has an excellent rant responding to my speculation on just when (probably not any time soon) Apple will announce the launch of its much-awaited augmented reality glasses. To which he replies:  

HMDs do exist in the wild. A company called Vuzix has been tackling hardware issues for decades. Their flagship product is called the Vuzix Blade. [Pictured here] In my humble opinion, their glasses are every bit as good as Magic Leap, Hololens, or the current darling that is sparkling everyone's eyes, Nreal. The hardware isn't an issue. My issue is Apple.

For some ungodly reason, the industry has taken a "wait-and-see" approach in development that centers around Apple. We all know that Apple has been working on developing AR for some number of years. At first, buying up companies and patents. Then releasing ARKit. And now the buzz is all about its U1 chip and Ultra Wideband tech. But somehow, they've been dragging their butts in releasing an HMD. So in the meantime, smaller companies are developing on what already exists.

That's not to ignore the big tech company other than Google (LOL Glass) which jumped into AR early:

Apple Apparently Testing AR Glasses, But Don't Expect Them Anytime Soon

Despite some insider rumors that Apple would announce its highly anticipated augmented reality glasses this week, nothing is so far forthcoming. Yesterday,  however, a developer more or less confirmed that those AR glasses do indeed exist, and that they're being tested -- because there's mention of a device that can be "worn" in the notes of its next operating system:

It comes to us in the form of this screenshot from iOS developer and reliable leak-finder Steve Troughton-Smith...

The screenshot pretty much says it all: Apple is testing an internal device known as “Garta,” and if you want to test your stereo AR apps with that device, you’ll want to set them to “worn” mode so that it can apply different “distortions” than if the app is being “held” instead.

Which implies that Garta is a stereoscopic AR headset — because that’s the kind of device you would wear, and which would adjust the way an augmented reality app would look to match lenses in front of your eyes. (AR and VR headsets typically adjust their computer graphics “projections” to properly stretch across your field of view while using as little processing power as possible.)

Click to embiggenate Troughton-Smith's screengrab of the OS notes. I'm far from an Apple Kremlinologist, but barring a miracle (more on that later) I wouldn't expect any mass market AR glasses announcement from Apple in the near future -- certainly not next year. A few reasons why: