Watch: Piloting a Drone With a VR Rig While Still Seeing the Drone in AR

This latest brain-expanding mixed reality hack from Toronto developer Lee Vermeulen is a bit difficult to process visually, but it's basically this: He is flying a Tello drone with a Unity-based controller that is hooked up to his Vive VR headset and Knuckles hand controllers. So the inset video you see is the stream from the drone's camera.

However, he's not navigating the drone completely through the VR headset, because he has its AR passthrough function enabled. That way, he can still see the drone in real life while also seeing through the drone's video camera:

"Usually controlling drones in VR is like that -- your vision is completely taken over by the drone cameras," he explains on Twitter. "I'd prefer something like this in AR -- where you can still see the drone from where you are."

Makes sense to me: If you were just piloting the drone through its camera feed, it would be difficult to navigate in dark/ visually complicated areas.  Whereas you can better orient its position and where it needs to go by first spotting it IRL.

Many more mind melting Vermeulen projects featured on NWN:

• Watch: Dazzling Demo Of Fluid Physics In Augmented Reality
• Watch: Modbox Sandbox Game Includes Mind-Bending "Mirror World" Mixed Reality Feature
• Watch: Amazing Mixed Reality Project -- Move 3D Objects Within Real Locations In VR... Which Then Seem To Appear In The Real World

Comments

[Cindy Bolero]

So Unity lifted the restriction on using the software to control machinery?

Or is it a special license, like the gambling restriction?