New World Notes

Watch: AI Agent "Learns" to Crash Into Physics-Enabled Wall Like a Stuntman

Watch this new tech demo from DeepMotion, the AI company which taught NPCs how to argue, and learn to dribble basketballs: An AI "stuntman" agent who can run through a physics-enabled breakable wall -- and adjusts his momentum based on the wall's thickness.

"[W]hat's changing each iteration is the strength of the wall," DeepMotion's Anna Ploegh tells me, "by way of adjusting the mass of each brick. So the running movement is responding each time to a difference in the wall obstacle. Over each iteration, that mass of the bricks increases until it is too difficult for the character to jump through."

This is why each crash in the demo is different, as is the way the AI agent falls. This is in contrast to technology often used in videogames to simulate a similar effect, where an NPC is suddenly turned into a physics-enabled ragdoll (i.e ragdoll physics). Here's why:

Lupita Nyong'o in Us Re-Imagined by Afrofuturist Artist & Deep Dream AI

Love these two images by Afrofuturist artist Nettrice Gaskins, who uses her technique of creating Black artist portraiture through the collaboration of Google's Deep Dream neural network on stills featuring Lupita Nyong'o from Us, Jordan Peele's new acclaimed mega-hit.

"I use images with abstract textures and them composite the results -- sometimes [with] Photoshop and sometimes Microsoft Word (true story)," Dr. Gaskins tells me. "My work in Deep Dream is similar to my collage work using traditional artist materials. The process I use to draw and paint is what I'm using in Deep Dream." More specifically:

Watch: DeepMotion Creating AIs Which "Learn" How to Argue With Each Other

Here's a cool new AI demo from DeepMotion, the company which recently taught an AI how to "learn" the art of dribbling. In this demo, they've simulated a group of AIs infinitely arguing; even if they gesticulate so wildly, they knock each other down, they right get back into the squabble:

With only a few learned skills, the character controllers used in this demo are relatively simple, as are the mechanics of the simulation. When the simulation is played, each character transitions between the three argumentative gestures at random, creating the appearance of a lively debate. When the characters experience external force from a user, the environment, or another character, they fall and then promptly get up. The primary goal is to continue arguing.

Fit a few hundred of these in an auditorium, and you could simulate a session of Congress! (With possibly less pratfalls.)  

"The short term plan," Deep Motion's Chief Scientist Libin Liu tells me, "is to let the characters learn argument and idling actions from those unstructured motion sequences, so that they can perform them freely based on scripts, user commands, or mimic user actions based on sparse input (like 3 point tracking)." 

Most game-focused AI is developed around core mechanics like pathfinding and combat, but Liu and his team believe subtle group interactions like this can do much to make a game world seem that much more real:

Watch: Machine Learning Music Composed from Re-synthesized Fragments from 100s of Terabytes of LA Phil Recordings

Here's a new high def version of the dazzling 3D video/AI-driven performance displayed on the Walt Disney Concert Hall last year. Refik Anadol created the projected visuals, and the music was co-created by adaptive music/sound artist Robert Thomas, who as longtime New World Notes readers know, once experimented with voice-activated installations and location-reactive soundtracks in Second Life. (Disclosure: Also a pal.) Here's how Robert and his collaborators created the music for this event -- and by "collaborators", I'm including the machine learning algorithms they created to assist them, which searched through the LA Phil's massive archive of performances, merging disparate fragments of music together in new and novel ways: 

Top 12 AR, VR & Avatar-Based AI Projects & Innovations of 2018 Featured in New World Notes

Watch An AI-Controlled Avatar "Learn" How To Dribble In Real Time - Huge Implications For Games, AR/VR, And Real Life Robotics

• HoleLenz Turns HoloLens Into A Literal (Virtual) Portal Into Other Worlds
• We Heard You Like AR & VR, So We Put AR In Your VR
• Project H.E.A.R.T: A Virtual Reality War Game Where Soldier Morale Is Controlled By The Player’s Real Emotions
• VR Pioneer Helping Advise $10 Million ANA Avatar XPRIZE, The Contest To Merge Robotics With VR/AR-Driven Human Expertise

IOS' ARKit Enables Reflection Of Real Objects On Virtual Surfaces

Many more beyond the break!

Watch: MICA, Magic Leap's Eerily Human AI Avatar Who Looks You in the Eye

Word on the street is this is the most impressive tech unveiled at this week's Magic Leap conference. Say hello to MICA -- because she'll soon say "hello" back, while looking you in the eye:

Mica was sitting at the table in the same room. She smiled at me and look at me. I was struck that she wasn’t just looking at me. She was looking in my eyes. She tilted her head from side to side... When I noticed how attentive she was, I moved my head forward and looked in her eyes. She did the same and looked at me. I moved my head back and she moved her head back too. She was mimicking some of the movements that she saw me make. She didn’t talk, but that is coming in the future.

UploadVR's Ian Hamilton was staggered on an ontological level:

I need to process some thoughts. The Mica AI left me in a deeply introspective mode. I didn’t want to think of it as “she” but I couldn’t control my reaction. They Ex Machina’d me a little bit. https://t.co/gHZyzIAXKi

— Ian Hamilton (@hmltn) October 11, 2018

Full keynote below, beginning with inspiring words from company CMO Brenda Freeman and acclaimed designer Robin Hunicke, an apparent (and admirable) attempt to make a clean break from the startup's history with internal sexism, followed by a fireside featuring Neal Stephenson and John Gaeta (the Matrix effects guy), with Andy Serkis and others soon after:

Now in LA: Experience Awesome 3D Video & Machine Learning-Based Music Projected on the Walt Disney Concert Hall -- Adaptive Sound Artist Robert Thomas on How the Soundtrack Was Created

Video shot during last night's WDCH preview by Brooke Erdmann

If you're anywhere near downtown Los Angeles between tonight and October 6, you absolutely have to experience WDCH DREAMS. I was lucky enough to attend a preview with my wife last night (almost bumping into the Mayor, who was also there) and we were utterly dazzled by this flood of images and music echoing off Frank Gehry's famed concert hall:

The Los Angeles Philharmonic has commissioned award-winning media artist Refik Anadol to create unprecedented, breathtaking, three-dimensional projections onto the steel exterior of Walt Disney Concert Hall to signal the commencement of the LA Phil’s 100-year anniversary celebrations. Free and open to the public, nightly performances are scheduled to occur every half hour, with the first performance at 7:30 p.m., and the last at 11:30 p.m., September 28 to October 6... WDCH Dreams’ accompanying soundtrack was created from hand-picked audio from the LA Phil’s archival recordings. Sound designers Robert Thomas, and Kerim Karaoglu augmented these selections by using machine-learning algorithms to find similar performances recorded throughout the LA Phil’s history, creating a unique exploration of historic audio recordings. 

Adaptive music/sound artist Robert Thomas, as longtime New World Notes readers know, has been creating ambitious audio experiences across many mediums and technologies over the years. (Full disclosure: Also a pal.) I first met him when he was creating voice-activated installations and location-reactive soundtracks in the virtual world of Second Life. After SL he went on to co-create projects with luminaries like Massive Attack, Imogen Heap, Ben Burtt (sound designer for the Star Wars movies), and Hans Zimmer. (Robert and Zimmer worked with director Chris Nolan to create an interactive audio app for his movie Inception.)

Based in London, Robert tells me how he and his collaborators used machine learning to take and shape disparate performances and composers from the LA Phil's massive archive of recordings, and sculpt them into a completely new soundtrack:

Watch an AI-Controlled Avatar "Learn" How to Dribble In Real Time - Huge Implications for Games, AR/VR, and Real Life Robotics

What you're looking at above appears to be a pretty big breakthrough in artificial intelligence -- an avatar "learning" how to dribble in real time through AI. DeepDribble is a project from researchers at DeepMotion and Carnegie Mellon, and the idea is to demonstrate how an AI-controlled avatar can acquire the complex and subtle set of integrated physical skills and coordination that humans take for granted:

Basketball at its peak is known for intricate ball handling, quick pivoting, and fake outs. A good player will fluidly transition between have a variety of dribbling skills. In order to recreate this, the trained characters need to be able to transition between separately trained behaviors... Libin Liu and Jessica Hodgins use the same methods for training single skills to create a multi-skill control graph (which allows transition between motion fragments of two different skills). The training is done incrementally, behavior by behavior, to ensure integration occurs with proper transitions for each skill. The result is a player that can perform multiple ball handling skills in a variety of orders, changing course with seamless blending.

"First we build a bio-mechanical model of the digital character, using physics to simulate bones and virtual muscles," Deep Motion's Libin Liu tells me. "We put the digital character in the physically simulated virtual world and let it mimic how real players perform the motions using their muscle forces."

The training process, as you might have imagine, requires quite a lot of power -- but Liu says the learning arc took about as long as a single televised NBA game -- or even half of one:

Coming Soon to Unity 3D, AI-Powered Agents Able to Use Tools to Overcome Obstacles

They learn to use the orange cube to jump over the wall. This demo will be included in the @unity3d ML-Agents 0.3 release in about a month. #MachineLearning pic.twitter.com/MSWsHU6Bcy

— PunchesBears ㅎ㉨ㅎ (@punchesbears) February 7, 2018

You thought the virtual worm was impressive, watch this blue block "learn" how to use a tool to get over an obstacle. Created by a deep learning and procedural animation coding wizard called Punches Bears (because of course), the demo will be featured as a demonstration of Unity 3D machine learning code in an update that's coming in a few weeks:

Ray Kurzweil Once Bet Mitch Kapor That A.I. Would Pass the Turing Test by 2029 -- Because He Assumed Full VR Would Be Feasible By Then; UPDATE -- Kapor Responds: "I told Ray I was prepared to double down on the bet..."

Today I learned that in 2002, tech pioneer Mitch Kapor (a Linden Lab founding investor, among many other things), bet against futurist Ray Kurzweil that we wouldn't create a computer that could pass the Turing Test by 2029, writing:

The metaphor of brain-as-computer is tempting and to a limited degree fruitful, but we should not rely on its distant extrapolation. In the past, scientists have sought to employ metaphors of their age to characterize mysteries of human functioning, e.g., the heart as pump, the brain as telephone switchboard (you could look this up). Properly used, metaphors are a step on the way to development of scientific theory. Stretched beyond their bounds, the metaphors lose utility and have to be abandoned by science if it is not to be led astray. My prediction is that contemporary metaphors of brain-as-computer and mental activity-as-information processing will in time also be superseded and will not prove to be a basis on which to build human-level intelligent machines (if indeed any such basis ever exists).

There's a lot more to Mitch's skepticism, which is brilliant, but just looking at the state of things today, I'd say he looks likely to win. There's been a lot of interesting advances in functional AI (i.e., self-driving cars), and simulations of human creativity, but we still haven't seen any AI that can maintain a human-like conversation for any substantial length of time. So unless there's some massive breakthrough within the next 12 years, Kurzweil's going to owe the EFF a $20,000 donation.

Kurzweil's argument for passing the Turing test by 2029 is worth reading too, because his prediction is partly based on a flawed assumption about the growth of VR and virtual worlds: