New World Notes

New iPad App Uses Machine Learning to Capture Real World Objects & Convert Them Into Standard 3D Files

Caveat emptor and all that, but the iPad LIDAR Scanner just published in Beta seems seriously impressive, able to "capture" real world objects and convert them into imperfect but pretty usable 3D models. It works, says developer Laan Labs, by automatically extracting objects via machine learning (i.e. extrapolating data for unseen surfaces).

More info from iOS app page: 

I Was Going to Blog an Uncanny Valley Think Piece About Samsung's NEON "Intelligent" Avatars But Then Laughed My Ass Off Instead

Samsung's NEON! It's getting much buzz at this year's CES, so much so that I planned to write a thought provoking think piece about them with a mandatory Uncanny Valley reference, but then I watched this video and got to 4:20 (no I'm not high) and just laughed. Does any human in the history of the world say "pizza" like that, unless they have a gun pointed to their head?

Anyway, more on the official homepage:

Listen: Music Inspired by Gödel, Escher, Bach Co-Created by an Artificial Intelligence Trained on Bach and Human Interaction With the AI

Click above to hear something fairly amazing: Music created by a merger of Bach, artificial intelligence, and live interaction by a human audience. It's from the new album "The Eternal Golden Braid" by my pal adaptive music/sound artist Robert Thomas, who's been creating works of music shaped by virtual reality, digital interaction, and AI for well over a decade. (As here with Chris Nolan and Hans Zimmer and more recently here with the LA Phil).

This current project, as the title suggests, was inspired by the landmark book Gödel, Escher, Bach: an Eternal Golden Braid, and incorporates music from Bach himself. As Robert explains in his liner notes:

This piece started life as a collection of renderings from a machine learning process which was trained on a large database of works by Bach. I then curated some fragments from it and used them as a starting point for a composition for string trio.... We wanted to explore having the audience interact live with the music in a collective strange loop, so I constructed the piece in two "levels" which the players could move between at any time. Level 1 was calmer and more introspective and represented by Blue. Level 2 was more chaotic and extroverted and represented by Red. When the piece was performed the audience held up colored cards, which were recognized by a machine learning camera on the stage. This fed a signal to the players which told them which musical notation to read. 

So this is, as Robert puts it to me now, "music composed in collaboration with an AI, which was trained on Bach, then performed by the London Contemporary Orchestra, with the audience interacting, guiding the music. Aspects of the music were inspired by themes in Gödel, Escher, Bach, though: At the start and end you hear kinda shepherd tones played by the strings; these are one of the strange loops referred to in Marcus du Sautoy's talk about [Hofstader's book]."

The live performance looked like this:

John Carmack, VR Pioneer Who Once Described Developing VR as a "Moral Imperative", No Longer Focused on Developing VR

Oculus CTO and VR pioneer John Carmack is stepping almost completely away from the company and virtual reality development in general to go in a totally different direction:

Starting this week, I’m moving to a "Consulting CTO” position with Oculus.

I will still have a voice in the development work, but it will only be consuming a modest slice of my time.

As for what I am going to be doing with the rest of my time: When I think back over everything I have done across games, aerospace, and VR, I have always felt that I had at least a vague “line of sight” to the solutions, even if they were unconventional or unproven. I have sometimes wondered how I would fare with a problem where the solution really isn’t in sight. I decided that I should give it a try before I get too old. I’m going to work on artificial general intelligence (AGI).

I'm not sure how many people in the VR industry grasp the significance of this move. The announcement comes only days after Carmack said this about the current state of VR:

"I'm often kind of grumpy around the office because I really haven't been satisfied with the pace of progress that we've been making. When I'm in VR I see the magic there, but my brain is always throwing up these giant 'to do' Post-It Notes on top of everything, reminding me of all the work that's yet to be done."

Putting the two statements together, it's hard to escape the conclusion that Carmack has decided that virtual reality won't or can't be improved, or sufficiently matured into a mass market product, within the span of his career. And so instead, he's devoting the rest of his work energies to developing AI.

The statements also stand in sharp contrast to how Carmack fairly recently described the importance of bringing VR to the masses -- as "a moral imperative".

As he explained to me for my Wired article back in 2016:

Summer Reading: JP Delaney's The Perfect Wife, the Story of a Woman Who Wakes Up Dead And An Artificial Intelligence

Cajsa Lilliehook has a review of an intriguing new high-tech thriller on her book blog: The Perfect Wife by JP Delaney, which seems like Philip K. Dick meets Gone Girl.

"It starts out like a conventional women's thriller," Cajsa explains. "She wakes up, feels strange, memories are there but confusing. It seems like the usual waking-up-after-accident-thriller where someone dies or loses memory. But no, son and husband are alive. She's dead. And a cobot (companion robot) is waking up with her consciousness."

As that suggests, the novel touches on themes and ideas many New World Notes readers may find appealing. More from Cajsa:

Rutger Hauer's Roy Batty as Depicted by Deep Dream AI & Nettrice Gaskins

Nettrice Gaskins, an artist whose work explores "techno-vernacular creativity", created this beautiful tribute to the most famous character played by actor Rutger Hauer, who died yesterday: Roy Batty, the lead replicant in Blade Runner.  As with her other illustrations, her medium for creating this image is Deep Dream, the neural network AI. She randomly played around with two-three algorithms to generate this image, she tells me, one of them likely being Leon Gatys’ Neural Style Transfer. So if you like, this is a portrait of an AI generated by an AI.

See many more of Nettrice's Deep Dream portraits here. And since roughly half of you are mouthing the words "Like tears in rain" while you read this, see that scene below:

Watch: AI Agent "Learns" to Crash Into Physics-Enabled Wall Like a Stuntman

Watch this new tech demo from DeepMotion, the AI company which taught NPCs how to argue, and learn to dribble basketballs: An AI "stuntman" agent who can run through a physics-enabled breakable wall -- and adjusts his momentum based on the wall's thickness.

"[W]hat's changing each iteration is the strength of the wall," DeepMotion's Anna Ploegh tells me, "by way of adjusting the mass of each brick. So the running movement is responding each time to a difference in the wall obstacle. Over each iteration, that mass of the bricks increases until it is too difficult for the character to jump through."

This is why each crash in the demo is different, as is the way the AI agent falls. This is in contrast to technology often used in videogames to simulate a similar effect, where an NPC is suddenly turned into a physics-enabled ragdoll (i.e ragdoll physics). Here's why:

Lupita Nyong'o in Us Re-Imagined by Afrofuturist Artist & Deep Dream AI

Love these two images by Afrofuturist artist Nettrice Gaskins, who uses her technique of creating Black artist portraiture through the collaboration of Google's Deep Dream neural network on stills featuring Lupita Nyong'o from Us, Jordan Peele's new acclaimed mega-hit.

"I use images with abstract textures and them composite the results -- sometimes [with] Photoshop and sometimes Microsoft Word (true story)," Dr. Gaskins tells me. "My work in Deep Dream is similar to my collage work using traditional artist materials. The process I use to draw and paint is what I'm using in Deep Dream." More specifically:

Watch: DeepMotion Creating AIs Which "Learn" How to Argue With Each Other

Here's a cool new AI demo from DeepMotion, the company which recently taught an AI how to "learn" the art of dribbling. In this demo, they've simulated a group of AIs infinitely arguing; even if they gesticulate so wildly, they knock each other down, they right get back into the squabble:

With only a few learned skills, the character controllers used in this demo are relatively simple, as are the mechanics of the simulation. When the simulation is played, each character transitions between the three argumentative gestures at random, creating the appearance of a lively debate. When the characters experience external force from a user, the environment, or another character, they fall and then promptly get up. The primary goal is to continue arguing.

Fit a few hundred of these in an auditorium, and you could simulate a session of Congress! (With possibly less pratfalls.)  

"The short term plan," Deep Motion's Chief Scientist Libin Liu tells me, "is to let the characters learn argument and idling actions from those unstructured motion sequences, so that they can perform them freely based on scripts, user commands, or mimic user actions based on sparse input (like 3 point tracking)." 

Most game-focused AI is developed around core mechanics like pathfinding and combat, but Liu and his team believe subtle group interactions like this can do much to make a game world seem that much more real:

Afrofuturist Artist Creates Stunning Portraits of Black Artists With Deep Dream AI Algorithms

This utterly gorgeous image of British singer Sade was created by Nettrice Gaskins, an African-American artist whose work explores "techno-vernacular creativity" and Afrofuturism. As that suggests, her medium for creating this image and others in her series of portraits is unique: Deep Dream, the neural network AI.

“I used Deep Dream Generator's algorithm, which is inspired by the human brain," she tells me. "The generator uses the stylistic elements of one image to draw the content of another. The first picture defines the portrait I want to create and the second picture I upload gives the program a style to simulate. The process may include more than one pass in the generator, using different style images, until I get the image I want." This portrait of Sade, for instance, "[Is] a composite of multiple Deep Dream images."

Here's another one I love below, a portrait of Pulitzer-winning hip hop artist Kendrick Lamar, re-imagined by Dr. Gaskins and Deep Dream through a symphony of sharp angles that evoke his epic, staccato rap style: