New World Notes

NVIDIA Already Launching AI Model to Generate 3D Objects: Think Midjourney Meets The Metaverse

Sooner than you might have expected and probably quicker than you were planning, NVIDIA just announced GET3D an AI model for generating virtual world objects -- think Midjourney meets the Metaverse:

NVIDIA is looking to take the sting out of creating virtual 3D worlds with a new artificial intelligence model. GET3D can generate characters, buildings, vehicles and other types of 3D objects, NVIDIA says. The model should be able to whip up shapes quickly too. The company notes that GET3D can generate around 20 objects per second using a single GPU... The model can create objects with "high-fidelity textures and complex geometric details," NVIDIA's Isha Salian wrote in a blog post. The shapes GET3D makes "are in the form of a triangle mesh, like a papier-mâché model, covered with a textured material," Salian added.

The NVIDIA blog post doesn't mention anything about rigging these 3D models to character/avatar skeletons, so that might not be part of the AI's purview. In any case, this is the kind of thing that metaverse artist and technologist Jeff "AM Radio" Berg was anticipating only two months ago:

Watch: Shopify Experimenting With Mixed Reality, AI-Powered Fashion

AI is enabling some really magical experiences. 🪄

Originally planned to share this tomorrow, @sergeyglkn scooped us! 😜

Keep on eye on this space. It moves fast! 💨 pic.twitter.com/Of8PgJWZMM

— Russ Maschmeyer (@StrangeNative) September 19, 2022

This is a fun and impressive demo: Online e-commerce site Shopify prototyping AI image generation used in fashion. (Russ Maschmeyer is AR/VR head at the tech company.) It was created in collaboration with Sergey Galkin, who explains how it was made on his own Twitter thread: "The prototype was made in Snapchat Lens Studio + post-processed in Adobe After Effects. Images were generated by Dalle 2." 

They're demonstrating this for a real world clothing context, but the virtual world applications are even more obvious. (I mean how many wacky T-shirts does the average consumer really buy, versus new avatar outfits?) Being able to adjust the style, fit, and composition of the clothing item itself seems like the next obvious next step.

Midjourney Solely Credited as Article Illustrator in Major News Publication

The sharp eyes of New World Notes' image curator Cajsa Lilliehook caught what looks to be a pretty major breakthrough (or first step toward disaster, depending on your point of view): The Bulwark, a major US news analysis publication, is using Midjourney to create illustrations for their articles -- and crediting Midjourney as the image source. (As opposed to, say, an actual human illustrator.) From a cursory search, this seems to be the first example of Midjourney used for a non-tech, non-grassroots illustration by a professional publication that would normally pay a graphic artist and/or image bank like GettyImages or Shutterstock for work like this. 

So this is notable! Also notable: It's not very good. Meant to illustrate Trump's questionable legal defense fund, the image doesn't convey any of that context at all, and actually looks more like a pro-Trump graphic. (Except, that is, the "Save America" slogan is washed out by the background -- a sloppy look, whatever your political perspective.) In other words, a human illustrator would have done a much better job.

But then again, hiring a human is a lot more costly.

Four Problems With the “AI Wins an Art Competition” Story -- And Insights from Acclaimed Metaverse Artist Jeff "AM Radio" Berg

I know it's a relatively slow news week in technology, but this story going viral seems so silly:

Artists in Colorado are up in arms over the first-prize winner at the annual state fair’s art competition: a digital collage made by artificial intelligence.

The work, titled Théâtre D’opéra Spatial, is attributed to Jason Allen via Midjourney, which cites the name of the program used to blend different images and the artist who entered the resulting work into the competition...

Allen, who is president of Colorado-based tabletop gaming company Incarnate Games, added that he believes his work is a natural extension of his own creativity. After all, he sets the parameters in which the A.I. creates; afterward, he processes them in Photoshop and then finishes them in Gigapixel. 

There will certainly be a time for debating if AI-generated images are art, but now is not that time. For starters:

Midjourney Tips: How to Get the Best Results from the AI Image Generator

One bright thing. #midjourney #midjourneyAi pic.twitter.com/0gy6imYdPK

— Arden_AiArt (@mj_arden) August 24, 2022

SLer Kala Bijoux has been playing around with Midjourney, the AI image generator featured as an art tool here last week, posting many of the most amazing results to her Twitter account. She very helpfully sent along some tips for people getting started with the platform: 

1) Troubleshooting: If you are having trouble with the website or Discord log out and log back in again. This has solved 99% of the problems I've had when setting people up.

2) Check out the community page on the Midjourney website for prompt ideas and for examples of how people are writing their prompts. I find this a very useful way to see what words and phrases people are using to get specific looks. The community is currently a bit divided on people just copying the prompt completely without changing or tweaking it. Some people don't care because they feel it is a recipe (and you will never get the exact same image as the first person), while others feel like they spend a lot of time crafting the perfect prompt and dislike people copying it. I usually change it a bit (or a lot!) to suit my needs.

3) Play with aspect ratios! The default image is a square, but you can change the image's aspect ratio by adding ' --ar ' and then the aspect ratio at the end of your prompt. One of my favorite aspect ratios is 9:16 so I would write --ar 9:16 at the end of my prompt. For example, one prompt might be "evil queen's castle on a hill, scary, dark, gothic, detailed oil painting --ar 9:16" [pictured at right - WJA]. This website has a long list of aspect ratios that work in Midjourney. 

 

And yes, you can add images from Second Life (or for that matter, any metaverse platform) to Midjourney:

Here's Why Midjourney-Generated AI Images Can Be Called Art

Really smart points by reader "Liv", on whether AI generated images like Dr. Nettrice Gaskins' Midjourney-based works can be called art:

Like all artforms new technology comes along, and equivalent debates arise in different fields. “Is electronic music really music?” Same question.

Some people will always value craft (or instrumentalism, in the music case) and so for them no it won't be art. And that’s fine.

For the rest of us art is about ideas. Brian Eno talks at length about how he allows machines (or software) to create sounds. and how his role becomes that of a curator. When the pseudo-random oscillators create a pleasing result only then does he mark that recording as one to keep. And what order does he edit and present them in? That is the art that makes his ambient music interesting.

Or consider another form of art where it's also possible to cheaply and quickly generate many, many digital images:

Afrofuturist Nettrice Gaskins Creates Utterly Stunning Art With Midjourney

 

Dr. Nettrice Gaskins, an acclaimed Afrofuturist scholar who pioneered the Deep Dream neural network as an art medium back in 2019, is now experimenting with Midjourney, the new AI project that uses Natural Language Processing (NLP) to generate new imagery.

The results, some of which you can see here, are deliriously stunning. Nettrice's image above, for example, was generated by a line from Toni Morrison's Song of Solomon (one of her favorite novels): 

“For now he knew what Shalimar knew: If you surrendered to the air, you could ride it."

"[Midjourney] tries to understand human language," Dr. Gaskins explains. "It's about the text; the order of words. The cool thing about NLP AI (Midjourney, DALL-E, etc.) is another person can use the same exact text prompts (same order, etc.) and never be able to duplicate the ones I generated. This means every single image is one of a kind / unique."

Here's another that she generated through inputting the text, "Afrofuturistic Black woman botanist in trippy alien utopia":

Watch: Guy Recreates Bizarre Images from Dall-E Mini in Fortnite Creative

As a fun companion video to two themes discussed last week, the future of art in the AI era, and the future of 3D creation on next gen metaverse platforms, here's both at the same time: A guy creating builds in Fortnite Creative based on images suggested by Dall-E mini. The images themselves are pretty mild (an alien mountain, Darth Vader celebrating his birthday), so I'd love to see someone try to recreate the more bizarre and nightmare images Dall-E often extrudes from its cursed system.

As demonstrated by the dulcet-voiced Mustard Plays, this is also a good demonstration of the Fortnite Creative tools, which include a huge assortment of prefab mesh objects and primitives, both of which can be imbued with physics and other interactivity. And yes, it also enables multi-user creation, which Mustard Plays leverages to host a collab creation contest -- watch below:  

Pioneer Metaverse Artist Jeffrey "AM Radio" Berg on the Future of Art in the A.I. / DALL-E Era

From AM Radio's Flickr

Update, 7/15: Bumped up as long-ish weekend read and conversation for the weekend!

Jeffrey Berg, the artist and technologist who became famous as the Metaverse artist known as "AM Radio", a Second Life avatar who created profoundly surreal and poignant immersive installations like "The Faraway" (above), a rusty train lost in the wheatfields, recently discovered a surprising thing:

It is now possible, using programs like Craiyon, a text-to-image generation tool powered by AI (a variation of OpenAI's DALL-E), to automatically create images strikingly similar to the Second Life installations which took him many hours if not days to create in 3D.

See below:

"Was messing with Craiyon today and I wondered the impacts it could have on metaverse space creation," as he mused to me recently, sharing the results above. "3D geometry rather than a generated image from plain text input doesn’t seem like a huge stretch."

That started a long, fascinating, and -- this being AM Radio, after all -- moving conversation with Jeff on the future of art in the era of DALL-E, AI and machine learning in general, and promising AI-based art like that by Dr. Nettrice Gaskin:

There's Not Enough Computing Power in the World for a Fully Sentient, Independent AI (Reader Opinion of the Week)

Smart thoughts from reader and tech exec veteran Luther Weymann, reflecting on the questions of AI sentience brought up by a recent Washington Post story on the Google developer who believes the company's chat bot has already achieved it:

Artificial intelligence programs running on machines use databases on computer systems to receive instruction for action and sound. These databases can deliver information from dedicated databases. Still, more likely for more human-like interaction, the data comes from many types of databases into a central repository and then to the machine.

The ability to have anything even approaching what we socially call “sentient AI” will require mainframe processing and a massive database repository at the ready. Conventional servers process about 300 million operations a second. To truly power AI into a world where the AI machine will be really human-like is going to take mainframe power into the 300 billion operations a second capacity. Mainframe databases, when appropriately distributed, can meet that challenge.

But now comes the exciting part: