« Sumptuous SL Roleplay Pics Inspired by Avatar: Way of Water & Beyond | Main | Story of a Sad Little Robot Told in Words & Whimsically Beautiful SL Images »

Wednesday, March 08, 2023


Feed You can follow this conversation by subscribing to the comment feed for this post.


Most of the key ML/AI papers come from Google Brain and DeepMind. The 2017 paper on transformers "Attention is all you need" came from them too. OpenAI based their GPT models on that (GPT = Generative Pre-trained Transformer).

Since OpenAI couldn't have an edge on research, what could they do to be competitive? Release earlier. That's pretty easy as Google and DeepMind are super cautious.
Microsoft then joined and invested on OpenAI, of course they want to compete against Google and now you have Bing powered by a language model. I've got the early access to it since a while and it looks more capable than ChatGPT.

However, as you could see, there have been issues immediately. Mostly because of Reddit users trolling the model, then journalists went there and took the sensationalist route. Then Microsoft had to take action by regulating and restricting the usage of the model and now it's harder to investigate its capabilities.

As for the NY journalist, he did the same move as Lemonine: prompt leading. The model then gives you what you request (or what you hint, even subconsciously: someone isn't aware of doing that) and the model follows your prompt.

ChatGPT, by default, essentially roleplays as a robotic assistant, so people use it as a sort of oracle or they ask calculations to... a language model, apt to, guess what, language tasks. You can ask it to summarize, to generate text of any kind in any style, etc.
Character.ai, developed by previous developers of Google's LaMDA, can pretend to be any character, from game characters to famous people, imitating their speech and personality.
And you can trick these models by making them generate anything else.
However, prompt engineering could also be malicious. OpenAI (and Microsoft with Bing Chat) tries to prevent that with some filter and pre-prompting (there is a hidden prompt before your input), but that works only so much.

At any rate, it's true that these models work with patterns. Neural networks (your brain too) are pattern recognition systems. This is also how you try to predict things, from the next word to finding patterns in stock market graphs. Often wrongly and that's also how stereotypes happen, but pattern recognition has its advantages too.
It's also how you learn, by training, repetition, until you "get it", as the mentioned song and the sentences you heard so many times; sadly propaganda, especially on social media, does the same by repeating misinformation, aided by bots, until people begin to parrot it; so, ironically, they are trained by propaganda bots.

The human brain not only predicts the next word and the next sentence, though, but also tries to predict what the interlocutor may reply. Having empathy and a developed theory of mind helps.
Since that also helps in predicting the next word (or better, the next token), there are papers investigating possible emergent abilities in language models:
https://arxiv.org/abs/2206.07682 "Emergent Abilities of Large Language Models"
https://arxiv.org/abs/2302.02083 "Theory of Mind May Have Spontaneously Emerged in Large Language Models"
(there are many more)

Hallucination is also another feature of neural networks, that fills the missing spots, beginning with the blind spot in the retina up to your dreams. So it's not something we are going to get rid. Instead, without hallucinations you couldn't have Stable Diffusion, nor in-painting techniques. Someone even thinks the human brain lives in a controlled hallucination and even the sense of self and consciousness may be illusory.
Anyway, that's true also for your "faulty", lossy fuzzy memories (that however can archive a huge amount of info), so that you have to check your notes and photos, because that red tulip you remember maybe it's your brain making it up for some missing information and it was a pink petunia instead.

Since these LLMs are good at making stuff up and they are language models, one of the best use cases is indeed chat-based roleplay gaming. We had some examples of that several years ago already with e.g. AI Dungeon. GPT-2 based AI Dungeon was hilarious with all the nonsense generated and it was quickly incoherent. GPT-3 based AI Dungeon was somewhat better, but still derailing, roleplaying for you, etc. But have you tried paragraph-roleplaying with ChatGPT? It's way more coherent. Essentially it's still GPT-3, but much improved (it's GPT-3.5) and it takes advantage of a larger context window, InstructGPT and several other things.

Betty Tureaud

If you want to try out chatGTP in Second Lifet go to BOO CLUB and chat withe "Betty Bot" Blais and I made a chatbot for you.
Taxi: https://maps.secondlife.com/secondlife/Danish%20Visions/188/233/250


Not only is content 'mediocre', but it gets *more* mediocre and fuzzy over time: https://www.newyorker.com/tech/annals-of-technology/chatgpt-is-a-blurry-jpeg-of-the-web

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Your comment could not be posted. Error type:
Your comment has been posted. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.


Post a comment

Your Information

(Name is required. Email address will not be displayed with the comment.)

Making a Metaverse That Matters Wagner James Au ad
Please buy my book!
Thumb Wagner James Au Metaverse book
Wagner James "Hamlet" Au
Valentine Dutchie Second Life gift
Bad-Unicorn Funny Second Life items
Juicybomb_EEP ad
My book on Goodreads!
Wagner James Au AAE Speakers Metaverse
Request me as a speaker!
Making of Second Life 20th anniversary Wagner James Au Thumb
my site ... ... ...
PC for SL
Recommended PC for SL
Macbook Second Life
Recommended Mac for SL

Classic New World Notes stories:

Woman With Parkinson's Reports Significant Physical Recovery After Using Second Life - Academics Researching (2013)

We're Not Ready For An Era Where People Prefer Virtual Experiences To Real Ones -- But That Era Seems To Be Here (2012)

Sander's Villa: The Man Who Gave His Father A Second Life (2011)

What Rebecca Learned By Being A Second Life Man (2010)

Charles Bristol's Metaverse Blues: 87 Year Old Bluesman Becomes Avatar-Based Musician In Second Life (2009)

Linden Limit Libertarianism: Metaverse community management illustrates the problems with laissez faire governance (2008)

The Husband That Eshi Made: Metaverse artist, grieving for her dead husband, recreates him as an avatar (2008)

Labor Union Protesters Converge On IBM's Metaverse Campus: Leaders Claim Success, 1850 Total Attendees (Including Giant Banana & Talking Triangle) (2007)

All About My Avatar: The story behind amazing strange avatars (2007)

Fighting the Front: When fascists open an HQ in Second Life, chaos and exploding pigs ensue (2007)

Copying a Controversy: Copyright concerns come to the Metaverse via... the CopyBot! (2006)

The Penguin & the Zookeeper: Just another unlikely friendship formed in The Metaverse (2006)

"—And He Rezzed a Crooked House—": Mathematician makes a tesseract in the Metaverse — watch the videos! (2006)

Guarding Darfur: Virtual super heroes rally to protect a real world activist site (2006)

The Skin You're In: How virtual world avatar options expose real world racism (2006)

Making Love: When virtual sex gets real (2005)

Watching the Detectives: How to honeytrap a cheater in the Metaverse (2005)

The Freeform Identity of Eboni Khan: First-hand account of the Black user experience in virtual worlds (2005)

Man on Man and Woman on Woman: Just another gender-bending avatar love story, with a twist (2005)

The Nine Souls of Wilde Cunningham: A collective of severely disabled people share the same avatar (2004)

Falling for Eddie: Two shy artists divided by an ocean literally create a new life for each other (2004)

War of the Jessie Wall: Battle over virtual borders -- and real war in Iraq (2003)

Home for the Homeless: Creating a virtual mansion despite the most challenging circumstances (2003)

Newstex_Author_Badge-Color 240px
JuicyBomb_NWN5 SL blog
Ava Delaney SL Blog