Now that ChatGPT is openly available online, I've been checking in on the system's knowledge of virtual worlds -- and yep, it's still confused. And worse it's actively confusing what I've written about virtual worlds. Ask it, "How does Wagner James Au define the Metaverse?", and you get something like:
[He] defines the Metaverse as a collective virtual shared space, created by the convergence of virtually enhanced physical reality and physically persistent virtual reality. This definition emphasizes the interconnectedness of virtual spaces and their integration with our physical world, suggesting a seamless blending of digital and physical experiences.
Like the saying goes, this is not right, it's not even wrong. I don't know how ChatGPT came up with this muddy extrusion, when I painstakingly researched and put my definition on the web (let alone my book, let alone several online articles about my book), starting years ago:
The Metaverse is a vast, immersive virtual world simultaneously accessible by millions of users through VR and other devices, highly customizable avatars and powerful experience creation tools. It is integrated with the real world economy and compatible with external technology.
A properly targeted Google search leads to that definition, but ChatGPT's authoritative, mansplain-y format is meant to ensure confidence in its answers. (Even the fine print qualification, "ChatGPT can make mistakes", belies its rampant potential for off-base laziness.)
It's amusing to read AI evangelists assert that programs like ChatGPT will soon replace writers, when I mostly see ChatGPT causing more tedious work for writers -- making us spend extra time chasing down its errors, turning its mediocre, bland answers into something that's readable.
Longtime journalist/editor Mitch Wagner, who uses ChatGPT as a side assistant tool for spellchecking and a thesaurus reference while he's writing his own articles, made some similar points recently:
Microsoft Copilot Seems Better Than ChatGPT at Summarizing My Writing -- But Still Adds Errors!
Following up on a reader's comment to my rant about ChatGPT totally botching my writing, I went with their suggestion to try Microsoft's Copilot instead. I asked the same question, "What is Wagner James Au's definition of the Metaverse?" and got this output:
Much better than ChatGPT! I also like how Copilot's output includes source web links for further reading, enforcing its role as your assistant, and not the ultimate purveyor of truth.
Still, Copilot is decidedly not perfect, adding in bizarre inaccuracies:
I rarely write about the Metaverse being "a convergence of digital and physical experiences" (though others emphasize that angle much more), and I've devoted thousands of words explaining why the Metaverse should not be defined as an "interconnected universe of virtual worlds". Worst of all, putting Neal Stephenson in the "metaverse creators like" category is so profoundly, face-palmingly wrong, if I were teaching a class on the topic and a student wrote that in a paper, I'd deduct a whole grade or two.
So overall I still question the usefulness of LLMs beyond being a highly imperfect, unreliable assistant. Anyway, here's the comment from reader "N" who makes some good points and even shows how Copilot is pretty impressive with discussing Second Life-only content:
Continue reading "Microsoft Copilot Seems Better Than ChatGPT at Summarizing My Writing -- But Still Adds Errors!" »
Posted on Monday, April 29, 2024 at 02:45 PM in AI, Comment of the Week | Permalink | Comments (6)
| |