ly bad.
Before I get into this post, I wanted to take a second to talk about my history with generative AI. I was an early adopter of AIDungeon, which means I was one of the first laypeople the get a taste for generative storytelling and “copy” as a medium. Back when AIDungeon ran on GPT1 and constantly lost context or inserted garbled markup into responses, it was still a ton of fun to just see what you could get it to say. The most impressive thing, of course, was the novelty of the text. Unlike Cleverbot or SIRI or Google’s results summaries, GPT never seemed to plagiarize – at least not in a way that could be searched and caught – even at its best. The text it generated felt unique, however confusing it may have been. The appearance of Sailor Moon fan characters or a datasheet about the M1 Abrams in your text was just silly flavor, and even searching that wouldn’t quite explain how it reached those outputs.
So, it’s 2023, and generative AI of all sorts is being touted as a “solution” to writing. Curious, then, that it only seems to be large corporate entities and scammers who believe it to be that solution, or that writing needs a solution to begin with.
When I saw the internet ablaze with news of the latest genius of ChatGPT, I went back to AIDungeon to see the updates and changes to generative language models that inspired this new wave of excitement, and it turned out… there were none. In fact, just as smartphones condensed into indistinguishable glass slabs and blockchain technologies concentrated into a bunch of men you absolutely do not want to sit next to in a social situation, it seems that text AIs almost instantly narrowed down into a set of tools to generate a small set of indistinguishable PR-sanctioned C O N T E N T. Not only is AI-generated text instantly recognizable to a modestly experienced reader, but it seems like in losing its unpredictability, it’s also lost its “talent” for spontaneous creativity.
Creative Writing
So much of early generative text was trained around, and specifically designed for creative writing. The original authors of GPT trained the software on “unpublished” fiction, feeding 7000 books of creative writing into the engine and nothing else. This is how the original GPT functioned – it was designed to generate believably human fiction text by replicating patterns in fiction text written by humans, and it did that (mostly). The mechanism of machine learning, at its core, is discovering and replicating the ingrained and otherwise invisible patterns in a body of data to generate more data with similar traits. If you have 7000 novels and you want 7001, AI is great for that.
As the problem set grows, the size of the dataset needed for training, and thus the entropy of that dataset grows as well. When published (public) and nonfiction content began percolating into GPT’s model, it was because the network was no longer being asked to generate outputs that mimicked a single kind of input – now, the point was for the model to be selective in which subset of training data should influence the output, based on a prompt. This is where our fanfiction characters and random datasheets start popping up in AI Dungeon.
As a tool for creativity, at least as a fun gimmick, this is intriguing. Now, instead of getting a new work of fiction that combines big concepts from a prompt and format/syntax from a dataset, you start getting dataset in your prompt and prompt-like things in your dataset. Someone may request a paragraph about the fall of Rome, and get something that includes My Little Pony characters. It’s hilarious, obviously wrong, and makes for great live entertainment. Major streaming channels like Jerma985 and Vinesauce ran weekly series playing AIDungeon specifically attempting to prompt these sort of outputs.
The key thing, though was that they understood that the engine was designed to generate fiction with a high degree of randomness. There was a mutual understanding that the software was a garbage-in, garbage-out system that got things wrong constantly.
ChatGPT ruins everything
With the launch of GPT-3 and GPT-4 (and ChatGPT), the software became much more accessible, as well as more attractive to people looking to use it for “serious” purposes. AIDungeon updated their model to improve understanding of background data and contexts, and to filter out inappropriate responses, while ChatGPT became the main engine for people to engage with these models.
Except, ChatGPT was garbage – not only was its main improvement “more data”, it made no disclaimers regarding the limitations of a neural network model. In 2022, it opened to the public and users-turned-advertisers almost immediately started making claims of “intelligence” and “sentience”, suggesting that this technology would be world-altering when it wasn’t actually generating anything new or different from the junk that GPT2 could, albeit with more believable syntax.
At launch, there was a race to the bottom where shills and scammers were attempting to prove ChatGPT’s competence at everything from creative writing, to advertising, to technical problem solving. Suddenly, this system that was widely understood to be good for 2 things (disguised creative plagiarism and hilariously incorrect output) was being treated like a genuine superintelligence. About a hundred identical competitors sprung up, all claiming to do something special with the same set of tools, and somehow, they just kept raking in VC money. They still are!
The problem, of course, is that generative language models are still garbage-in, garbage-out. Even if you look at purpose built filters for those models, like AIDungeon’s Dragon, it is only effective as long as the user stays well away from the edge cases and gives clean, pattern-matching inputs that can be answered with clean, pattern matching outputs from the database. And that’s just for fiction.
But of course, it doesn’t stop at fiction. Fiction doesn’t win VC money – that’s what lies are for, and oh boy, are generative text models good at lying.
Garbage Out
Remember when I said that the draw of AIDungeon was that it would often generate hilariously wrong outputs that lacked context? It’s important to clarify that this was not a bug, it is a feature – it is the foundational principal of operation for these products. A high degree of randomness and noise is required in these programs to prevent the network from generating identical text every time, and sometimes that leads to the output having no correlation at all to the input. Rather than devise literally anything better than an 80-year-old computer science thought experiment, “Machine Learning” experts in the industry have simply accepted this as a fact of life, and instead focused on improved input and output sanitization to keep the machines in check.
It’s sort of like if you had a cow that produced raw sewage 50% of the time someone attempted to milk it, so rather than figure out what’s wrong with the cow, they feed it larger meals so that it can be milked more often, make sure that only approved professionals (who will not disclose their training or what they witness) are allowed to milk the cow, and you pay them a fee to sort through the buckets of “milk” to only give you ones they’re pretty sure aren’t raw sewage.
Now, with models focused on generating clean syntax, we end up with enormous players like Google and Microsoft shitting out software that advertises its ability to summarize web content by… generating blatantly incorrect summaries of web content that reads like natural language. Imagine a future where everything you read is completely wrong, but it sounds super relatable and convincing – a true utopia!
While the end-user experience has cleaned up wonderfully, the underlying process is still extremely dumb. The data scientists have crammed more data into the training engine, but the patterns the machine discovers are only those that produce beautiful output, not those that work more efficiently, accurately or to any other end than that original design.
Pure, Filtered Sewage
The end result of all this, I believe, is a misplaced faith in the capabilities of generative text, as well as a growing problem with these engines being used to generate “content”. The enormous volume of pattern-matching but unique data created is suffocating the internet, and the demented belief that it this data is worthwhile is suffocating actual writers, educators and other professionals.
With the writers’ strike approaching its 4th month without resolution, it’s become abundantly clear that corporations don’t like the idea of paying creatives a living wage. This makes their threats to use ChatGPT for screenwriting even more hilarious, because it has time and time again proved itself to only be worthwhile as a joke about how bad generative text is for creative writing. The pop culture reaction to the software even captures this sentiment, with videos from massive creators poking fun at how bad a job the software does by letting it write segments or entire videos for them.
It is blatantly obvious to anyone who has worked with generative language software, whether its ChatGPT, Jasper, Copy.ai, Bard, Bing Assistant, or any of the hundreds of imitators, that it is only good at one job – making things sound syntactically correct. Context and accuracy are an afterthought added on in postprocessing where possible (but not without much effort). The bots can easily be manipulated to deliver false information or argue for incorrect conclusions without the user even knowing they’re asking for these changes, and for every clean, attractive output, you are just as likely to get absolute garbage.
Sometimes it writes you a news story, sometimes it writes you a short novel about a character who works for the news, sometimes it writes you an InfoWars article that claims the man who works for the news actually works for lizard people living in the hollow earth. Anyone who has worked with this software knows not to leave it unsupervised, or it will begin cranking out blog posts about how World of Warcraft players are clamoring for the introduction of Glorbo to the new Glorbo update.
And that’s the other big problem – these AIs are absolutely not autonomous, however people attempt to make them so. The farmer that sells you buckets of hopefully-not-sewage is still going to sell you a few buckets of sewage, and as the customer, it falls upon you to taste-test them to figure out which ones are good. Barring that, you risk bottling and shipping mysteriously white raw sewage to unsuspecting customers.
But what if you don’t care?
This age old question has puzzled raw milk salesmen and blogspammers for ages, and thankfully, the answer is simple: Sell it anyway, because there are no ramifications!
Raw Milk
Anyway, I decided to try out some free AI software today using a trial on Copy.ai. I used up about half of my free trial on responses that explained to me that it felt responding would be violating the law, and the other half on completely incorrect nonsense about technology that I would have totally believed, had I not known better.
Right before I ran out of credits, I asked it to generate an ad for peanut butter in the voice of a Dyson UK commercial. It sounded extremely good and in-character until I realized it was just directly plagiarized from a real ad, with the product name replaced with “peanut butter” and random adjectives replaced with “creamy”. I guess “copy dot ai” lived up to its name.
We are truly living in the future.
Profanity Filter Testing Domain 