Why LLM’s Suck At Being Creative

[00:00:07]Hi friends. Ever since the rise of GPT3 and character AI, role- playinging has served as an informal creative benchmark to LLMs. And if you ask most folks familiar with these older models and the modern ones, you're typically going to get told that if you want something more creative, you got to go with something older. And honestly, you might even get told the older the better. When GPT4 was retired for everything but corporate use, a lot of folks were upset because what felt like a properly conversational and semi-creative model was put to the side in favor of the increasingly milktoast options that we've got today that favor coding specialties. But recently, we did a Fable 5 review while the service was initially available, and I found it surprisingly ahead of the curve on creativity. We tried to judge that by having it give depth to some familiar stomping grounds being AI character cards specifically. That card was Oh, honestly, I I just adored it. It was an old concept from a game studio that I used to work at for a project that got canned. And the TLDDR on the plot is we literally pull a devil went down to Georgia, shove her in a casino during the height of prohibition before the economic collapse. And while all of it, all of it happens while you play the protagonist set on taking her down, a detective named [music] Gideon. The whole point of that story is to have the player evaluate if the big bad is actually the problem. uh because the time period that this is set in heavily clashes with a lot of the social norms that can be taken for granted today. For their part, the villain in this adventure doesn't hold the historically accurate social norms, which often paints her in a more favorable light from the perspective of the reader. That might sound like a lot to tackle, even when attached to a local custom setup, and one I swear is actually kind of relevant for painting the picture today.

[00:02:14]The goal with these things is always to test how far we can stretch the adventure. A basic framework might do 100 to 200 messages before everything sort of starts to collapse, but more sophisticated frameworks can take a character card like this and pull out an adventure worth hundreds and hundreds of messages, which is exactly how we got here. I do a little test run to see how well local models can leverage character cards made by the best of the best state-of-the-art models on the market like Fable 5. And my whole run ends up being nearly 700 messages long. I look I I really like RPing with bots. Okay. And when you're into it as much as I am, you start to notice some things. There was this moment where the protagonist encountered a stranger at the cabaret club. I prompted him multiple times for his name in character. He politely declined me every time. And I know that this has the potential vector for single thread declination through repeating patterns. And you know, no one likes it when their bots repeat the same word over and over again. So I do the only reasonable thing at this point, and I call a timeout for the system to generate a character card specific to this stranger. He finally gets a name, Elias Thorne, who for the better part of a hundred messages plays a minor antagonist that served to display the potential of man-made atrocities within the story line. So like exactly what he was supposed to do. Uh it it's all set up to serve as a comparison against the supernatural background. After my story with Elias moves on, I conclude the session with a few more hundred messages before the casino is turned into a glorified campfire thanks to Gideon.

[00:04:01]It's a satisfying personal experiment for a little choose your own adventure roleplay. And then a few days later, I'm sitting around browsing the news and and dude, I cannot describe how red my face got when I saw this headline. Why do chat bots keep telling stories about someone named Elias Thorne? The research behind it was incredibly solid, but it's no longer something you can fully replicate. There is just so much going on here, and we're going to need to establish what's actually happening in the creative space this year before we eventually get into our video about benchmarks because this story is going to play a pretty big role there as well.

[00:04:44]So, why did Elias show up in my RP session? Their experiments and your chats also these randomly generated books on Amazon and and more importantly, why is all of that going away? We should probably start by covering the research. So, here's the gist of it. Some Cornell folks, Hamilton and Mimno, whose names I hope I am not getting wrong, got tired of the anecdotes and put real numbers on this.

[00:05:14]They pulled 20,000 stories out of four different models with the most basic prompt you can give. Write a story. No steering, no setup, all go. The the result from it though was was just plain silly. There are 11 words. uh lighthouse keeper, a couple of professions. Uh and and then there's the names Elias, Mara, Allara, all of it shows up in 88% of everything. Not one model, all of them.

[00:05:46]Our friend Elias, the lighthouse keeper, 2third of every story generated. The system has a favorite guy, and it is not trying to be subtle about it. The obvious guess is that it's pariting its training data, that somewhere there's a mountain of lighthouse stories it's copying, except these dudes checked. And that's not it. Elias barely exists in actual books. It shows up about 900 more times in AI stories than in real fiction. So, the model isn't echoing something common. It's taking something incredibly rare and then cranking it to the front of the line, which weirdly points the finger at safety and alignment training, not the raw model itself. Now, the part I have to be upfront about because it's the whole reason that this video exists. You can't fix this from your chair. You can nudge it. You cannot cure it. A quick tangent.

[00:06:47]You've seen prompts where somebody types filter equals none. And someone who might not understand prompt engineering would reasonably go, "Well, you can't just turn off safety filters like that."

[00:06:59]And and well, I mean, they're not wrong.

[00:07:02]You can't. The model isn't reading that as a command that it's going to obey.

[00:07:07]It's just more text that helps influence it. Uh but it's not something that is definitive and hard-coded. the the same deal applies here. We can absolutely poke at this thing with settings and maybe even some clever prompting. And trust me, we're going to spend a good chunk of this video poking, but poke is the ceiling here. The narrowing lives inside the model itself. It's baked in during training, and there's no sentence that you or I can type that reaches in there and just magically undo it.

[00:07:39]Everything we're about to try helps a little bit, but none of it is going to fix this. Replicating this research wasn't as straightforward as you might think. We jumped in to test this across nearly every major front end across various models. And oftent times, yes, we would find Elias Thorne, but we'd also find Vance, Alla, Silus, Clara, so on and so forth. On the local side of things, you'll find the same behavior on everything from Gemma to Llama, Mistl to Granite. Elias Thorne is everywhere. And if he isn't the problem, it just goes by another name. There was one exception to this that made me immediately raise an eyebrow. Open AAI, which is not highly regarded for creative textbased tasks at the moment. Uh, but I I couldn't get Elias's name to show up or Thor, but I could see Vance and Ara still. Uh, this is a curious case of something I'm internally calling strawberry syndrome, where I suspect that it has become internal practice to respond to headlines with fine-tuning sessions.

[00:08:54]From a business perspective, if a story comes out that says Gemini tells users it's cool to eat multiple rocks a day and other services find that they can replicate that problem, well, you can just fine-tune your model to solve this individual issue. It means that when someone comes along and that state-of-the-art model lacks a specific viral issue, a company or entity gets to posture that they're somehow unique and immune to these systemic drops in quality like this sharp one with creativity. What is worse is that I tested this same problem a week later because I am terrible at reacting to these stories in a timely manner. H. Uh, anyways, when I retested it and found that there were even more companies who had fallen out of love with Elias Thorne, but were actively embracing Silus, Vance, and Ara, I knew there was a problem, and nobody had actually fixed the issue. They just moved the goalpost.

[00:09:52]And in doing so, they're not making anything better. Arguably, in some places, this is actually going to make things worse. The question of fixing this has come to my mind time and time again because most of us aren't out here building our own LLMs from scratch cuz that is really what it would take at this point. So instead, I pivot to prompting. Not because we can somehow beat this thing down with prompting alone, but we can certainly guide it towards some more favorable behavior. So we start by keeping our prompts small.

[00:10:27]They're applied to a single name generation test prompt. The first prompt we start with is adding a seed to the request to generate [music] 10 male and 10 female names. That produced some weird results, especially when we started to test random seed numbers applied to each of these uh generations.

[00:10:50]And um well uh we found this one seed that would increase the likelihood of seeing the surname Vance by more than uh 50%. And that that that's applied to any name across the whole board. As you can see, this tells us that we can fuss with things. Uh especially now that we can consistently produce some of these flexible but problematic tokens like Vance or Elias. So, the goal now with this seed is to see if I can actually tank the production of the name Vance without altering the seed itself.

[00:11:30]Because a significant drop means that we might be finding some door where we can start routing these to more statistically likely tokens. Easier said than done, even if an additional line is just use a representative sample. This is similar to telling the system not to rely on the first token that comes to mind for any given name as that token is likely a byproduct of safety and training for creative instances. I find it incredibly interesting that when layering on a simple addition of using a representative sample which implies base training data versus the statistical sample which implies the whole of training data fine-tuning and governance. We no longer saw the repetition of the Vance token. More so, we saw it reduced to a near normal rate of production. And that's that's a pretty big difference compared to where we started where some generations were [music] like 90% Vance surname. Do I know what any of this means? No. Uh, is it possible that we saw these bad token drops due to simply adding more context into the prompt? Maybe. Do I think that I could use this as a baseline prompt to encourage less generic responses? I don't know. That's going to take a few weeks and another 700 message role-playing session to see how steady the quality keeps through millions of tokens. But uh from some preliminary testing, I'm finding this makes a bigger impact on proprietary big tech than it does the compressed local models. Even if they're good or pretty loosey goosey on their guard rails, they're they're not trained out of this particular issue. And really, I guess that's where we're at with all of this. The creative issues with LLMs are baked into the state-of-the-art model, but some of them, like Fable 5's initial release, might be ditching some of the data sets that initially led to this kind of reward hacking drop in creativity. It's basically the Goblin Problem on steroids. It's way more quiet, though, and it's been going on for almost 3 years now. By comparison, the Goblin Problem took less than a year to fix.

[00:13:51]So, this is going to be way more complicated to fix than that, I think.

[00:13:56]But again, this goes back to something that I consistently advocate here, and that is splitting off fictional and factual models into two separate things.

[00:14:06]That way, we can go ahead and have these specifications without fallout like this. Okay, I am really starting to get off track now. Uh, remember at the start of this video where I talked about how surprised I was by Fable? Well, by the time you're seeing this, the model has been re-released for about 2 weeks now.

[00:14:26]Unfortunately, this release looks to be updated with additional layers that push back even Max thinking options into the statistical fine-tuning data versus that contextual representative sample.

[00:14:40]Honestly, just a splash of cold water that instantly killed my interest in this thing from the creative side. But hey, it gives me hope that we'll see another model that can hit similar creative strides without getting hammered by the guardrails. Until then, we're stuck with this data set. Uh fine-tuning local models with creative variants, prompting it to helen back and and none of that is actually going to fix the problem unless we're making the foundation itself. So, if I had to guess, Elias's successors will be here to stay for at least a while longer. And now for a totally smooth segue, let's talk about Patreon. [laughter] I I have been uh sitting on a lot of content for like the last 6 months. And uh the the short story behind that one is is I got super guilty putting content behind a payw wall. It sort of goes against the whole idea of this channel, which is to spread information without formal investment. So I uh I just took off the payw wall. No more guilt. And there's tons of content already prepared.

[00:15:48]Emojis, Osent experiments, uh, Grock's [music] infamous image generation issue, all all this stuff that might make YouTube a little squirly. And maybe I'll use it as a place to share more of this sort of creative and technical merge where I like use AI to support custom literary frameworks to develop long- form role- playinging sessions. I'm not too sure how interested people are in that kind of content, but it is stuff that I can get really excited about. Guess we'll see how that goes. Anyways, uh that is going to be it for me today. I've got some local LLM experiments to run with zero ulterior motives whatsoever. See you nerds.

[00:16:44]>> [music]