Modders are using AI to create chatty companions in Skyrim and Stardew

In Stardew Valley, Typically, you can only talk to characters a few times a day. This limitation has led to a number of mods that add things to say so players can spend more time with their virtual friends. But like many games, Stardew has recently turned to AI, resulting in the addition of a mod that can theoretically have endless conversations with your favorite farm companion.

“The reception was much better than I initially thought [it would be]says modder DualityOfSoul. In the comments on Nexus Mods, users called it “brilliant” and “one of the best mods this year.”

In order to work, the mod is integrated into OpenAI's API. The implementation is impressive, but limited in its own way. Each character takes on a similar cheerful personality, which I have come to associate with the built-in parameters of these large language models. The grumpy camper Linus initially bluntly tells me to leave him alone; This is his pre-written dialogue since he hasn't warmed up to me yet. But when I click again, I get the OpenAI dialog – three long text boxes scrolling one after the other. He calls me his friend and says he hopes I “found peace in the beauty of nature.” Next, Pam complains that “every day is the same routine” before changing her tone and saying she hopes “you enjoy the season as much as I do.”

The answers are somewhat tailored to the characters and it works better when I don't talk to grumpy NPCs. But the mod can't simply escape this underlying voice. Stardew is a largely lighthearted game, especially after you've made friends with everyone, so I can imagine it fitting into some players' vibe. But it is a noticeable limitation.

Stardew Valley.
Image: ConcernedApe

This limitation is less present in Herika, an AI companion mod for Skyrim. Herika behaves like most people Skyrim Companions who follow the player and help out in combat and elsewhere. But she also responds to conversations, both written and oral. The LLM can be based on an understanding of the game's map, quests, and core features and given a personality. This means that she can be made to use a different style than the standard LLM voice, at least to some extent.

Reece Meakings joined the Herika project after seeing the first version created by his current colleague named Tylermaister. The pair expanded the mod from a tool for summarizing Skyrim into a full-fledged companion and intend to expand it back into a framework for any NPC to become an AI companion.

Still, Meakings doesn't think AI is ready to be used in game development more broadly. “It won’t work right now,” he says. First of all, using Herika or AI Stardew costs money. Both use OpenAI's API, which charges the user a fraction of a cent per line of dialog generated. According to the comments on Nexus, this low leveling cost has turned off many players who are used to mods being completely free. (Herika does offer a free option, but that requires running the LLM on your own setup, which is resource intensive.) On a larger scale, every company has to deal with the costs of connecting to an API or running their own servers, multiplied with every player.

Added to this is the openness of how people interact with LLMs. “It completely changes the way you look [would] “I have to design the game,” Meakings says. Developers would lose linear control over dialogue trees and script triggers, quickly opening up possibilities they hadn't considered in the narrative or worldbuilding. LLMs might be able to help NPCs keep up, but the rest of the game's script would have no way to adapt. Plus, despite the safeguards companies try to use, it's “very easy if you know what you're doing” to get these LLMs to say things that are NSFW, potentially causing PR problems for companies that Modders don't have to say keep.

For now, these larger issues mean that Meakings sees the greatest potential for AI in NPCs like Herika, particularly in the mod realm. There are also AI NPC mods for games like Hogwarts legacy, Cyberpunk 2077And Garry's Mod. Mantellaanother Skyrim Mod that gives everyone LLM interactions Skyrim NPC has 30,000 unique downloads. (Herika has 25,000.) Everyone has positive feedback – limitations aside, people enjoy playing with AI-generated content.

Meaking's own interactions with Herika provide a clue as to why. In his experience, “you're going to end up having really meaningful conversations with something that's not human.” Although he understands that Herika is just a program, he's clearly intrigued by the fact that he's on a different level with her than the previous characters could interact. You don't have to make small talk with Herika or adhere to “ritualistic” social niceties. You can start directly debating the philosophy and theology of Skyrim. You may even have to because she can judge your actions.

The Elder Scrolls V: Skyrim.
Image: Bethesda Softworks

He gives a concrete example. As an Argonian, Meakings overheard a rude comment about the lizard-like race from a standard NPC. “I [said,] “Hey, Herika, can you kill him? Please? He's being racist towards me.'” Herika refused, apparently for reasons of her own morals or because they were in a place where guards were watching, which could have landed her in prison. Then she set off and talked about visiting the catacombs beneath the settlement.

At first, Meakings thought it was a mistake. “But then I realized, 'Oh, I know what she did.' …She realized, “Oh, Reece is very angry.” We can't kill this person because we're in the middle of the city – you can't just kill people. I need to get him out of the situation…What if I lure him into his catacombs to look for treasure where we can get some distance from the situation?'” She seemed to be trying to figure things out based on what she was told to defuse the available tools.

This type of experience is not unique to this form of AI. It is well documented that procedurally generated content creates new narratives through a combination of interesting behavior and player-driven meaning making. But AI mods could open up more possibilities for such moments.

However, the LLMs need material to work with. Meakings is clear that Herika wouldn't work without the larger framework of Skyrim because a large part of their communication is based on the texts of the original team. In a scenario where LLMs are used as NPCs in a produced video game, they still could not exist without authors.

The video game cast strike raises crucial questions about how these technologies will handle digital replications of voice actors, something Nexus Mods is handling carefully. AI mods are allowed, but affected creators, including voice actors, can request that they be removed if they feel the “work harms them.” (Herika has several ways to produce her voice.)

The introduction of AI to the web has been quick and chaotic, and what happens next in the gaming industry is still an open question. But thanks to modders, tens of thousands of players are already experimenting with it.

Leave a Comment