christos-antono/NPCTalk
christos-antono/NPCTalk has 232 GitHub stars and 22 forks. GitHot measured +1 stars in 24 hours and +4 in 7 days. Last checked 2026-11-16 23:52 UTC. Last push 2026-11-12.
Measured star growth
- in 24h
- +1
- in 7 days
- +4
- in 15 days
- Update delayed
- in 1 month
- Update delayed
Net star changes over overlapping periods, not daily totals. Missing measurements are left blank. Stars measure attention, not software quality.
About
Make any NPC in any game talk back with Fish Audio: skills, a self-hosted voice server and tools for AI coding agents, with a full GTA V example.
AIREADME.md
OriginalNPCTalk
By Universal Modder, the toolkit that lets AI agents mod almost any game you own · Join the Discord
Make any NPC in any game talk back, with Fish Audio voices.
Skills, a self-hosted voice server and tools that let an AI coding agent wire it up: the player speaks, the NPC
answers in character, in a cloned voice, about a second later, and acts on what was said.
Claude Code · Codex · Cursor · Gemini CLI · Copilot · OpenCode · how it was made · GTA V example · Discord · MIT
Built first for GTA V: walk up to anyone in Los Santos and just talk. Lamar roasts your hoodie, a cop tells you to file a civil claim and flips you off, Michael defends his robe like it's a business suit. Every line is written live in character and spoken in that character's cloned voice by Fish Audio Drama 3 (or S2 Pro), self-hosted on a serverless GPU. Most of the code doesn't know it's in GTA, so the same pieces work in other games: the skills say what a game has to provide, and how to build the rest.
Install
Pick your agent. Each gets the same skills (Agent Skills format), the fal MCP server and the npctalk CLI.
| Agent | Install |
|---|---|
| Claude Code | /plugin marketplace add christos-antono/NPCTalk/plugin install npctalk@npctalk |
| Codex | codex plugin marketplace add christos-antono/NPCTalkcodex plugin add npctalk@npctalk |
| Gemini CLI | gemini extensions install https://github.com/christos-antono/NPCTalk |
| VS Code / Copilot | Enable chat.plugins.enabled, run Chat: Install Plugin From Source, enter this repo's URL |
| Cursor | Clone it (Cursor reads AGENTS.md and .cursor/mcp.json) |
| OpenCode | Clone and run opencode inside it (opencode.json adds the skills and the fal MCP server) |
| Skills only (any agent) | npx skills add https://github.com/christos-antono/NPCTalk |
| Anything else | git clone https://github.com/christos-antono/NPCTalk and start your agent inside it |
Inside a clone, .agents/skills and .claude/skills are copies of skills/, and AGENTS.md holds the
instructions (CLAUDE.md and GEMINI.md point to it).
The npctalk CLI. Plugin installs and clones put it on PATH (bin/). Anywhere else:
uv tool install git+https://github.com/christos-antono/NPCTalk.
Accounts and keys (environment variables, never files):
| What for | Service | Variable |
|---|---|---|
| The voice server (GPU) | Modal | modal setup; NPCVOICE_TOKEN = the value you put in the npcvoice-auth secret |
| Speech-to-text, MiniMax voices, the MCP server | fal | FAL_KEY |
| The NPC's lines (any OpenAI-compatible LLM) | Cerebras | CEREBRAS_API_KEY |
| Actions and spoken orders | TypeSafe | TYPESAFE_API_KEY |
You also need Git, Python 3.10+, uv and ffmpeg. The GTA V example needs Windows, the .NET SDK (8+) to build it,
and the game.
Try it
Make the villagers in my game answer when I talk to them, in their own voices.
Set up a Fish Audio Drama 3 voice server on Modal and clone a voice from
narrator.wav.
In GTA V story mode, let me talk to anyone near me with my mic.
My NPC's cloned voice sounds thin. Is it the reference clip or the model? Measure it.
Record five 8-second clips of NPCs arguing that look like normal gameplay.
The agent starts with the make-npcs-talk skill and runs the same loop every time:
- recon the game: version, how code runs in it, single-player or not;
- pick the engines (voice, LLM, speech-to-text, actions);
- stand up the voice server first, outside the game, and test it with
npctalk say; - make reference voices from recordings you have the rights to;
- one NPC answering typed text in the game: the four hooks, nothing else;
- voice input (push-to-talk, then the open mic);
- characters: personas, their own lines, scene facts;
- actions, and doing what they're told;
- verify in the real game, with every step's timing logged;
- show it, and measure the voices before claiming anything;
- write a field note for the next agent.
What's inside
Skills (skills/, Agent Skills format)
| Skill | What it does |
|---|---|
make-npcs-talk |
The whole loop, the four hooks a game must provide (who's talked to, scene facts, voice at the NPC, actions), engine choices, latency budget, costs, verification, rules; per-engine notes for GTA V, Unity, Unreal, Bethesda, Minecraft and your own game |
fish-voice-server |
Host Fish Audio Drama 3 or S2 Pro on Modal (or fal) behind one streaming API; settings that measurably help likeness; best-of-N for pre-rendered lines; keep-warm and cost; the S2 Pro fine-tune; every gotcha |
voice-references |
From a recording you have the rights to, to clean reference clips, a 25 s clone clip and mood clips, on the server |
npc-dialogue |
Personas, a fresh sample of the character's own lines in every prompt, scene facts, short spoken lines with delivery tags, output cleaning, content rules, speed |
npc-actions |
What NPCs physically do while they talk, and following spoken orders, with TypeSafe Jev; per-action thresholds; mapping to animations |
voice-input |
Open mic with voice activity detection, push-to-talk, speech-to-text, the NPC's own voice and the room's noise |
voice-likeness-eval |
Measure whether a voice change really helped: speaker similarity to held-out real clips, word error, enough runs, independent checks |
npc-showcase-clips |
Gameplay-style clips of conversations: fixed gameplay camera, lines rendered first, sync, 5-10 s cuts |
The npctalk CLI (Python). Every command has --help with examples.
| Command | What it does |
|---|---|
npctalk health |
Is the voice server up? Waits through a cold start |
npctalk say |
Speak a line in a voice, like a game does; prints time to first audio, writes a WAV |
npctalk voice make |
Recording → voices/<Name>/clone.wav, ref_0..2, mood clips, with transcripts |
npctalk voice upload |
A voice folder → the server's Modal volume |
npctalk check |
Blocks what must never ship: audio, transcripts, keys, real server URLs |
The voice server (server/): drama3.py (Fish Audio Drama 3, the default), s2pro.py (S2 Pro, with
fine-tuned voices), keepwarm.py, fal/ (S2 Pro on fal over a realtime WebSocket), finetune/ (dataset, LoRA,
and the scorer behind every number here).
The GTA V example (examples/gta-v): the full mod (open mic, any NPC, 19 characters, homes,
actions, dev mode, a director for recording clips), its tools, and a
field note.
Field notes (knowledge/): one per game where NPCs were made to talk. Add yours.
How good are the voices?
Measured on one character (Lamar, ten test lines): voice match is speaker similarity to 60 real clips that were
never used as references. Details and the method: skills/voice-likeness-eval.
| Setup | Voice match | Word error |
|---|---|---|
| ElevenLabs instant clone | 0.612 | 10.5% |
| Fish Audio Drama 3, default settings | 0.643 | 7.1% |
| Drama 3, tuned sampling, one take | 0.660 | 7.9% |
| Drama 3, best of 8 takes, checked | 0.704 | 8.6% |
| Fish Audio S2 Pro, its best reference clip | 0.707 | 5.6% |
| S2 Pro fine-tuned on the character (needs training) | 0.785 | 7.5% |
Without training, Drama 3 and S2 Pro tie on likeness; Drama 3 acts [crying] and [laughing] better and starts
talking faster. One early result here didn't survive a second run:
how that was caught.
Rules it follows
- Single-player games, or servers you run. It never injects into online games with anti-cheat (GTA V: never launch GTA Online with a script hook installed) and never bypasses anti-cheat, DRM or ownership checks.
- Voices you have the rights to. It never commits or publishes voice clips, cloned voices, game audio or a
game's dialogue;
npctalk checkblocks them. - No secrets in files. Keys come from environment variables, and voice server URLs stay private (their
/healthwakes the GPU). - It asks before driving your keyboard and mouse in a game, installing into game folders, keeping a GPU up, or publishing anything.
- It measures before it claims a voice setting is better.
Full reasoning: skills/make-npcs-talk/references/safety.md.
Community
NPCTalk is a Universal Modder project. Universal Modder's skills and tools cover the rest of a mod (engine recon, decompiling, fal-generated art, 3D and audio, in-game testing, showcase videos), and NPCTalk adds NPCs that talk back. Install both and an agent can take a game from "can I mod this?" to characters you can talk to.
Show what you made, ask for help, and follow along in the Universal Modder Discord.
Credits
- Voices: Fish Audio (Drama 3, S2 Pro, fish-speech), served with SGLang-Omni on Modal and fal.
- Lines: Cerebras. Speech-to-text: NVIDIA Nemotron ASR on fal. Actions: TypeSafe Jev.
- GTA V: Script Hook V, ScriptHookVDotNet, NAudio.
- A Universal Modder project; the repo layout and principles come from universal-modder.
- Built in real sessions with an AI coding agent (Claude Code), from the first line of the mod to the measurements.
MIT licensed. GTA V and its characters belong to Rockstar Games; this project isn't affiliated with Rockstar, Fish Audio or any service above.