StoryDaemon

Agentic Novel Generation System with Emergent Narrative

StoryDaemon is a Python-based system that generates long-form fiction through an autonomous agent that plans, writes, and evolves stories organically. Unlike my previous story generator, StoryDaemon emphasizes emergence over pre-planning, allowing narrative structure to develop naturally through iterative "story ticks."

YouTube Explainer

An introduction and overview.

• StoryDaemon: The Future of Story Generation?

Features

• 🤖 Agentic Architecture - LLM-driven agent makes autonomous decisions using structured tools
• 📖 Deep POV Writing - Strict point-of-view discipline for immersive narrative
• 🧠 Evolving Memory - Characters, locations, and story elements dynamically update
• 🎯 Emergent Structure - No pre-outlining; story develops organically
• 📚 Story Foundation - Optional genre, premise, setting, tone to guide emergence
• 🎯 Goal Hierarchy - Protagonist goals emerge naturally or can be user-specified
• 📋 Plot-First Mode - Optional emergent plot beats guide scene generation for forward momentum
• ⚡ Tension Tracking - LLM-rated scene tension scoring (0-10) for pacing awareness
• 🦴 Scene Skeletons (experimental, off by default) - typed paragraph plans sampled from a block grammar measured on 21 classic novels guide the writer toward master-like prose structure, with per-paragraph compliance tracking
• 🎚️ Coherence Pressures - Arc-tension targeting, a throughline gate, and contradiction detection keep emergent prose on canon and on arc (see the Emergent Coherence roadmap)
• 💰 Flexible LLM Backends - Codex CLI, Gemini CLI, Claude Code CLI (zero additional cost), or API backends (GPT-5.x, Claude Fable 5 / Opus 4.8 / Sonnet / Haiku, Gemini 3.x and 2.5, a self-hosted endpoint, OpenRouter, or Venice), shared with the sibling llm-backends package
• 🔧 Tool-Based System - Extensible tool registry for character generation, memory search, etc.
• 🔍 Rich Inspection Tools - Status, list, inspect, goals commands for full project visibility
• 💾 Automatic Checkpointing - Snapshot and restore project state at any point
• 📝 Manuscript Compilation - Export to Markdown, HTML, prose, EPUB, or PDF with scene filtering
• 🎲 Unique Name Generation - Syllable-based name generator with 4M+ combinations
• 🔄 Resume Workflow - Easily continue recent projects with novel resume
• 🆔 UUID Safety - Automatic project IDs prevent accidental overwrites

Quick Start

Prerequisites

• Python 3.11+
• Codex CLI installed and authenticated (default backend)

  npm install -g @openai/codex-cli
  codex auth
  

• (Optional) API access for OpenAI / Claude / Gemini backends
  • OpenAI GPT-5.5 (default) / 5.4 / 5.4-mini / 5.2
  • Anthropic Claude Fable 5 / Opus 4.8 / Sonnet 4.6 / Sonnet 4.5 / Haiku 4.5
  • Google Gemini 3.1 or 3 (flash/pro preview) or 2.5 (pro/flash)
  • Self-hosted, OpenAI-compatible endpoint via HOSTED_LLM_URL, HOSTED_LLM_PORT, HOSTED_LLM_API_KEY, HOSTED_LLM_MODEL env variables (hosted-llm)
  • OpenRouter (https://openrouter.ai), a hosted OpenAI-compatible router over many models, via OPENROUTER_API_KEY and OPENROUTER_MODEL env variables (openrouter)
  • Venice (https://venice.ai), an OpenAI-compatible host of open-weight models including uncensored variants, via VENICE_API_KEY and VENICE_MODEL env variables (venice)
• (Optional) Gemini CLI installed (for llm.backend = gemini-cli)
  • https://github.com/google-gemini/gemini-cli
• (Optional) Claude Code CLI installed (for llm.backend = claude-cli)
  • https://github.com/anthropics/claude-code

Installation

# Clone the repository
git clone https://github.com/EdwardAThomson/StoryDaemon.git
cd StoryDaemon

# (Recommended) Create and activate a virtual environment
python3 -m venv venv
source venv/bin/activate  # On Windows use: venv\\Scripts\\activate

# Install dependencies into the virtual environment
pip install -e .

# Or install with development dependencies
pip install -e ".[dev]"

# For PDF export, add the optional export extra (pulls in WeasyPrint, which
# also needs the Pango/Cairo system libraries; pandoc is used automatically if
# present). EPUB export works out of the box.
pip install -e ".[export]"

Create Your First Novel

# Recommended: create a new novel project with interactive story foundation
novel new my-story --dir work/novels
#  Creates: work/novels/my-story_a1b2c3d4/
#  You will be prompted for genre, premise, protagonist, setting, tone, etc.,
#  and to choose the LLM backend/model to store in this project's config.
#  The protagonist's name and archetype are asked separately: the name is a
#  field, not something parsed back out of the archetype text.

# Advanced: skip the interactive wizard (no story foundation)
novel new my-story --dir work/novels --no-interactive

# Or create from a YAML foundation file (non-interactive)
novel new my-story --foundation foundation.yaml

# Or via command-line arguments (non-interactive)
novel new my-story \
  --genre "science fiction" \
  --premise "A lone engineer discovers an alien signal" \
  --protagonist "Curious, isolated technical expert" \
  --setting "Near-future Mars colony" \
  --tone "Contemplative, mysterious"
#  Note: --protagonist is the *archetype*, not the name. Every project gets a
#  protagonist character (C000, set as the active character) minted in Python at
#  creation time — from the name you gave, or generated when you gave none —
#  rather than waiting for the planner to call character.generate on tick 0.

# Generate your first scene
cd work/novels/my-story_a1b2c3d4
novel tick

# Or generate from anywhere
novel tick --project work/novels/my-story_a1b2c3d4

# Check project status (includes story foundation)
novel status

# View goal hierarchy
novel goals

# Generate multiple scenes with automatic checkpointing
novel run --n 10 --checkpoint-interval 10

# See recent projects (shows UUIDs)
novel recent

# Resume most recent project (no need to type full path!)
novel resume

# Resume specific project by UUID
novel resume --uuid a1b2

# Resume and run multiple ticks
novel resume --n 5

# List what was created (includes tension levels!)
novel list characters
novel list locations
novel list scenes  # Shows tension: 6/10 (rising), 5/10 (rising), etc.

# Inspect a character
novel inspect --id C000

# Compile a manuscript
novel compile --output draft.md

# Save prompts for inspection (Phase 7A.5)
novel tick --save-prompts  # Saves to prompts/ directory

# View lore items (Phase 7A.4)
novel lore list
novel lore list --category world_rules
novel lore list --importance high

# Plot-first mode commands
novel plot generate --count 5    # Generate plot beats
novel plot status --detailed     # View beat status
novel plot next                  # See next pending beat
novel plot revise                # Abandon pending beats and regenerate from current canon
novel plot clear                 # Clear all beats (with confirmation)

# Use API backend instead of Codex
novel tick --llm-backend api --llm-model gpt-5.5          # OpenAI GPT-5.5 (default)
novel tick --llm-backend api --llm-model claude-sonnet-4-5  # Anthropic Claude Sonnet 4.5
novel tick --llm-backend api --llm-model gemini-3-pro-preview  # Gemini 3 Pro
novel tick --llm-backend api --llm-model hosted-llm      # Self-hosted OpenAI-compatible endpoint
novel tick --llm-backend api --llm-model openrouter      # OpenRouter (routes to whatever OPENROUTER_MODEL names)
novel tick --llm-backend api --llm-model venice          # Venice (uses whatever VENICE_MODEL names, e.g. venice-uncensored)

# Use Gemini CLI backend (local `gemini` binary)
novel tick --llm-backend gemini-cli --llm-model gemini-3-flash-preview

# Use Claude Code CLI backend (local `claude` binary, headless mode)
novel tick --llm-backend claude-cli --llm-model claude-sonnet-4-5

How It Works

StoryDaemon uses a story tick loop where each tick produces one scene. The layering principle: Python owns where the story should be, the LLM owns how it gets there. LLM calls propose; Python validates, judges, and persists.

Architecture

flowchart TD
    subgraph PLAN["Plan"]
        P0["0.5 Construction-pressure detector<br/><i>instrument only</i>"]
        P1{"Plot-first mode?"}
        P1a["Beat selection<br/>regenerate beats if low · next pending beat<br/><i>finale tick: sacred finale beat</i>"]
        P2["1. Story context<br/>synopsis · recent scenes · rosters with exact IDs<br/>· relationships · beat"]
        P3["2. Multi-stage planner<br/>intention → semantic search → tactical plan"]
        P4["3–4. Validate + execute tools<br/>name / character / location / faction /<br/>memory / language"]
        P0 --> P1
        P1 -- yes --> P1a --> P2
        P1 -- no --> P2
        P2 --> P3 --> P4
    end

    subgraph WRITE["Write"]
        W6["6. Writer context<br/>beat + postconditions · Established Canon ·<br/>synopsis · scene skeleton (opt.)"]
        W6b["Scene writer<br/>write until concluded · sectioned writing (opt.)"]
        W7["7. Scene evaluator<br/>continuity / POV gate"]
        W75["7.5 Tension scorer (0–10)"]
        W76["7.6 Tension control<br/>partial paragraph revision ·<br/><i>finale: fresh re-rolls</i>"]
        W6 --> W6b --> W7 --> W75 --> W76
    end

    subgraph COMMIT["Commit and extract"]
        C8["8. Commit scene<br/>summary · vector index · tension · new-character detection"]
        C9["9. Fact extractor<br/>facts · open loops · status-change claims"]
        C10["10. Entity updates"]
        C105["10.5 Judged deaths and destruction<br/>evidence check → one-claim judge → entity status"]
        C11["11. Re-index updated entities"]
        C112["11.2 Story synopsis<br/>close arc every N scenes → fold into running synopsis"]
        C8 --> C9 --> C10 --> C105 --> C11 --> C112
    end

    subgraph VERIFY["Verify and account"]
        V115["11.5 Beat verification<br/>planner claim / semantic score / contracts"]
        V116["11.6 Judged loop closure<br/><i>finale: expire remaining loops</i>"]
        V118["11.8 Thread registry"]
        V12["12. Lore extraction + contradiction detection"]
        V13["13. Goal promotion (ticks 10–15)"]
        V14["14. Save state + metrics.jsonl"]
        V115 --> V116 --> V118 --> V12 --> V13 --> V14
    end

    P4 --> W6
    W76 --> C8
    C112 --> V115

    MEM[("Project memory<br/>characters · locations · scenes · lore ·<br/>open loops · threads · synopsis ·<br/>entity status · vector index")]
    MEM -.-> P2
    MEM -.-> W6
    C10 -.-> MEM
    C105 -.-> MEM
    C112 -.-> MEM
    V12 -.-> MEM
    V14 -. next tick .-> P0

Tick 0 takes a separate two-phase path that creates entities before writing, so the first scene sees real IDs. Every step after the scene is written is wrapped so that a failure in an extractor, judge, or metric never loses the scene.

The steps

1. Select a beat (plot-first mode) - Regenerate beats when few are pending, then take the next one. On the final tick, Python owns the ask and supplies a finale beat.
2. Build story context - The planner sees a running synopsis of the whole story, the most recent scenes in order, character and location rosters with exact IDs (dead, missing, and destroyed entities marked), and relationships.
3. Plan - The multi-stage planner forms a strategic intention, gathers semantic context from the vector index, and writes a tactical plan of tool calls.
4. Execute tools - Names, characters, locations, and factions are minted in Python and selected by the LLM, never invented by it.
5. Write - The writer gets the plan, the beat and its postconditions, an Established Canon section (relevant lore, open loops, and every dead or destroyed entity), and optionally a paragraph-level scene skeleton. It writes until the scene reaches a real ending.
6. Evaluate - Check continuity and POV integrity; only concrete issues fail a scene.
7. Score and control tension - Rate tension 0-10 against an anchored rubric, then make one bounded correction toward the arc's target (revising only the tension-carrying paragraphs).
8. Commit - Save the scene, its summary, and its tension, and index it.
9. Extract facts - Update characters and locations, and open or resolve story loops.
10. Judge deaths and destruction - A claimed death, disappearance, or destroyed location must quote evidence from the scene and pass a one-claim judge before it is recorded. After that, the entity leaves rosters and beats, and the writer is told.
11. Update the synopsis - Every few scenes, close an arc summary and fold it into a running synopsis whose length stays flat however long the novel runs.
12. Verify the beat - Check the scene accomplished its beat, by planner claim, semantic similarity, or contract postconditions.
13. Close loops and attribute threads - Each claimed loop resolution is confirmed by a focused judge; the scene joins its thread's trace.
14. Extract lore - Identify world rules and capabilities, and detect contradictions.
15. Promote goals and record metrics - Promote a story goal when conditions are met, save state, and append the tick's coherence metrics.

Real-World Example

Here's what tension tracking looks like in a generated story:

$ novel list scenes

 Scenes (5 total)

  file          word_count  pov_character  tension_level
  ------------  ----------  -------------  -----------------
  scene_000.md  451         Jynyn          None
  scene_001.md  281         Jynyn          6/10 (rising)
  scene_002.md  340         Jynyn          5/10 (rising)
  scene_003.md  863         Jynyn          6/10 (rising)
  scene_004.md  518         Jynyn          5/10 (rising)

Scene 1 (6/10 - rising): Discovery of mysterious conduit under ice shelf
Scene 2 (5/10 - rising): Descending into the structure
Scene 3 (6/10 - rising): Examining ancient machinery
Scene 4 (5/10 - rising): Uncovering hidden messages

The tension naturally oscillates between 5-6/10, maintaining engagement without exhaustion - perfect for a mystery/investigation narrative!

Memory System

Each novel maintains its own working directory with:

• Story Foundation - Optional immutable constraints (genre, premise, protagonist, setting, tone, themes, primary goal)
• Plot Outline - Optional emergent plot beats for forward momentum (plot-first mode)
• Characters - Dynamic character data with goals, relationships, emotional state
• Locations - Evolving location descriptions with sensory details
• Scenes - Scene metadata and summaries
• Story Synopsis - Arc summaries folded into one running synopsis of the story so far
• Entity Status - Judged deaths, disappearances, and destroyed locations, with the evidence that confirmed them
• Open Loops - Unresolved narrative threads with mention tracking
• Goal Hierarchy - Protagonist immediate/arc/story goals with progress tracking
• Lore - World rules, constraints, and capabilities with contradiction detection
• Story Threads - Registry of persistent narrative strands each beat serves (novel threads)
• Vector Index - Semantic search for context retrieval

Project Structure

StoryDaemon/
├── novel_agent/          # Python package (code)
│   ├── agent/           # Agent runtime and orchestration
│   │   ├── agent.py              # StoryAgent: the tick loop
│   │   ├── context.py            # Planner context builder
│   │   ├── story_context.py      # Story-so-far context for planner and writer
│   │   ├── multi_stage_planner.py # Three-stage planner (default)
│   │   ├── plan_manager.py       # Plan/error snapshots
│   │   ├── schemas.py            # Plan validation schema
│   │   ├── writer_context.py     # Writer context (beat, canon, synopsis)
│   │   ├── writer.py             # Scene prose generator
│   │   ├── segments.py           # Write-until-concluded + sectioned writing
│   │   ├── scene_skeleton.py     # Masters-grammar paragraph plans (Slice 4)
│   │   ├── evaluator.py          # Scene quality gate
│   │   ├── tension_evaluator.py  # 0-10 tension scorer
│   │   ├── arc_pressure.py       # Target tension curve and arc phases
│   │   ├── partial_revision.py   # Revise only tension-carrying paragraphs
│   │   ├── scene_committer.py    # Scene persistence
│   │   ├── fact_extractor.py     # Facts, open loops, status-change claims
│   │   ├── entity_updater.py     # Applies facts to characters/locations
│   │   ├── consequences.py       # Judged deaths and destruction
│   │   ├── character_detector.py # New named entities in prose
│   │   ├── loop_closure.py       # Judged open-loop closure
│   │   ├── loop_aging.py         # Loop-age gauge and horizons
│   │   ├── lore_extractor.py     # World rules from prose
│   │   ├── lore_contradiction_detector.py
│   │   ├── finale.py             # Sacred finale
│   │   ├── thread_registry.py    # Story-thread registry
│   │   ├── thread_construction.py # Construction-pressure detector
│   │   ├── throughline.py        # Goal-relevance judge
│   │   ├── coherence_metrics.py  # Per-tick metrics.jsonl
│   │   ├── language_situation.py # Who understands whom in a scene
│   │   └── prompts.py            # LLM prompt templates
│   ├── tools/           # Planner tools and LLM backend shims
│   │   ├── name_generator.py   # Python-minted names (name.generate)
│   │   ├── language_tools.py   # language.phrase / language.word
│   │   ├── memory_tools.py     # Entity generation and memory search
│   │   ├── llm_setup.py        # Backend initialization (llm-backends)
│   │   └── ...                 # Compatibility shims over llm-backends
│   ├── memory/          # Memory management
│   │   ├── manager.py          # MemoryManager
│   │   ├── entities.py         # Entity dataclasses
│   │   ├── entity_resolver.py  # Resolves references to canonical IDs
│   │   ├── entity_status.py    # Dead/missing/destroyed status
│   │   ├── synopsis.py         # Arc summaries and running synopsis
│   │   ├── summarizer.py       # Per-scene summaries
│   │   ├── vector_store.py     # ChromaDB integration
│   │   └── checkpoint.py       # Checkpoint system
│   ├── plot/            # Plot management (emergent plotting)
│   │   ├── manager.py          # PlotOutlineManager
│   │   ├── dedup.py            # Beat-level dedup at authoring time
│   │   └── entities.py         # PlotBeat, PlotOutline
│   ├── contracts/       # Beat contracts (closed checker vocabulary, Slice 1)
│   ├── cli/             # Command-line interface
│   │   ├── main.py             # CLI entry point
│   │   ├── project.py          # Project management
│   │   ├── foundation.py       # Story foundation setup
│   │   ├── protagonist.py      # Mints the protagonist at project creation
│   │   ├── recent_projects.py  # Recent projects tracker
│   │   └── commands/           # One module per command (status, goals,
│   │                           # lore, loops, metrics, threads, plot, list,
│   │                           # inspect, plan, compile, checkpoint, titles)
│   ├── export/          # Manuscript writers (EPUB, PDF)
│   ├── world_import.py  # Seed a project from a worldgen world
│   ├── languages_import.py # Import a world's constructed languages
│   ├── configs/         # Configuration
│   ├── data/            # Static data files
│   │   ├── names/       # Name generation syllables and titles
│   │   └── block_grammar_v1.json  # Masters block grammar (scene skeletons)
│   └── utils/           # Utilities
├── work/                # Development working directory (gitignored)
│   ├── recent_projects.json  # Recent projects tracker
│   ├── novels/          # Test novels with UUID suffixes
│   │   └── my-story_a1b2c3d4/
│   │       ├── memory/      # Entity storage
│   │       ├── scenes/      # Generated scene markdown files
│   │       ├── plans/       # Plan JSON files
│   │       ├── plot_outline.json  # Plot beats (if using plot-first mode)
│   │       ├── checkpoints/ # Project snapshots
│   │       ├── .git/        # Version history: one commit per tick (novel history)
│   │       └── errors/      # Error logs
│   ├── experiments/     # Quick experiments
│   └── scratch/         # Temporary files
├── scripts/             # Analysis and maintenance scripts (corpus tables, repairs)
├── experiments/         # Standalone research experiments (block grammar PoC)
├── tests/               # Test suite
└── docs/                # Documentation
    ├── spec.md                      # Technical specification
    ├── plan.md                      # Legacy implementation plan
    ├── EMERGENT_COHERENCE_PLAN.md   # Active roadmap
    ├── PLOT_FIRST_MODE_GUIDE.md     # Plot-first user guide
    ├── progress_report_*.md         # Dated run reports
    └── archive/                     # Superseded phase documentation

CLI Commands

Core Generation Commands

# Create new novel project (interactive foundation by default, UUID suffix)
novel new <name> [--dir <path>]
# Example: novel new my-story  creates my-story_a1b2c3d4/ and runs the foundation wizard

# Disable interactive wizard (no story foundation; a protagonist is still minted)
novel new <name> --no-interactive [--dir <path>]

# Create with story foundation (non-interactive variants)
novel new <name> --foundation <yaml_file>         # From YAML file
novel new <name> --genre <genre> --premise <text> # Command-line args

# Start from a generated world (see "Starting from a worldgen world" below)
novel new <name> --world <world.json> --no-interactive
novel new <name> --world <world.json> --languages <languages.json> --no-interactive

# Generate one scene
novel tick [--project <path>]

# Generate multiple scenes with automatic checkpointing
novel run --n <count> [--checkpoint-interval 10] [--project <path>]

# Compile scene summaries
novel summarize [--project <path>]

Starting from a worldgen world

--world <path> seeds a new project from a JSON world written by the world-building generator (schema worldgen/[email protected], an SF star sector, or worldgen/[email protected], a fantasy region). No LLM is involved in the import; the file is mapped in Python (novel_agent/world_import.py) and persisted through the normal MemoryManager/VectorStore path, so the planner sees the world exactly as it sees entities created during ticks:

• Region: each realm becomes a faction (government form, ruler, succession and gender law, population and area, its magic tradition); every city and the largest towns above a population floor become locations (population, realm, biome, river and coast); the magic system, the calendar, each named mountain range and river, and each story hook become lore.
• Sector: each faction becomes a faction (type, tech level, capital, population, fleets, its psionic tradition); every capital's main world and every temperate world become locations (star, gravity, temperature, day and year, plant colour, who holds it); psionics, the FTL model, each chokepoint lane and each story hook become lore.

Ranges and rivers are lore rather than locations because a StoryDaemon location is a stage a scene is set on; a 300 km river is something a scene refers to. Hooks are lore tagged hook: the planner can draw on them, but none of them is promoted to the premise.

The foundation's genre and setting default to the world's when you do not give them, so --world region.json --no-interactive alone yields a project that can tick; --genre, --setting, --premise, --protagonist, --tone and the interactive wizard all still win over the world's text. An unknown schema or a missing file is refused before any project directory is created. The source path and schema are recorded under world in state.json. How many towns or worlds are kept is set by world.max_towns, world.town_population_min and world.max_worlds in the config.

Languages. World-building can also write every people's constructed language (npm run cli -- region --seed demo --languages-json languages.json, schema worldgen/[email protected]). Pass it with --languages <path>; when the flag is absent and <world stem>-languages.json sits beside the world file, it is used and the command says so. The file is keyed by the world's realm or faction ids, so one whose peoples do not match the world is refused before anything is created (--languages without --world likewise). Still no LLM: novel_agent/languages_import.py reduces each language to

• a notes block under world.languages in state.json (language name, family and relatives, word order in a phrase, a dozen everyday words), plus one sentence on who speaks what appended to the world's setting text;
• a bank per people in memory/languages.json: the file's meaningful place and person names, its whole lexicon, and its phrases array when the export carries one (greetings, farewells, thanks, oaths, curses, toasts, proverbs, questions). name.generate, character.generate and location.generate take an optional affiliation (a realm or faction by name) and draw from that people's own tongue; the people is recorded on the character.

No lore is written for a language. The first version wrote one lore item per tongue and the fact extractor fed it back into every scene as a paragraph about the language; a language is texture in the prose, not a fact the reader is told. So the words reach the writer through two tools, not facts, registered only when languages were imported:

• language.phrase(people, kind): one set phrase of that kind from the people's phrasebook, with its gloss and English. Without a phrasebook it answers with the single nearest word and says so (two lexicon words pasted together are not a phrase).
• language.word(people, english): one word with its gloss and IPA, exact meaning first, nearest meaning second, found: false rather than a coined word.

Both resolve the people the way name.generate does (id, people name, language name, a near miss, a people named inside the text) and never halt a tick. What the plan requested arrives in the writer prompt as a short "native speech available this scene" list, and the writer is held to it: never explain or describe a language, at most one native word or phrase per scene unless the scene is about language, never gloss a word in narration, untranslated speech only where the POV character would not understand it.

The POV character's own line is one clause, speaks Quqo (related to Gaipo), followed by the scene's language situation, computed in Python from the POV's people, the realm that holds the location and the characters the plan puts in the room: shared (one tongue), related (same family; they follow each other with effort, not fluently), foreign (no shared tongue; speech reaches the POV as sound) or unknown (no data, no line). The planner sees the same line, so it can request a phrase or stage a misunderstanding on purpose.

Project Management Commands

# Show recently accessed projects with UUIDs
novel recent [--limit 10]

# Resume most recent project
novel resume [--n 1]

# Resume specific project by UUID (supports partial match)
novel resume --uuid <uuid> [--n 1]
# Example: novel resume --uuid a1b2

Inspection & Management Commands

# Show project overview (includes story foundation)
novel status [--json] [--project <path>]

# Show goal hierarchy and protagonist goals
novel goals [--json] [--project <path>]

# Show per-tick coherence metrics (loop churn, contradictions, tension vs. target, goal relevance)
novel metrics [--json] [--project <path>]

# List story threads (scenes served, tick run pattern, tension range, members, last activity)
novel threads [--json] [--project <path>]

# Show world lore and rules
novel lore [--group-by category|type|none] [--category <cat>] [--type <type>] [--importance <level>] [--stats] [--json] [--project <path>]

# List entities
novel list characters [-v] [--json] [--project <path>]
novel list locations [-v] [--json] [--project <path>]
novel list loops [-v] [--json] [--project <path>]
novel list scenes [-v] [--json] [--project <path>]

# Inspect entity details
novel inspect --id <ID> [--history-limit 5] [--raw] [--project <path>]
novel inspect --file <path> [--raw]

# Preview next plan without executing
novel plan [--save <file>] [-v] [--project <path>]

# Compile scenes into manuscript
novel compile [--output <file>] [--format markdown|html|prose|epub|pdf] [--include-metadata/--no-metadata] [--scenes <range>] [--project <path>]

# Generate LLM title suggestions from the story's foundation and content
novel titles [--count 10] [--output <file>] [--project <path>]

# Manage checkpoints
novel checkpoint create [--message <msg>] [--project <path>]
novel checkpoint list [--project <path>]
novel checkpoint restore --id <checkpoint_id> [--project <path>]
novel checkpoint delete --id <checkpoint_id> [--project <path>]

# Version history: each novel is its own git repository, one commit per tick
novel history [--limit 10] [--project <path>]   # last commits
novel history enable [--project <path>]         # turn it on for an existing novel

# Plot-first mode commands
novel plot generate [--count 5] [--project <path>]
novel plot status [--detailed] [--project <path>]
novel plot next [--project <path>]
novel plot revise [--project <path>]
novel plot clear [--yes] [--project <path>]

Configuration

Global configuration in ~/.storydaemon/config.yaml:

llm:
  backend: codex              # "codex" (Codex CLI), "api" (multi-provider API), "gemini-cli" (Gemini CLI), or "claude-cli" (Claude Code CLI)
  codex_bin_path: codex
  model: gpt-5.5             # Generic API model (gpt-5.5/5.4/5.4-mini/5.2, claude-fable-5, claude-opus-4-8, claude-sonnet-4-6, claude-sonnet-4-5, claude-haiku-4-5, gemini-3.1-pro-preview, gemini-3-pro-preview, gemini-3-flash-preview, gemini-2.5-pro)
  openai_model: gpt-5.5       # Legacy OpenAI-specific key (still honored)
  planner_max_tokens: 4000    # Tactical-planning budget, read by the multi-stage planner.
                              # Load-bearing: over the ceiling the plan is cut off mid-JSON
                              # and the tick completes on a stub plan with no POV character
                              # and no tool actions. Raise it if you add fields to the plan.
  writer_max_tokens: 3000     # Legacy flat ceiling; superseded by generation.scene_word_targets
                              # (kept so old project configs load, nothing reads it)
  timeout: 300                # Per-call timeout (seconds), applied on CLI and api backends

paths:
  novels_dir: ~/novels

generation:
  max_tools_per_tick: 3
  recent_scenes_count: 3           # Context for planner
  include_overall_summary: true    # Include story-wide summary
  # Scene length is flexible - planner can optionally suggest "brief", "short", "long", or "extended"
  scene_length_preset: house       # "house" (the shipped 400/800/1400/2200 targets) or
                                   # "masters" (1600/2150/3150/4450, calibrated to the
                                   # 21-masterwork per-chapter distribution; roughly
                                   # doubles prose per tick). An explicit
                                   # scene_word_targets dict that differs from the
                                   # defaults wins over the preset.

  # Scene skeletons (experimental, disabled by default): typed paragraph
  # plans sampled from the masters block grammar guide prose structure
  enable_scene_skeleton: false

  # Sectioned scene writing (experimental, disabled by default): write one scene
  # across several plan-addressed calls instead of one request, so a
  # masters-length chapter is reachable. Requires enable_scene_skeleton and
  # falls back to single-shot on any failure.
  subblock_generation: false
  subblock_section_blocks: 10      # skeleton blocks per call (1 = per-sub-block)

  # Plot-first mode (optional, disabled by default)
  use_plot_first: false            # Enable emergent plot-first architecture
  plot_beats_ahead: 5              # Generate this many beats at a time
  plot_regeneration_threshold: 2   # Regenerate when pending beats < this
  verify_beat_execution: true      # Verify beat was accomplished via LLM
  allow_beat_skip: false           # Allow skipping beats that aren't accomplished
  fallback_to_reactive: true       # Fall back to reactive mode if beat generation fails

Project-specific configuration in <project>/config.yaml.

API Environment Variables

For API backends, set these environment variables (e.g., in a .env file):

OPENAI_API_KEY   # OpenAI GPT-5.5 / 5.4 / 5.2
ANTHROPIC_API_KEY   # Anthropic Claude Sonnet 4.5 / Haiku 4.5 (legacy CLAUDE_API_KEY still works as a deprecated fallback)
GEMINI_API_KEY   # Google Gemini 3 / 2.5
OPENROUTER_API_KEY   # OpenRouter (model "openrouter")
OPENROUTER_MODEL     # OpenRouter model to route to, e.g. "anthropic/claude-3.7-sonnet"
VENICE_API_KEY       # Venice (model "venice")
VENICE_MODEL         # Venice model to use, e.g. "venice-uncensored" or "llama-3.3-70b"

Development Status & Roadmap

StoryDaemon has a mature end-to-end pipeline (agent, memory, writer, evaluator, CLI, multi-stage planner, and optional plot-first mode). The active direction is Emergent Coherence — emergent content under high structural constraint: the LLM decides what happens while Python holds it to canon, arc shape, and a short revisable "rolling horizon" of beats regenerated from the prose just written.

The active roadmap is docs/EMERGENT_COHERENCE_PLAN.md. Current status:

• Phase 1 — Grounded identity (the LLM selects names/IDs, never authors them) — shipped. Python-grounded name.generate, resolved entity references, similarity-pre-filtered + LLM-judged contradiction detection.
• Phase 2 — Rolling horizon (lookahead emerges from the prose; beats are revisable) — shipped. Plus the novel plot revise trigger.
• Phase 3 — Constraint-as-pressure — in progress. Shipped: the per-tick coherence rubric (novel metrics), contradiction enforcement (disputed-lore quarantine), an LLM tension scorer + arc-pressure (a target tension curve injected into planner and writer, with named coherence.curve_preset control-point sets), the arc-phase planner mandate, a throughline gate with an LLM goal-relevance judge, honest loop accounting (judged loop closure, creation dedup and cap), the sacred finale (coherence.sacred_finale: on plot-first runs Python owns the last scene), the write-until-concluded scene loop (scenes are sized from generation.scene_word_targets and continued rather than truncated), the story-thread registry with thread identity by selection and a construction-pressure detector (novel threads), and the first slice of the block/sub-block DSL: scene skeletons (generation.enable_scene_skeleton), typed paragraph plans sampled from a block grammar measured on the masters corpus, validated by a production A/B (see the grammar study and the Slice 4 results), plus chapter-length calibration (generation.scene_length_preset, opt-in masters targets drawn from the corpus per-chapter distribution) and sectioned scene writing (generation.subblock_generation, default off: one scene written across several plan-addressed calls, its quality A/B still owed). Also shipped, instrument-only: the loop-aging gauge (novel loops, per-tick loop ages and a horizon per loop, corpus baseline in the loop-aging baseline), which found that only 14 percent of open loops ever close and that a third of them are scene-local suspense that was never story debt. Still to come: the loop-aging pressure itself, thread construction, and the remaining contract slices.
• Phase 4 — Setup/payoff foresight (planted-element ledger for clues/reveals) — deferred until 1–3 prove out.

Plot-first mode (Phase 5 of the legacy roadmap) is complete and available — automatic beat generation, beat-constrained writing, and beat verification — see the guide below. Note the two phase-numbering schemes differ: the legacy roadmap lives in docs/plan.md, the active one in docs/EMERGENT_COHERENCE_PLAN.md.

• Emergent Coherence Roadmap - active direction and status
• Plot-First Mode User Guide
• Phase 5 Implementation Summary
• Legacy Implementation Plan

Historical phase documents and older roadmap notes live under docs/archive/.

Testing

# Run all tests
pytest

# Run with coverage
pytest --cov=novel_agent

# Run specific test file
pytest tests/unit/test_file_ops.py

# Run tension tracking tests
pytest tests/unit/test_tension_evaluator.py -v
pytest tests/integration/test_tension_integration.py -v

# Run lore tracking tests
pytest tests/unit/test_lore_tracking.py -v

# Run manual test suite (comprehensive verification)
python tests/manual_tension_test.py

Test Coverage

• Automated suite: 800+ tests across tests/unit/ and tests/integration/, covering the tick loop, memory/entities, planner, tension scoring, arc-pressure, lore + contradiction detection, the coherence rubric, scene skeletons, and the CLI commands. Run with pytest.
• Manual checks: python tests/manual_tension_test.py exercises tension scoring on real scene generation (not auto-discovered by pytest).

Documentation

Core Documentation

• Technical Specification - Detailed system design
• Implementation Plan - Phase-by-phase roadmap

Phase Documentation

• Phase 1 Guide - Core framework implementation
• Phase 2 Detailed Plan - Memory system design
• Phase 2 Completion - Implementation summary
• Phase 3 Detailed Plan - Planner and execution loop design
• Phase 3 Completion - Implementation summary
• Phase 4 Detailed Plan - Writer and evaluator design
• Phase 4 Implementation Summary - Implementation summary
• Phase 5 Detailed Plan - Dynamic entity updates design
• Phase 6 Detailed Plan - CLI enhancements design
• Phase 6 Implementation Summary - Implementation summary
• Phase 6 Quick Reference - Command reference guide
• Phase 6 Complete - Completion summary

Feature Documentation

• Phase 7A: Bounded Emergence Framework - Story foundation and goal hierarchy
• Plot-First Mode Guide - User guide for emergent plot-first generation
• Phase 5 Implementation Summary - Technical details of plot-first implementation
• Masters Block Grammar Study - Block-ordering statistics measured on 21 classic novels; the data behind scene skeletons
• The Block DSL in the Writer Path - Diagram of how scene skeletons, sectioned writing, selective revision and beat contracts fit into the writer, and which are switched on
• Scene Skeleton Results - Production A/B results, surface-quality pass, and reading notes for Slice 4
• Name Generator Implementation - Syllable-based name generation
• Project Safety Improvements - UUID system and scene titles
• Resume Workflow - Recent projects and resume commands
• Agent vs CLI Tools - Understanding tool layers

Philosophy

StoryDaemon emphasizes:

• Emergence over planning - Discovery writing powered by structured reasoning
• Deep POV realism - Narrative written through character perception
• Memory evolution - Entities grow and change organically
• Tool autonomy - LLM decides which tools to use based on story needs

License

MIT License - See LICENSE file for details.

Acknowledgments

• Inspired by my own NovelWriter
• Originally built in Windsurf; later development with Claude Code
• Runs on multi-provider LLM backends (OpenAI, Claude, Gemini, OpenRouter, Venice) and local agent CLIs (Codex, Claude Code, Gemini)
• Uses Typer for CLI, Chroma for vector storage

---

Created by Edward A. Thomson
GitHub