Dual Licensed · Open source under AGPL-3.0 · Closed-source under Proprietary License (written consent required). Commercial inquiries: [email protected] / bilibili [柏斯阔落]

Changelog

Table of Contents

1. Quick Start
2. Feature Overview (Main Panel)
3. Chat
4. Mind Sea
5. Skill Workshop
6. Naga Network
7. Ark Market
8. Terminal Settings
9. MusicBox
10. Floating Ball
11. Global Features
12. Backend Architecture
13. Optional Configuration
14. Ports
15. Troubleshooting

Quick Start

Requirements

• Python 3.11 (>=3.11, <3.12)
• Optional: uv — faster dependency installation
• Optional: Neo4j — local knowledge graph memory

Installation

git clone https://github.com/Xxiii8322766509/NagaAgent.git
cd NagaAgent

# Frontend
cd frontend
npm install
cd ..

# Backend
# Option 1: uv (recommended)
uv sync

# Option 2: Manual
python -m venv .venv
source .venv/bin/activate   # Windows: .\.venv\Scripts\activate
pip install -r requirements.txt

Minimal Configuration

Copy config.json.example to config.json and fill in your LLM API credentials:

{
  "api": {
    "api_key": "your-api-key",
    "base_url": "https://api.deepseek.com",
    "model": "deepseek-v3.2",
    "provider": "deepseek",
    "use_gateway": true,
    "api_format": "openai"
  }
}

Works with any OpenAI-compatible API (DeepSeek, Qwen, OpenAI, Ollama, etc.), and also supports the native Anthropic format (set api_format to "anthropic"). After login, NagaModel Gateway is used by default; disable use_gateway in Terminal Settings to use your local provider credentials.

Configuration is loaded from config.json in the project root first. If that file does not exist, NagaAgent falls back to the user data config (~/.naga/config.json on Linux/macOS, %APPDATA%/NagaAgent/config.json on Windows). The settings page writes back to the currently active config file.

Launch

cd frontend && npm run dev   # one-click launch (configured)

Feature Overview (Main Panel)

After launch, you enter the Main Panel (PanelView) with a 3D parallax effect (perspective rotation driven by mouse movement). All features are accessible through eight entry buttons on the main panel:

1. Chat · MessageView

Streaming Tool Calls

The chat engine streams output via SSE, simultaneously sending to the frontend display and TTS sentence splitting. Tool calls do not rely on OpenAI's Function Calling API — the LLM embeds JSON inside ```tool``` code blocks, so **any OpenAI-compatible provider works out of the box**. Anthropic native API format is also supported via api_format: "anthropic".

Single-round tool call flow:

LLM streaming output ──SSE──▶ Frontend real-time display
       │
       ▼
parse_tool_calls_from_text()
  ├─ Phase 1: Extract ```tool``` code blocks
  └─ Phase 2: Fallback to bare JSON extraction
       │
       ▼
  Route by agentType
  ├─ "mcp"      → MCPManager.unified_call()
  ├─ "openclaw" → Agent Server /openclaw/send
  └─ "live2d"   → UI animation notification
       │
       ▼
  asyncio.gather() parallel execution of all tools
       │
       ▼
  Inject results into messages, start next LLM round (up to 5)

• Text parsing: json5 tolerant parsing, fullwidth characters auto-normalized
• SSE format: data: {"type":"content"|"reasoning","text":"..."}\n\n (direct JSON, no base64)
• Loop limit: max_loop_stream = 5 (configurable)

Source: apiserver/agentic_tool_loop.py

Context Compression

Automatically triggered when session tokens exceed 100k, preventing context overflow:

Summary structure (6 sections): Key facts / User preferences / Important decisions / To-dos / Background info / Recent state. The <compress> tag persists to the session file but is not counted in LLM token statistics.

DeepSeek Reasoning Chain Display

When using DeepSeek, the reasoning field is pushed in real-time via SSE and displayed in a distinct style in the frontend.

2. Mind Sea · MindView

GRAG Knowledge Graph Memory

GRAG (Graph-RAG) automatically extracts quintuples from conversations, stores them in Neo4j, and retrieves relevant memories as LLM context.

Quintuple structure: (subject, subject_type, predicate, object, object_type)

Extraction pipeline:

1. Structured extraction (preferred): beta.chat.completions.parse() + Pydantic QuintupleResponse, temperature=0.3, up to 3 retries
2. JSON fallback: on parse failure, extract content from the first [ to the last ]
3. Filtering: keep only facts (behaviors, relations, states, preferences); filter metaphors, hypotheticals, emotions

Entity types: person / location / organization / item / concept / time / event / activity

Task manager:

• 3 asyncio workers consuming asyncio.Queue(maxsize=100)
• SHA-256 deduplication: identical pending/running tasks are skipped
• Hourly cleanup of tasks older than 24h

Dual storage:

• Local: logs/knowledge_graph/quintuples.json
• Cloud: Neo4j graph database, graph.merge() upsert

RAG retrieval: Keyword extraction → Cypher query → formatted as subject(type) —[predicate]→ object(type) injected into context

Remote memory: Logged-in users prefer NagaMemory cloud; local GRAG remains available when not logged in. To avoid performance cost, the cloud path no longer auto-falls back to local Neo4j.

Source: summer_memory/

Mind Sea 3D Visualization

Canvas 2D + hand-rolled 3D projection (not WebGL), spherical coordinate camera, perspective division 700 / depth.

7-layer render order: Background gradient → floor grid → water surface → volumetric light (3 god rays) → particle system (3 layers, 125 particles) → bioluminescent plankton (10 with trails) → knowledge graph nodes & edges (depth-sorted)

Graph mapping: subject/object → nodes, predicate → directed edges, degree centrality → node height weight, 100-node limit

Interactions: Drag to orbit, middle-click to pan, scroll to zoom, node click/drag, keyword search filter

3. Skill Workshop · SkillView

Built-in MCP Agents

A pluggable tool architecture based on the Model Context Protocol, each tool running as an independent Agent:

Registration & discovery: mcp_registry.py glob-scans **/agent-manifest.json, dynamically instantiates via importlib.import_module.

MCP Management UI

The frontend McpAddDialog.vue provides a graphical MCP tool management interface — add or remove tools at runtime without restarting.

Community Skill Installation

The Skill Workshop supports one-click installation of community-published Skills (Agent Browser, Brainstorming, Context7, Firecrawl Search, etc.). Backend endpoints: GET /openclaw/market/items, POST /openclaw/market/items/{id}/install

Source: mcpserver/

4. Naga Network · Community Forum

Community Forum

Accessible from the "Naga Network" block on the main panel, with a fully embedded community:

Source: frontend/src/forum/

Credits & Affinity System

A gamified interaction system exclusive to logged-in users:

Related APIs (proxied through API Server to the Naga portal): /api/checkin, /api/affinity, /api/credits

5. Ark Market · MarketView

The Ark Market consolidates all resource acquisition and management, organized into seven tabs:

6. Terminal Settings · ConfigView

Settings page redesigned as a single page with three tabs (3-in-1 unification):

Character Card System

The characters/ directory manages switchable AI characters, each described by a JSON config file:

{
  "ai_name": "Najezhda",
  "user_name": "User",
  "live2d_model": "NagaTest2/NagaTest2.model3.json",
  "prompt_file": "conversation_style_prompt.txt",
  "portrait": "Naga.png",
  "bio": "An AI assistant created by developer 柏斯阔落, nicknamed Naga."
}

• Each character directory contains an independent conversation style prompt, Live2D model assets, and portrait image
• Once a character is activated, AI name and Live2D model are managed entirely by the character JSON and cannot be manually overridden in the UI
• Default character: Najezhda

Source: characters/

7. MusicBox · MusicView

A standalone music player that shares the same playback instance as the main interface BGM (unified BGM architecture):

• Playlist Editor (MusicEditView): Manage track list; changes sync to the global player immediately on save
• Playback state sync: Play / pause icon updates in real time with audio events
• Loop: Automatically advances to the next track when the current one ends
• Live2D lip sync: During TTS playback, AdvancedLipSyncEngineV2 drives Live2D mouth shapes at 60FPS

8. Floating Ball · FloatingView

Click the "Float" button on the main panel to enter the lightweight floating ball window, cycling through four states:

ball (100×100 circle) → compact (420×100 collapsed bar) → full (420×N expanded) → classic (normal window)

Appearance & animation:

• Sprite-frame blink animation: 5 frames (open → half-closed → closed → half-closed → open), 70ms/frame, random interval trigger
• While generating a reply: glowing halo pulse effect
• On hover: brightness lift effect
• Transparent frameless window, freely draggable

Features:

• Chat input available directly in floating state; message history viewable in compact / full states
• Screenshot capture panel: select a screen window as an image attachment
• File upload support
• Right-click menu implemented via Electron native menu (prevents clipping in small window)

Global Features

Voice Interaction

TTS (Text-to-Speech)

• Engine: Edge-TTS, OpenAI-compatible endpoint /v1/audio/speech
• Architecture: 3-thread pipeline — sentence queue → TTS calls (Semaphore(2) concurrency) → pygame playback
• Live2D lip sync: 60FPS extraction of 5 parameters (mouth_open / mouth_form / mouth_smile / eye_brow_up / eye_wide)
• Port cleanup: auto-detects and releases occupied ports at startup

ASR (Speech Recognition)

• Local engine: FunASR, with VAD endpoint detection and WebSocket real-time streaming
• Three-mode auto-switch: LOCAL (FunASR) → END_TO_END (Qwen Omni) → HYBRID

Realtime Voice Chat (requires DashScope API Key)

• Full-duplex WebSocket voice interaction via Qwen Omni
• Echo suppression, VAD detection, audio chunking (200ms), session cooldown control

{
  "voice_realtime": {
    "enabled": true,
    "provider": "qwen",
    "api_key": "your-dashscope-key",
    "model": "qwen3-omni-flash-realtime"
  }
}

Source: voice/

Live2D Avatar

Renders Cubism Live2D models using pixi-live2d-display + PixiJS WebGL. SSAA super-sampling: Canvas rendered at width × ssaa, CSS transform: scale(1/ssaa).

4-channel orthogonal animation system (live2dController.ts):

Merge order: body state → mouth → action → manual override → emotion blend → tracking blend

OpenClaw Computer Control

Interfaces with the OpenClaw Gateway (port 20789) to dispatch AI coding assistants for local tasks via natural language.

• 3-tier fallback startup: packaged binary → global openclaw command → auto npm install -g openclaw
• Supports sessionKey hooks (2026.2.17+), configurable custom hooks path
• POST /openclaw/send sends instructions, waits up to 120 seconds

Task Scheduler (TaskScheduler):

• Task step recording (purpose / content / output / analysis / success status)
• Auto-extraction of "key findings" markers
• Memory compression: when steps exceed threshold, LLM generates CompressedMemory (key_findings / failed_attempts / current_status / next_steps)
• schedule_parallel_execution() runs task lists in parallel via asyncio.gather()

Source: agentserver/

Splash Animation

Backend Architecture

NagaAgent consists of five independent microservices, all orchestrated by main.py:

┌─────────────────────────────────────────────────────────┐
│                   Electron / PyQt5 Frontend              │
│  Vue 3 + Vite + UnoCSS + PrimeVue + pixi-live2d-display │
│                                                         │
│  PanelView · MessageView · MindView · SkillView         │
│  MarketView · ConfigView · MusicView · FloatingView     │
│  ForumListView · ForumPostView · ForumQuotaView …       │
└──────────┬─────────────┬──────────────┬─────────────────┘
           │             │              │
   ┌───────▼──────┐ ┌────▼────┐  ┌─────▼──────┐
   │  API Server  │ │  Agent  │  │   Voice    │
   │   :8000      │ │  Server │  │  Service   │
   │              │ │  :8001  │  │   :5048    │
   │ Chat / SSE   │ │         │  │            │
   │ Tool calls   │ │ Task    │  │ TTS / ASR  │
   │ Compression  │ │ schedule│  │ Realtime   │
   │ Doc upload   │ │ OpenClaw│  │ voice      │
   │ Auth proxy   │ │         │  │            │
   │ Memory API   │ └────┬────┘  └────────────┘
   │ Skill Market │      │
   │ Config mgmt  │  ┌───▼──────────┐
   └──────┬───────┘  │  OpenClaw    │
          │          │  Gateway     │
   ┌──────▼──────┐   │  :20789      │
   │ MCP Server  │   └─────────────┘
   │   :8003     │
   │ Tool registry│
   │ Agent disco │
   │ Parallel    │
   └──────┬──────┘
          │
  ┌───────┴───────────────────────┐
  │      MCP Agents (pluggable)   │
  │ Weather | Search | Crawl      │
  │ Launcher | Guide | Doc | MQTT │
  └───────────────────────────────┘
          │
   ┌──────▼──────┐
   │    Neo4j    │
   │   :7687     │
   │  Knowledge  │
   │   Graph     │
   └─────────────┘

Directory Structure

NagaAgent/
├── main.py                   # Unified entry point, orchestrates all services
├── build.py                  # Cross-platform build script
├── config.json               # Runtime config (copied from config.json.example)
├── pyproject.toml            # Version 5.1.5, project metadata & dependencies
│
├── apiserver/                # API Server (:8000)
│   ├── api_server.py         #   FastAPI main app
│   ├── agentic_tool_loop.py  #   Multi-round tool call loop
│   ├── llm_service.py        #   LiteLLM unified LLM interface
│   └── streaming_tool_extractor.py  # Streaming sentence split + TTS dispatch
│
├── agentserver/              # Agent Server (:8001)
│   ├── agent_server.py
│   └── task_scheduler.py     #   Task orchestration + compressed memory
│
├── mcpserver/                # MCP Server (:8003)
│   ├── mcp_server.py
│   ├── mcp_registry.py       #   Manifest scanning + dynamic registration
│   ├── mcp_manager.py        #   unified_call() routing
│   ├── agent_weather_time/
│   ├── agent_open_launcher/
│   ├── agent_game_guide/
│   ├── agent_online_search/
│   ├── agent_crawl4ai/
│   ├── agent_playwright_master/
│   ├── agent_vision/
│   ├── agent_mqtt_tool/
│   └── agent_office_doc/
│
├── summer_memory/            # GRAG knowledge graph memory
│   ├── quintuple_extractor.py  #   Supports OpenAI / Anthropic formats
│   ├── quintuple_graph.py
│   ├── quintuple_rag_query.py
│   ├── task_manager.py
│   ├── memory_manager.py
│   └── memory_client.py      #   NagaMemory remote client
│
├── voice/                    # Voice service (:5048 / :5060)
│   ├── output/               #   TTS + lip sync
│   └── input/                #   ASR + realtime voice
│
├── characters/               # Character config directory
│   └── Najezhda/             #   prompt / Live2D model / portrait
│
├── frontend/                 # Electron + Vue 3 frontend
│   ├── electron/             #   Main process
│   │   └── modules/          #   backend / hotkeys / menu / tray / updater / window
│   └── src/
│       ├── views/            #   All page views
│       ├── forum/            #   Forum module
│       ├── components/       #   Shared components
│       ├── composables/      #   useAuth / useBackground / useAudio …
│       └── utils/            #   live2dController / session / config
│
├── system/                   # Config loader, env checker, system prompts
├── guide_engine/             # Game guide engine
└── logs/                     # Runtime logs, knowledge graph files

Optional Configuration

{
  "grag": {
    "enabled": true,
    "neo4j_uri": "neo4j://127.0.0.1:7687",
    "neo4j_user": "neo4j",
    "neo4j_password": "your-password"
  }
}

{
  "system": { "voice_enabled": true },
  "tts": {
    "port": 5048,
    "default_voice": "zh-CN-XiaoxiaoNeural"
  }
}

{
  "voice_realtime": {
    "enabled": true,
    "provider": "qwen",
    "api_key": "your-dashscope-key",
    "model": "qwen3-omni-flash-realtime"
  }
}

{
  "web_live2d": {
    "ssaa": 2,
    "model": {
      "source": "injected by the GUI or character system",
      "x": 0.5,
      "y": 1.3,
      "size": 6800
    },
    "custom_model_id": "generated model ID after upload",
    "face_y_ratio": 0.13,
    "tracking_hold_delay_ms": 100
  }
}

{
  "mqtt": {
    "enabled": true,
    "broker": "mqtt-broker-address",
    "port": 1883,
    "topic": "naga/agent/topic",
    "client_id": "naga-agent-client"
  }
}

Ports

Troubleshooting

python main.py --check-env --force-check  # Full environment diagnostics
python main.py --quick-check              # Quick check

Contributing

Issues and Pull Requests are welcome. For questions, join the QQ channel nagaagent1.

Star History