wbopan/flashtrace
wbopan/flashtrace has 44 GitHub stars and 1 forks. GitHot measured 0 stars in 24 hours. Last checked 2026-11-15 23:45 UTC. Last push 2026-11-12.
Measured star growth
- in 24h
- 0
- in 7 days
- Collecting history
- in 15 days
- Collecting history
- in 1 month
- Collecting history
Net star changes over overlapping periods, not daily totals. Missing measurements are left blank. Stars measure attention, not software quality.
About
Efficient multi-token attribution for reasoning language models — Python package, CLI, and HTML token traces
AIREADME.md
Original
FlashTrace
ICML 2026 Oral · Fast token attribution for reasoning language models.
Wenbo Pan et al. · Live demo
FlashTrace traces generated answers back to the prompt tokens that shaped them. Use it from Python or the command line, export JSON traces, and render standalone HTML heatmaps for inspection and sharing.
📄 Paper · 🚀 Quickstart · 💻 CLI · 📝 Citation
Why FlashTrace
Reasoning models produce long generated chains, final answers, and intermediate spans that deserve targeted inspection. FlashTrace gives researchers a package-first workflow for tracing a selected generated span back to its supporting prompt tokens.
The method combines span-wise aggregation with recursive attribution, delivering more than 130x acceleration on long generated spans while preserving strong faithfulness on RULER, MATH, MoreHopQA, HotpotQA, and Aider code generation tasks.
You get:
- top-k prompt tokens ranked by attribution score
- JSON traces for downstream analysis
- standalone HTML token heatmaps
- optional per-hop attribution panels
- inclusive generation-token span controls for answer and reasoning segments
Install
From PyPI:
pip install flashtrace
From a local checkout:
pip install -e .
For development:
pip install -e ".[dev]"
FlashTrace uses PyTorch, Transformers, Accelerate, NumPy, and tqdm. A CUDA-capable GPU is recommended for public-scale Hugging Face models.
Quickstart
from flashtrace import FlashTrace, load_model_and_tokenizer
prompt = """Context: Paris is the capital of France.
Question: What is the capital of France?"""
target = "Paris"
model, tokenizer = load_model_and_tokenizer("Qwen/Qwen3-8B", device_map="auto")
tracer = FlashTrace(model, tokenizer, chunk_tokens=128, sink_chunk_tokens=32)
trace = tracer.trace(
prompt=prompt,
target=target,
output_span=(0, 0),
hops=1,
)
print(trace.topk_inputs(10))
trace.to_json("trace.json")
trace.to_html("trace.html")
trace.topk_inputs(10) returns TokenScore objects aligned to prompt-token indices:
rank index token score
1 2 Paris 0.184
2 7 capital 0.131
3 10 France 0.119
trace.html is a standalone heatmap that highlights prompt tokens by final attribution score and includes trace metadata for the selected generated span.
FlashTrace(..., use_chat_template=True) formats prompts with the tokenizer chat template for chat-tuned models.
Vision-language models
Install the processor dependency and load Qwen3-VL with eager attention:
pip install -e '.[vlm]'
from PIL import Image
from flashtrace import FlashTrace, load_vlm_and_processor
model, processor = load_vlm_and_processor(
"Qwen/Qwen3-VL-8B-Instruct",
device_map="auto",
dtype="bfloat16",
)
tracer = FlashTrace(model, processor, chunk_tokens=128, sink_chunk_tokens=32)
trace = tracer.trace(
prompt="What is shown in this image?",
images=Image.open("example.jpg").convert("RGB"),
target="A red car.",
output_span=(0, 3),
method="ifr-span",
)
For VLM traces, trace.prompt_tokens and trace.scores contain the visual
patch tokens followed by the user text tokens in original sequence order.
trace.metadata["multimodal"] records the absolute and projected visual-token
spans, the processor grid, the post-merge visual grid, and the spatial merge
size. Multiple images may be passed as a list.
Qwen3-VL and Qwen2.5-VL always use model-produced stored attention, even when
recompute_attention=True, because their multimodal RoPE layouts are not
compatible with FlashTrace's 1-D RoPE recomputation path. Multimodal prompts
always use the processor's official chat template so that visual placeholders
and M-RoPE token types stay aligned.
Run the reproducible end-to-end smoke test with:
CUDA_VISIBLE_DEVICES=0 uv run --extra vlm \
python -m evaluations.qwen3_vl_smoke \
--model Qwen/Qwen3-VL-8B-Instruct --max-new-tokens 4
Replace the model ID with Qwen/Qwen3-VL-8B-Thinking to validate the Thinking
checkpoint through the same path.
For a visually checkable spatial-grounding example, generate a four-quadrant image and trace the final answer through the model's short reasoning:
CUDA_VISIBLE_DEVICES=0 uv run --extra vlm \
python -m evaluations.qwen3_vl_quadrant_example --strict
The upper-left quadrant contains a red circle; the other quadrants contain a
blue square, green triangle, and yellow star. The script prints the generated
Reasoning: and Final answer: lines, aggregates attribution over all four
quadrants, and writes the input plus visual-token heatmap under
/tmp/flashtrace-qwen3-vl-quadrant/. A successful strict run requires both the
answer and the dominant attribution quadrant to match the upper-left evidence.
Command Line
Create prompt and target files:
printf "Context: Paris is the capital of France.\nQuestion: What is the capital of France?\n" > prompt.txt
printf "Paris" > target.txt
Run a trace:
flashtrace trace \
--model Qwen/Qwen3-8B \
--prompt prompt.txt \
--target target.txt \
--output-span 0:0 \
--hops 1 \
--html trace.html \
--json trace.json
The command prints a compact top-k table and writes the requested artifacts.
Useful flags:
--model: Hugging Face model id or local model path--prompt: UTF-8 prompt text file--target: UTF-8 target text file--output-span: inclusiveSTART:ENDindices over generated tokens--reasoning-span: inclusiveSTART:ENDindices for a reasoning segment--method:flashtrace,ifr-span,ifr-tokenwise, orifr-matrix--recompute-attention: lower-memory attention recomputation path--use-chat-template: format prompts with the tokenizer chat template--device-map: Transformers device map, defaultauto--dtype:auto,float16,bfloat16, orfloat32
Token Spans
output_span and reasoning_span use inclusive generation-token indices. The first generated token has index 0.
Use an initial trace to inspect tokenization:
for index, token in enumerate(trace.generation_tokens):
print(index, repr(token))
Then choose spans:
trace = tracer.trace(
prompt=prompt,
target=target,
reasoning_span=(0, 79),
output_span=(80, 85),
hops=1,
)
Scores are aligned to trace.prompt_tokens. trace.per_hop_scores stores the same prompt-token alignment for each hop.
Interpreting Results
High-scoring prompt tokens are the tokens FlashTrace attributes most strongly to the selected generated span. For answer inspection, use output_span around the final answer tokens. For chain-of-thought or reasoning inspection, use reasoning_span around the generated reasoning segment.
Recommended workflow:
- Run a trace with your prompt and target.
- Inspect
trace.generation_tokens. - Select the answer or reasoning span.
- Export
trace.html. - Compare top-k tokens with the source prompt and any expected evidence.
Supported Models
FlashTrace targets Llama/Qwen-style decoder-only Hugging Face causal LMs with:
model.layers- Q/K/V/O attention projections
- RMSNorm or LayerNorm
- RoPE metadata
Validated model families for the first public release:
- Qwen2
- Qwen3
- Qwen3-VL (Instruct and Thinking)
- Qwen2.5-VL
- Llama
Python API
The public package exports:
from flashtrace import (
FlashTrace,
TraceResult,
load_model_and_tokenizer,
load_vlm_and_processor,
)
FlashTrace.trace(...) accepts:
prompt: strtarget: str | Noneoutput_span: tuple[int, int] | Nonereasoning_span: tuple[int, int] | Nonehops: intmethod: "flashtrace" | "ifr-span" | "ifr-tokenwise" | "ifr-matrix"renorm_threshold: float | Noneimages: image | list[image] | None
TraceResult includes:
prompt_tokensgeneration_tokensscoresper_hop_scoresthinking_ratiosoutput_spanreasoning_spanmethodmetadata
Export helpers:
trace.topk_inputs(20)
trace.to_dict()
trace.to_json("trace.json")
trace.to_html("trace.html")
Examples
python examples/quickstart.py --help
python examples/quickstart.py \
--model Qwen/Qwen3-8B \
--prompt "Context: Paris is the capital of France. Question: What is the capital of France?" \
--target "Paris" \
--output-span 0:0 \
--html trace.html
Heavy model examples are intended for GPU environments. CPU smoke tests use tiny randomly initialized models.
Repository Map
flashtrace/: reusable Python packageexamples/: public quickstartstests/: CPU smoke testsexp/: paper experiments and research artifactsdocs/superpowers/: design and implementation planning documents
Research Experiments
The exp/ directory contains the paper-era experiment runners, case studies, and saved artifacts. The public package API lives in flashtrace/; experiment scripts keep compatibility imports during the package migration.
Troubleshooting
CUDA memory
Use smaller models, lower precision, device_map="auto", shorter prompts, or --recompute-attention.
Span selection
Print trace.generation_tokens and select inclusive generated-token indices. Tokenization can split visible words into multiple model tokens.
Deterministic generation
Pass a target file for attribution against a known output. Leave --target out when you want the CLI to generate with deterministic defaults.
Tokenizer alignment
Inspect trace.prompt_tokens and trace.generation_tokens when scores appear shifted from visible text. Attribution scores follow tokenizer-level alignment.
HTML export
trace.to_html("trace.html") writes a standalone file that can be opened locally or shared as an artifact.
Paper
FlashTrace implements the method described in Towards Long-Horizon Interpretability: Efficient and Faithful Multi-Token Attribution for Reasoning LLMs.
Citation
@misc{pan2026flashtrace,
title={Towards Long-Horizon Interpretability: Efficient and Faithful Multi-Token Attribution for Reasoning LLMs},
author={Pan, Wenbo and Liu, Zhichao and Wang, Xianlong and Yu, Haining and Jia, Xiaohua},
year={2026},
eprint={2602.01914},
archivePrefix={arXiv},
primaryClass={cs.LG}
}