Awesome Graph Engineering

Engineer the organization, not just the agent.
A field guide, open dataset, and interactive atlas for graph-structured multi-agent systems—programmable AI-agent organizations.

🌐 Explore the Atlas · 🤗 Use the Dataset · 📥 Download CSV · ➕ Suggest a Resource

🧭 Definition · 🧱 Taxonomy · ⚖️ Boundaries · 📚 Resources · 📣 Share · 🧾 Cite · 📜 License

🌍 Languages: English · 简体中文 · Español · Français · Deutsch · 日本語 · 한국어 · Português (Brasil) · Help translate

Here, graph engineering means engineering graph-structured AI-agent systems. Graph databases, knowledge graphs, graph ETL, GraphRAG, and graph neural networks are outside this scope unless they directly support an agent graph. See the boundary guide.

Every number above is generated from data/resources.jsonl and checked in CI, so it cannot drift from the catalog. Full breakdowns by section, layer, type, and evidence label are in the resource directory and the interactive atlas.

🧭 Working definition

Graph engineering is the practice of specifying, executing, observing, and evolving a graph-structured agent system—its roles and runtime instances, the contracts that connect them, the state and artifacts they share, and the evidence by which their collective behavior is judged—so that the system can be controlled, tested, and improved as an engineered whole.

The graph must be load-bearing rather than decorative: its declared topology, realized run graph, or graph-generating policy materially constrains execution and remains inspectable enough to version, trace, evaluate, or deliberately change.

Connected improvement loops can monitor, constrain, or veto one another, while external measurements, fixed rules, and human decisions provide grounding. Carlos E. Perez’s 2026 “graph of loops” essay articulates this control perspective. Applied to AI-agent organizations, agent roles and runtime instances are the primary adaptive nodes; deterministic tests, audit cycles, human decisions, and real-world observations may act as control nodes or gates.

Graph engineering is used here as an emerging, non-standard term. Its scope synthesizes established work in agent-oriented programming, multi-agent systems, blackboard architectures, distributed systems, workflow orchestration, and graph-based language-agent research such as GPTSwarm. The evidence map separates source-backed claims from analytical inferences. Steinberger’s and Perez’s July 2026 posts document recent practitioner usage; they do not establish coinage or consensus. The first survey to adopt the label, Graph Engineering in the Era of LLM Agents (August 2026 preprint), places it at the system level above prompt, context, harness, and loop engineering, with each agent's own loop running inside the system graph.

The minimum test

A system is in scope when all three conditions are load-bearing:

1. Plural bounded agency — at least two separately accountable agentic runtime instances each make bounded execution decisions; they may share a role or model.
2. Explicit coordination semantics — edges say what may move, when control transfers, and how results are accepted or rejected.
3. An inspectable graph artifact — topology, run graph, or graph-generating policy can be versioned, traced, evaluated, or changed deliberately.

A single agent with many tools is still one node. A deterministic DAG of ordinary functions is workflow engineering. A “swarm” of personas without contracts, state boundaries, or evidence gates is branding, not graph engineering.

🧩 Core primitives

Primitive	Engineering question	Typical artifact
Agent node	Who owns this context, objective, capability, and permission boundary?	Role spec, runtime identity, tool allowlist
Typed edge	What crosses this relationship, under which preconditions and schema?	Handoff contract, protocol message, artifact reference
Org graph	Which reusable roles may coordinate, delegate, verify, or escalate?	Versioned roles-and-permissions topology
Run/work graph	What execution structure does this particular job require?	Run-scoped DAG, event graph, trace, lineage
Gate	What evidence allows work to advance?	Test, evaluator, quorum rule, human approval
State boundary	What is shared, isolated, checkpointed, or authoritative?	Blackboard, artifact store, snapshot, worktree
Graph policy	Who may create, rewrite, cancel, or route nodes and edges?	Scheduler, budget policy, topology generator

Org graph ≠ run graph

	Org graph	Run/work graph
Describes	Stable roles, capabilities, authority, and allowed relationships	The tasks, dependencies, branches, and evidence produced in one run
Changes	Deliberately, through architecture or policy updates	Dynamically, as planning and execution reveal new work
Primary question	“Who may do and verify what?”	“What must happen next for this outcome?”
Safety boundary	A planner may select permitted relationships	A planner must not silently expand its own permissions

Org graph and run/work graph are analytical views, not standardized object types. An implementation may represent both in one runtime graph if standing authority remains distinguishable from run-specific execution.

✅ Do you need a graph?

Start with the smallest architecture that closes the quality loop. One well-instrumented agent loop is usually enough until work needs at least one of these:

• Specialization: independent roles need different context, tools, models, or permissions.
• Parallelism: separable work should fan out, then reconcile under an explicit contract.
• Independent verification: a producer should not be the only judge of its own output.
• Fault or trust isolation: failure, untrusted input, or privileged actions must be contained.

If none applies, invest in the loop first: context, stopping conditions, tests, retries, and observability. Loop engineering is primarily temporal—how one agent progresses through repeated work. Graph engineering is primarily relational—how multiple scoped actors and control surfaces compose. The two concerns are complementary; neither is a maturity stage.

🗺️ Choose your path

If you are…	Start with	Then inspect
🛠️ Building your first multi-agent flow	Two roles, one typed handoff, one evidence gate	Start Here → Frameworks & SDKs
🏗️ Designing an agent organization	Org/run separation, topology, authority, state ownership	Taxonomy → Protocols & Handoffs
🚦 Operating a production graph	Durable execution, traces, budgets, replay, escalation	Reliability → Observability & Cost
🔬 Researching adaptive systems	Foundations, benchmarks, topology optimization, limits	Research Foundations → Critiques & Limits

🧱 The nine engineering layers

#	Layer	The design question
01	Roles	Who exists, and what does each node own?
02	Topology	How are the roles arranged, and why this shape?
03	Handoffs	What may cross each edge, in what schema?
04	Work graphs	What execution structure does this run need now?
05	State	What is shared, isolated, durable, or authoritative?
06	Gates	What evidence advances, rejects, or escalates work?
07	Reliability	How does the graph fail, recover, replay, and stop?
08	Observability & cost	Can operators explain the path, latency, and spend?
09	Evolution	When and how may topology or policy redesign itself?

The layers are concerns, not compulsory stages. A two-role pipeline with a typed handoff and a test gate can be a complete graph. Read the full taxonomy, comparison guide, and anti-patterns.

📚 Resource directory

The resource directory prioritizes primary research, official documentation, maintained projects, standards, reproducible benchmarks, and production reports. Evidence labels identify source type; they do not score quality. Each entry contains an original summary, a distinct engineering rationale, and one primary layer.

Corpus composition: 620 resources drawn from 248 distinct sources. By evidence label: 144 peer-reviewed research, 108 practitioner analysis, 154 research preprints, 71 official documentation, 65 maintained OSS projects, 40 benchmarks or datasets, 13 industry standards, 14 community resources, 11 books or courses. 351 entries were published or materially updated in 2026.

Use the interactive Resource Atlas to search and filter the same records by section, source type, evidence label, and layer.

Contents

• Start Here — 53 resources
• Research Foundations — 163 resources
• Frameworks & SDKs — 61 resources
• Protocols & Handoffs — 41 resources
• State, Memory & Artifacts — 32 resources
• Verification & Evals — 38 resources
• Reliability & Durable Execution — 41 resources
• Observability & Cost — 39 resources
• Benchmarks & Datasets — 51 resources
• Production Case Studies — 39 resources
• Critiques & Limits — 62 resources

Start Here

Resource	Source	What it contributes	Evidence
📝 A practical guide to building agents Blog · Architecture guide	OpenAI Guides & Resources OpenAI · 2025	Explains when agents are appropriate, how to define their tools and instructions, and how to move from one agent to manager and handoff-based multi-agent patterns. Why: A useful first check on whether distinct roles justify the coordination cost of a graph.	Practitioner analysis Roles
📝 Building effective agents Blog · Architecture guide	Anthropic Engineering Erik Schluntz, Barry Zhang (Anthropic) · 2024	The pattern vocabulary most work-graph designs are built from: prompt chaining, routing, parallelization (sectioning and voting), orchestrator-workers, and evaluator-optimizer, plus the workflow-vs-agent distinction (predefined code paths vs model-directed control). The orchestrator-workers pattern — subtasks determined by the orchestrator at runtime, not predefined — is the minimal dynamic work graph. Why: The pattern vocabulary most work-graph designs are assembled from, and the reference practitioners actually cite.	Practitioner analysis Topology
📝 From Loop Engineering to Graph Engineering? Blog · Contemporary framing	X Articles Carlos E. Perez · 2026	Frames the shift as loop architecture: networks of improvement cycles that monitor, feed, constrain, and correct one another, with reliability located in their edges. Why: Adds the grounding requirement missing from topology-only accounts: independent counter-metrics, frozen tests or rules, external anchors, and human ownership of root objectives.	Practitioner analysis Gates
📄 The Five Ws of Multi-Agent Communication: Who Talks to Whom, When, What, and Why – A Survey from MARL to Emergent Language and LLMs Paper · Communication survey	Transactions on Machine Learning Research Jingdi Chen; Hanqing Yang; Zongjun Liu; Carlee Joe-Wong · 2026	Synthesizes multi-agent communication across reinforcement learning, emergent language, and LLM systems through sender, recipient, timing, content, and purpose decisions. Why: Provides a design-oriented map for engineering edge selection, message timing, payloads, grounding, scalability, and interpretability.	Peer-reviewed research Handoffs
📚 Multi-Agent Reference Architecture Docs · Production reference architecture	Microsoft Microsoft · 2026	Maps agent registries, communication, memory, observability, evaluation, security, governance, and deployment into an end-to-end reference architecture for multi-agent systems. Why: Provides a broad engineering checklist grounded in customer implementations; it is evolving guidance in an official repository, not a versioned standard.	Official documentation Topology
🎬 Steve Yegge on AI Agents and the Future of Software Engineering Video · Agent-organization pre-history	The Pragmatic Engineer Gergely Orosz, Steve Yegge · 2026	February 10, 2026 podcast (with written companion) where Yegge lays out his eight-level agent adoption spectrum, culminating at level 8: building your own orchestrator to coordinate many agents. Captures the "you fire up another agent, then another... you can't leave" coordination pain that made programmable agent organizations feel inevitable. Why: Lays out an adoption progression for agent organizations that gives teams a vocabulary for where they currently are.	Practitioner analysis Roles
📝 Welcome to Gas Town Blog · Agent-organization pre-history	Medium Steve Yegge · 2026	The January 1, 2026 essay introducing the first widely-adopted agent organization: 20-30 parallel Claude Code instances under a role hierarchy — Mayor (chief-of-staff), long-lived Crew, ephemeral swarming polecats feeding a merge queue — at a reported ~$100/hour token burn. The clearest pre-discourse instance of a stable org graph plus ephemeral work units, cited throughout the July 2026 debate. Why: The essay that popularized running dozens of parallel coding agents under a coordinator, and the concrete origin of much later practice.	Practitioner analysis Topology
🧰 loop-engineering: Practical Patterns, Starters & CLI Tools Tool · Discourse-to-tooling	GitHub Cobus Greyling · 2026	Open-source repo that converted the Osmani/Cherny discourse into runnable artifacts: production loop patterns, loop-audit, loop-init, and loop-cost CLI tools that scaffold and score agent loops. Evidence of how fast the founding discourse turned into tooling — the same trajectory graph engineering began following in July 2026. Why: Turns the loop discourse into runnable patterns and starters, which is the fastest way to get a node worth coordinating.	Maintained OSS project Work graphs
📝 Context vs Prompt vs Loop vs Harness Engineering: The 2026 Stack Blog · Engineering-stack framing	explainx.ai explainx.ai · 2026	The layered-stack taxonomy the graph engineering discourse extends: prompt (how do I word this message), context (what the model sees per call), loop (what autonomous workflow repeats until a goal), harness (what code runs it reliably). Graph engineering was pitched on July 18 as the fifth layer on exactly this ladder, so this is the reference frame most reactions assume. Why: Distinguishes the neighboring dimensions cleanly, useful when a team is arguing across layers without noticing.	Practitioner analysis Topology
📝 Agentic Loops: From ReAct to Loop Engineering (2026 Guide) Blog · Loop-engineering pre-history	Data Science Dojo Data Science Dojo · 2026	Traces the full lineage from the 2022 ReAct paper through 2024-25 agent harnesses to the June 2026 loop-engineering moment, giving the discourse its research pedigree. Handy for readers who want the deep history of the loop layer that graph engineering composes over. Why: Traces the lineage from early reasoning-and-acting agents to current practice, giving newcomers the arc rather than a snapshot.	Practitioner analysis Work graphs
📝 Anthropic's Coding Chief Doesn't Write Prompts Anymore. He Writes Loops. Blog · Loop-engineering pre-history	Medium The Latency Gambler (Kanishk S.) · 2026	One of the widest-read June 2026 amplifications of the Cherny quote, reconstructing what his loop setup actually consists of and why "the loop is the easy part" while context and stop conditions are the work. Representative of the wave of secondary coverage that carried the loops framing to a general engineering audience just before the graphs turn. Why: Reconstructs what a working practitioner loop setup actually contains, beyond the quotable claim.	Practitioner analysis Work graphs
🎬 Boris Cherny: "I don't prompt Claude anymore. I have loops prompting Claude" (clip) Video · Loop-engineering pre-history	X Boris Cherny (clip), shared by Paweł Huryn · 2026	Widely-shared June 2026 clip of Claude Code's creator saying he no longer prompts Claude directly but runs loops that prompt it and figure out what to do next — the insider confirmation that pushed loop engineering from meme to practice. Cherny called it "the transition we're going to see for the rest of the year"; the multi-loop, multi-agent implications set up the graphs turn a month later. Why: First-hand statement from the author of a widely used coding harness on why he stopped prompting directly.	Practitioner analysis Work graphs
📝 Demystifying Loop Engineering: Get More From AI Agents, Avoid Loopmaxxing Blog · Loop-engineering pre-history	TechTalks TechTalks (bdtechtalks.com) · 2026	June 22, 2026 explainer that added the discourse's main cautionary term, "loopmaxxing" — over-automating loops past the point of verification. Its warnings about unverified autonomous cycles carry directly into graph engineering, where the failure mode compounds across every node in the organization. Why: Introduces a useful cautionary term for over-automating work that did not need it, a failure mode that scales badly into graphs.	Practitioner analysis Reliability
📝 Introducing Loop Engineering Blog · Loop-engineering pre-history	Medium Valentina Alto · 2026	July 2026 essay framing loop engineering as the emerging human-agents interaction model rather than a coding trick — one of the last substantial loops-era pieces published in the days before the graphs pivot, useful for seeing exactly what the discourse looked like at the moment it shifted. Why: Frames the shift as an interaction model between humans and agents rather than a coding trick, which is the right altitude for org design.	Practitioner analysis Roles
📝 Loop Engineering: Designing Loops That Prompt Coding Agents Blog · Loop-engineering pre-history	addyosmani.com Addy Osmani · 2026	The June 7, 2026 essay (republished by O'Reilly Radar on June 22) that named and structured loop engineering: five components — automations, worktrees, skills, plugins/MCP connectors, and subagents — plus external state so progress survives between runs. The subagents-and-external-state sections are where the essay already reaches toward multi-agent graph structure, making it the immediate intellectual substrate of graph engineering. Why: The essay that structured the loop layer into named components, and the substrate every graph node sits on.	Practitioner analysis Work graphs
📝 Peter Steinberger: "You should be designing loops that prompt your agents" Blog · Loop-engineering pre-history	X Peter Steinberger (@steipete) · 2026	The June 2026 post (~6.5M views) that made loop engineering a mass movement: "you shouldn't be prompting coding agents anymore, you should be designing loops that prompt your agents." The direct ancestor of the July graphs post — graph engineering is explicitly framed as the layer above this idea. Why: The post that moved the field from prompting to designing systems that prompt, and the direct ancestor of the coordination discourse.	Practitioner analysis Work graphs
📝 Prompt Engineering Is Dead. Long Live Loop Engineering Blog · Loop-engineering pre-history	HackerNoon Fractera · 2026	The succession-framing piece whose title became shorthand for how each layer of the engineering stack dethrones the last — the same rhetorical template the graph engineering discourse reused three weeks later ("loop engineering is dead, long live graph engineering" jokes in the July 18 replies trace here). Why: Useful as a record of how each layer of the stack reframes the one below, including the hype that comes with it.	Practitioner analysis Topology
📝 Stop Prompting AI and Start Building Loops Blog · Loop-engineering pre-history	Product Market Fit Product Market Fit (Substack) · 2026	Newsletter breakdown of Boris Cherny's loops-not-prompts shift, unpacking the 16-minute source interview into the working parts: the context, the stop condition, the budget cap, and the verifiable target. Useful as the most careful secondary account of what Cherny actually described versus what the viral quote implied. Why: Unpacks a viral claim into its working parts: context, stop condition, budget cap, and verifiable target.	Practitioner analysis Reliability
🧭 AgenticOrgChart.com: AI Agent Org Chart Patterns List · Topology patterns	agenticorgchart.com agenticorgchart.com · 2026	Vendor-neutral reference documenting seven canonical multi-agent topologies — single agent, supervisor, hierarchical, human-in-the-loop, and others — each with cited public production examples. The closest thing the young field has to a pattern catalog for org graphs, published before the term existed. Why: A vendor-neutral catalog of canonical topologies, handy for naming the shape you are already building.	Community resource Topology
📝 Shubham Saboo: SKILL graphs for a six-agent team Blog · Two-graph precursor	X Shubham Saboo (@Saboo_Shubham_) · 2026	Early-2026 precursor to Saboo's two-graph framing quoted in the founding discourse: he describes running a team of six agents, each with a SOUL file, memory files, and a traversable graph of skill docs, coordinating through shared files. Concrete example of graph-shaped shared knowledge serving a standing agent organization rather than a retrieval pipeline. Why: A concrete early example of graph-shaped shared knowledge serving a standing agent team rather than a retrieval pipeline.	Practitioner analysis State
📝 The Shift from Models to Compound AI Systems Blog · Compound-systems economics	Berkeley AI Research blog Matei Zaharia, Omar Khattab, Lingjiao Chen et al. · 2024	The essay that reframed AI engineering as system design over budgets: tripling model training spend might buy 30%→35% on coding, while a sample-test-filter system reaches 80% with existing models, so builders must allocate latency, cost, and compute budgets across components. The economic argument underneath the whole engineering stack that graph engineering now sits atop. Why: Reframes AI engineering as system design over budgets, which is the premise the whole coordination layer rests on.	Practitioner analysis Topology
📝 Are we still talking loops or did we shift to graphs yet? Blog · Founding discourse	X Peter Steinberger · 2026	The July 2026 post that pushed the discourse from loop engineering to graph engineering: one line asking whether single-agent loops are already obsolete as the unit of design. Its virality reframed planning and decomposition as the question of what work graph your agents should form per task. Why: The post that named the shift from loops to graphs, and the entry point to the whole coordination discourse.	Practitioner analysis Topology
📝 Graph Engineering: Designing Multi-Agent Organizations Blog · Work graph vs org graph	explainx.ai Yash Thakker · 2026	Canonical framing article (July 18, 2026) separating the stable org graph of long-lived agents from the ephemeral work graph whose task nodes split, merge, reorder, or disappear as evidence arrives. Argues graphs force you to declare every node, edge, and failure mode that loops let you defer. Why: The framing article that separates the standing org graph from the per-run work graph, which is the field's most useful distinction.	Practitioner analysis Topology
🧭 Awesome Harness Engineering List · Adjacent curated lists	GitHub ai-boost · 2026	Curated resources for the harness layer directly beneath agent organizations: agent loops, context management, tool design, MCP and skills, permissions, memory, verification, observability, sandboxing, and reference implementations, at roughly 5.0k stars. Why: The neighbouring layer's catalog, useful when a graph problem turns out to be a single node's harness problem.	Community resource Reliability
🧭 Awesome Loop Engineering List · Adjacent curated lists	GitHub Chaoyue He · 2026	Sibling list covering the loop layer: 1,022 resources, 22 operational patterns, 22 loop contracts, and 8 runtime starters for recurring agent systems, plus an interactive atlas and a structured dataset. 59 stars, last pushed 26 Oct 2026; complements graph engineering by covering the time axis (re-triggering, supervision) where this list covers the org axis. Why: The sibling catalog for the loop layer beneath this one, where a single node's behavior cycle is designed.	Community resource Work graphs
🧭 Awesome Agentic Engineering List · Adjacent curated lists	GitHub Jordi Mas · 2026	Practitioner-oriented list of agentic engineering resources, weighted toward coding agents, MCP, AGENTS.md, and team workflow patterns rather than research papers. 115 stars, last pushed 27 Jun 2026. Why: A practitioner-oriented catalog of agentic engineering resources adjacent to this list's scope.	Community resource Topology
🧭 Awesome AI Agents (e2b) List · Adjacent curated lists	GitHub e2b · 2026	The largest general inventory of autonomous agent products and projects, with use-case filters and per-project metadata. 30,775 stars, last pushed 21 Aug 2026. Mostly single-agent products — use it for ecosystem coverage, not for coordination semantics. Why: One of the broadest agent catalogs, useful for finding tools this list deliberately does not carry.	Community resource Topology
🧭 Awesome AI Agent Papers List · Adjacent curated lists	GitHub VoltAgent · 2026	Weekly-refreshed arXiv digest restricted to papers published from January 2026 onward, bucketed into multi-agent coordination, memory/RAG, tooling, evaluation/observability, and security. 1,884 stars, last pushed 2 Nov 2026. The coordination bucket is the directly relevant slice. Why: A running collection of agent research papers, useful as a feed for what to evaluate next.	Community resource Topology
🧭 Awesome LLM-Powered Agent List · Adjacent curated lists	GitHub Yupeng Hou · 2025	Long-running academic reading list on LLM-powered agents with a dedicated multi-agent-collaboration section. 2,260 stars, but effectively dormant — last push 30 Apr 2025, so it misses the entire 2025-2026 orchestration wave. Useful for pre-2025 lineage only. Why: An academic reading list on LLM-powered agents, complementing the engineering emphasis here.	Community resource Topology
🧭 Awesome Graphs Meet Agents List · Disambiguation	GitHub Yuanchen Bei et al. · 2026	Companion list to the survey "Graphs Meet AI Agents," covering graph-empowered agents (planning, memory, coordination over graph data) and agent-assisted graph learning. 211 stars, last pushed 28 Jul 2026. Included as the key disambiguation reference: this is graph-DATA-meets-agents, not multi-agent org topology. Why: The graph-data-meets-agents list, and the clearest reference for the boundary this project draws around its own name.	Community resource State
🧭 Awesome MCP Servers List · Adjacent curated lists	GitHub Frank Fiegel · 2026	The de facto directory of Model Context Protocol servers, at roughly 97.2k stars and 19.1k forks. Relevant as the tool-edge inventory: MCP servers are what nodes in an agent graph reach for, distinct from the agent-to-agent edges themselves. Why: The registry practitioners actually use to find tool-edge servers, which is where most handoff surface now lives.	Community resource Handoffs
🎓 Hugging Face Agents Course Course · Courses and MOOCs	Hugging Face Hugging Face · 2026	Free five-unit course covering agent fundamentals, then smolagents, LangGraph, and LlamaIndex, with a unit on multi-agent collaboration and a leaderboard-graded final assignment. Two certificate tiers; the companion repo has 34,399 stars and was last pushed 14 Oct 2026. Why: A free hands-on course that gets someone to a working agent before they attempt an organization of them.	Book/course Roles
🎓 Berkeley Agentic AI MOOC (CS294/194-196) Course · Courses and MOOCs	UC Berkeley RDI Dawn Song, Xinyun Chen · 2025	Free lecture series with a 32K+ learner community (about 40K counting the earlier Advanced LLM Agents run), with dedicated sessions on multi-agent AI systems, post-training verification, agent evaluation, and safety. Slides and recordings stay public; the Fall 2025 page is the current edition and no Spring/Fall 2026 edition is linked as of Aug 2026. Why: University-level lectures covering agent systems, with material on coordination rather than only single-agent tricks.	Book/course Topology
🎓 Multi AI Agent Systems with crewAI Course · Courses and MOOCs	DeepLearning.AI João Moura, DeepLearning.AI · 2026	Three-hour, 18-lesson beginner course on assigning specialized roles to agents and coordinating them in series, parallel, and hierarchical structures, with error handling and loop prevention. Applied to six business processes; free during the DeepLearning.AI platform beta. Why: A short course on role-based crews, the fastest path to a first working multi-agent design.	Book/course Roles
🎓 Introduction to LangGraph (LangChain Academy) Course · Courses and MOOCs	LangChain Academy LangChain · 2026	Free six-module, 55-lesson (~6 hour) course on LangGraph as an explicit graph of nodes and edges: state, memory, human-in-the-loop interrupts, long-term memory, and deployment. The closest thing to a canonical hands-on course on programmable agent topology. Why: The vendor course for graph-native orchestration, covering state and control flow rather than prompt tricks.	Book/course Work graphs
🎓 AI Evals for Engineers & PMs Course · Courses and MOOCs	Maven Hamel Husain, Shreya Shankar · 2026	Paid cohort course that has trained 2,000+ engineers and PMs, including teams at OpenAI and Anthropic; material fully refreshed for the September 2026 cohort. Covers error analysis on agentic traces (tool calls, retrieval) and validating LLM-as-judge against expert labels — the measurement half of running a multi-agent org. Why: Evaluation practice aimed at engineers, which is the skill an agent-organization gate depends on.	Book/course Gates
📚 AAMAS 2027 — 26th International Conference on Autonomous Agents and Multiagent Systems Docs · Conferences and workshops	IFAAMAS IFAAMAS; general chairs William Yeoh and Neil Yorke-Smith · 2027	The field's primary academic venue, which describes itself as the largest and most influential conference on agents and multiagent systems, held 3–7 May 2027 in Hanoi, Vietnam. Its call covers agentic technology alongside the established agent and multiagent topics. Why: The long-running academic venue for autonomous agents and multi-agent systems, where much of this theory is still developed.	Official documentation Topology
📚 AI Engineer World's Fair 2026 Docs · Conferences and workshops	AI Engineer swyx and the AI Engineer team · 2026	Largest practitioner conference for AI engineers: 29 June - 2 July 2026 at Moscone Center, San Francisco, with 6,000+ attendees, 300+ speakers, and 10 parallel tracks including agent orchestration, evaluation, sandbox/platform engineering, memory and continual learning, and context engineering. Talks are posted publicly to YouTube after the event. Why: The practitioner conference where agent-engineering patterns are presented before they reach papers.	Official documentation Topology
📚 Interrupt 2026 — The Agent Conference by LangChain Docs · Conferences and workshops	LangChain LangChain · 2026	Vendor-run but production-focused agent conference: two days in San Francisco (13-14 May 2026), plus NYC on 24 September and London on 13 October 2026. Heavy on real-team case studies of agents in production; LangChain used the 2026 SF edition to ship LangSmith Engine, Managed Deep Agents, SmithDB, and Context Hub. Why: A conference dedicated to agent engineering, run by a framework author, useful for tracking where tooling is heading.	Official documentation Topology
📚 Agentic AI Summit 2026 (Berkeley RDI) Docs · Conferences and workshops	UC Berkeley RDI Berkeley Center for Responsible, Decentralized Intelligence · 2026	Academic-industry summit at UC Berkeley on 1-2 August 2026, expecting 5,000+ in-person attendees plus a global livestream, spanning foundation models, agent frameworks, evaluation, infrastructure, and deployment. Speakers include Sergey Levine, Ion Stoica, Dawn Song, Andrej Karpathy, Wojciech Zaremba, Ali Ghodsi, and Andrew Ng. Why: An academic-industry summit on agentic AI, with material on coordination and safety.	Official documentation Topology
📚 Who Verifies the Agents? — NeurIPS 2026 Workshop Docs · Conferences and workshops	NeurIPS 2026, Sydney Organizers from UC Berkeley, MIT-IBM Watson AI Lab, Stanford, Meta · 2026	Workshop on verification as the bottleneck for reliable agent systems, 11-12 December 2026 in Sydney; submissions of 4-9 pages due 29 August 2026 on OpenReview. Explicit tracks on meta-agents, self-improving systems, and scalable oversight of long-horizon behavior — i.e. the verification edges in an agent graph. Why: A workshop specifically about verifying agents, which is the unsolved half of agent-organization design.	Official documentation Gates
📚 LaMAS 2026 — LLM-based Multi-Agent Systems workshop at AAAI-26 Docs · Conferences and workshops	AAAI-26, Singapore LaMAS organizing committee · 2026	Held 27 January 2026 at Singapore EXPO, subtitled "Towards Responsible, Reliable, and Scalable Agentic Systems." The call names organization and coordination of multiple LLM agents as a first-class topic alongside emergent-behavior evaluation and safety — the closest venue match to graph engineering's core question. Accepted papers remain linked from the site. Why: A workshop dedicated to LLM-based multi-agent systems, the closest academic venue to this list's subject.	Official documentation Topology
📚 Agents in the Wild: Safety, Security, and Beyond (2nd workshop, ICML 2026) Docs · Conferences and workshops	ICML 2026, Seoul Agents in the Wild organizers · 2026	Held 11 July 2026 in Hall B2, Seoul, on agents reasoning and acting safely in open-ended environments; multi-agent coordination and long-horizon safety is an explicitly welcomed topic. Submission deadline was 8 May 2026, so the current value is the accepted-paper list. Why: A workshop on agent safety and security in deployment, where failures across edges get examined.	Official documentation Reliability
📚 Model Context Protocol — Contributor Communication and Working Groups Docs · Communities and forums	Model Context Protocol / Agentic AI Foundation MCP core maintainers · 2026	Official map of where MCP work actually happens: contributor Discord for working/interest groups, live calls at meet.modelcontextprotocol.io, GitHub Discussions for proposals, and SEPs submitted as PRs to the seps/ directory. Any Discord discussion leading to a decision must be moved to a GitHub Discussion or Issue for a searchable record. Why: How the tool-edge protocol is actually governed, useful before betting an architecture on its roadmap.	Official documentation Handoffs
📐 Agentic AI Foundation (AAIF), Linux Foundation Standard · Communities and forums	Linux Foundation Linux Foundation · 2025	Neutral governance body formed December 2025, anchored by the Model Context Protocol (donated by Anthropic), goose (Block), and AGENTS.md. Launched with 150+ member organizations, described as the fastest-growing foundation in Linux Foundation history; A2A, donated by Google in June 2025, sits alongside it. Why: Neutral governance for agent infrastructure, which determines whether protocol choices stay portable.	Industry standard Handoffs
📐 AGNTCY — the Internet of Agents collective Standard · Communities and forums	Linux Foundation / AGNTCY Cisco, LangChain, Galileo and contributors · 2026	Open-source collective, now part of the Linux Foundation, building the cross-vendor plumbing for agent collaboration: a federated agent-discovery registry, a network-level agent messaging protocol, an identity/verification layer, and multi-agent observability collectors. Supporting orgs include LlamaIndex, Glean, AG2, Traceloop, MongoDB, and Weaviate. Why: A cross-vendor collective for agent interoperability, worth tracking if your nodes span organizations.	Industry standard Handoffs
📝 Agentic AI Weekly (Berkeley RDI) Blog · Newsletters, blogs and podcasts	Substack Berkeley Center for Responsible, Decentralized Intelligence · 2026	Weekly research-leaning roundup from the group behind the Berkeley Agentic AI MOOC and the AgentX-AgentBeats competition (1,300+ teams across 100+ countries, $1M+ in prizes, Phase 2 launched 23 February 2026). Consistent academic coverage of coordination and evaluation that practitioner newsletters skip. Why: A weekly digest from an academic center, a lower-noise way to track the field than social feeds.	Practitioner analysis Topology
📝 Latent Space: The AI Engineer Podcast and Newsletter Blog · Newsletters, blogs and podcasts	Substack swyx, Alessio Fanelli · 2026	194,000+ subscribers and 217 episodes as of late July 2026, with recurring deep dives on multi-agent orchestration — including a 7 April 2026 episode on Symphony, OpenAI's internal Elixir-based layer for spinning up, supervising, and coordinating large fleets of coding agents across tickets and repos. Why: The podcast and newsletter where practitioners describe agent architectures before they are written up.	Practitioner analysis Topology
📝 Anthropic Engineering blog Blog · Newsletters, blogs and podcasts	Anthropic Anthropic engineering teams · 2026	First-party writeups of production agent architecture, including "How we built our multi-agent research system" (13 Jun 2025, orchestrator-worker with parallel subagents), "Building a C compiler with a team of parallel Claudes" (5 Feb 2026), "Harness design for long-running application development" (24 Mar 2026), and "Scaling Managed Agents: Decoupling the brain from the hands" (8 Apr 2026). Why: The first-party engineering blog behind several of the production accounts in this catalog.	Practitioner analysis Roles
📝 3 Years of Graph Engineering with LangGraph Blog · Framework retrospective	LangChain Sydney Runkle, Harrison Chase · 2026	A July 22, 2026 retrospective from the team behind the reference graph-native framework, arguing production agents need cycles rather than DAGs, that a loop is a simplified graph rather than an alternative to one, and that nodes have grown from single model calls into full agent runs. Reports 65M+ monthly downloads. Why: The framework author's own account of what three years of building agent graphs taught them, including where the DAG mental model breaks.	Practitioner analysis Topology
📄 From Static Templates to Dynamic Runtime Graphs: A Survey of Workflow Optimization for LLM Agents Paper · Field survey	arXiv Ling Yue, Kushal Raj Bhandari, Ching-Yun Ko, Dhaval Patel, Shuxin Lin, Nianjun Zhou, Jianxi Gao, Pin-Yu Chen, Shaowu Pan · 2026	Surveys how agent systems build and optimize executable workflows, treating them as agentic computation graphs and separating reusable templates from the run-specific graph and its execution trace. Organizes the literature by when structure is fixed, what is optimized, and which signals guide it, and argues for structure-aware evaluation alongside task scores. Why: A workflow-optimization survey whose split between reusable templates and run-specific graphs matches the org and run graph distinction used throughout.	Research preprint Topology
📄 Graph Engineering in the Era of LLM Agents: From Individual Intelligence to System Intelligence Paper · Field survey	arXiv Yuyuan Feng, Zhishang Xiang, Chaobin Yang, Qichao Ma, Zerui Chen, Yujing Zhang, et al. · 2026	The first survey to adopt the graph engineering label for LLM agent systems. It places prompt and context engineering at the model level, harness and loop engineering at the level of a single agent, and graph engineering at the system level, where explicit graphs organize tasks, coordinate agents, and manage runtime state, with a further part on system evolution. It also reviews benchmarks, open-source libraries, and applications, and proposes ontology engineering as a future semantic layer. A system qualifies when changing its structure changes how it executes; a knowledge graph used only as a retrieval source does not. Why: The field survey that names this catalog's subject, with a stack in which each agent's loop sits inside a system-level graph and an inclusion rule close to the boundary drawn here.	Research preprint Topology
🧭 Awesome Graph Engineering (DEEP-JLU) List · Adjacent curated lists	GitHub DEEP-JLU · 2026	Companion list to the survey Graph Engineering in the Era of LLM Agents, arranged in the survey's order from model intelligence through individual intelligence to system intelligence, with papers, benchmarks, and open-source libraries at each level. 394 stars, last pushed 30 Oct 2026. Broader than this list: its model-level, single-agent, and ontology-engineering sections fall outside the scope used here. Why: The survey authors' own running index, useful for following new work in the categories the survey defines.	Community resource Topology

Research Foundations

Resource	Source	What it contributes	Evidence
📄 Agent-Oriented Programming Paper · Agent foundations	Artificial Intelligence Yoav Shoham · 1993	Defines a programming paradigm in which agents are first-class components described through mental state and governed by explicit interaction rules. Why: Establishes the intellectual lineage for treating an agent role as a programmable organizational unit.	Peer-reviewed research Roles
📄 Intelligent Agents: Theory and Practice Paper · Agent foundations	The Knowledge Engineering Review Michael Wooldridge; Nicholas R. Jennings · 1995	Surveys the properties, architectures, and engineering approaches that distinguish autonomous agents from ordinary software modules. Why: Grounds the agency-at-the-nodes boundary that separates an agent graph from a deterministic workflow.	Peer-reviewed research Roles
📄 The Blackboard Model of Problem Solving and the Evolution of Blackboard Architectures Paper · Shared-state architectures	AI Magazine H. Penny Nii · 1986	Describes systems in which independent specialists coordinate opportunistically through a shared problem state and a control component. Why: Supplies a durable model for shared state without requiring every node to exchange its full context directly.	Peer-reviewed research State
📄 Learning to Communicate with Deep Multi-Agent Reinforcement Learning Paper · Learned communication	NeurIPS Jakob Foerster; Ioannis Alexandros Assael; Nando de Freitas; Shimon Whiteson · 2016	Introduces reinforcement-learning methods that let agents learn communication protocols alongside their task policies, including discrete messages for execution. Why: Shows that edge content and communication policy can be engineered or learned rather than treated as free-form chat.	Peer-reviewed research Handoffs
📄 TarMAC: Targeted Multi-Agent Communication Paper · Learned communication	ICML Abhishek Das et al. · 2019	Uses attention to let agents address different messages to selected recipients instead of broadcasting the same information to the whole team. Why: Motivates selective, recipient-aware handoffs when all-to-all communication is wasteful or distracting.	Peer-reviewed research Handoffs
📄 AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversations Paper · LLM multi-agent systems	COLM Qingyun Wu; Gagan Bansal; Jieyu Zhang; Yiran Wu; Beibin Li; Erkang Zhu; Li Jiang; Xiaoyun Zhang; Shaokun Zhang; Jiale Liu; Ahmed Hassan Awadallah; Ryen W. White; Doug Burger; Chi Wang · 2024	Presents a framework for composing customizable conversational agents that can combine language models, tools, code execution, and human input. Why: An early, influential demonstration that agent roles and conversation links can be expressed as an executable topology.	Peer-reviewed research Topology
📄 GPTSwarm: Language Agents as Optimizable Graphs Paper · Topology optimization	ICML Mingchen Zhuge et al. · 2024	Represents language-agent systems as computational graphs and optimizes graph components from task feedback. Why: Makes the graph itself an optimization target rather than a fixed orchestration diagram.	Peer-reviewed research Evolution
📄 A Dynamic LLM-Powered Agent Network for Task-Oriented Agent Collaboration Paper · Dynamic topology	COLM Zijun Liu et al. · 2024	Constructs task-specific collaboration networks that can vary which agents participate and how they communicate instead of relying on one fixed team. Why: Provides evidence for adapting the work graph to the task while keeping the available agent roles reusable.	Peer-reviewed research Evolution
📄 Cut the Crap: An Economical Communication Pipeline for LLM-based Multi-Agent Systems Paper · Communication efficiency	ICLR Guibin Zhang, Yanwei Yue, et al. · 2025	First paper to formally define communication redundancy in LLM multi-agent pipelines, then one-shot prunes the spatial-temporal message-passing graph. Matches state-of-the-art topologies at $5.6 vs $43.7 cost, cuts 28-73% of tokens as a drop-in for existing frameworks, and incidentally defends against message-injection attacks — the canonical 'sparser edges are often free' result. Why: Defines communication redundancy formally, then removes it, which is the rare optimization that helps quality and cost together.	Peer-reviewed research Observability & cost
📄 G-Designer: Architecting Multi-agent Communication Topologies via Graph Neural Networks Paper · Topology optimization	ICML Guibin Zhang et al. · 2025	Uses graph neural networks to design communication structures for multi-agent systems instead of assuming a complete or manually chosen graph. Why: Connects task performance to explicit topology search and exposes communication structure as an engineering variable.	Peer-reviewed research Evolution
📄 Automated Design of Agentic Systems Paper · Automated agentic system design	ICLR Shengran Hu; Cong Lu; Jeff Clune · 2025	Formulates the automated invention of entire agentic systems (prompts, control flow, multi-agent structure) as search in code space: a Meta Agent Search loop programs new agent designs conditioned on an archive of prior discoveries. Discovered designs beat hand-crafted state-of-the-art agents and transfer across domains and models, establishing 'design the designer' as a research agenda. Why: Demonstrates automated search over coordination logic while retaining executable artifacts that engineers can inspect.	Peer-reviewed research Evolution
📄 AFlow: Automating Agentic Workflow Generation Paper · Workflow search	ICLR Jiayi Zhang et al. · 2025	Searches over reusable workflow operators to generate task-specific agentic workflows and improve them from evaluation results. Why: Offers a concrete method for evolving work graphs against measurable objectives rather than intuition alone.	Peer-reviewed research Evolution
📄 Multi-Agent Design: Optimizing Agents with Better Prompts and Topologies Paper · Joint prompt-topology optimization	ICLR Han Zhou, Xingchen Wan, Ruoxi Sun, Hamid Palangi, Shariq Iqbal, Ivan Vulić, Anna Korhonen, Sercan Ö. Arık (Google / Cambridge) · 2026	Google's three-stage Multi-Agent System Search: block-level prompt optimization, then topology optimization over a pruned configurable space, then workflow-level prompt optimization. Its headline empirical finding — most topology configurations hurt, and prompt quality dominates topology choice unless both are optimized together — is a core calibration result for anyone designing agent org graphs. Why: Optimizing prompts and topology in sequence, with evidence that the order and the interaction between them matter.	Peer-reviewed research Evolution
📄 Improving Factuality and Reasoning in Language Models through Multiagent Debate Paper · Debate and councils	ICML Yilun Du et al. · 2024	Tests rounds of proposal and critique among multiple language-model instances as a way to improve factual and reasoning answers. Why: Supplies an empirical basis for debate-style gates while leaving room to examine correlated errors and added cost.	Peer-reviewed research Gates
📄 Improving Multi-Agent Debate with Sparse Communication Topology Paper · Debate and councils	Findings of EMNLP Yunxuan Li et al. · 2024	Examines multi-agent debate under sparse communication structures rather than defaulting to full information exchange among every participant. Why: Isolates topology as a factor in debate quality and communication efficiency.	Peer-reviewed research Topology
📄 Reflexion: Language Agents with Verbal Reinforcement Learning Paper · Feedback and memory	NeurIPS Noah Shinn et al. · 2023	Lets an agent convert feedback into textual reflections stored in episodic memory and reused on later attempts. Why: Clarifies how a node loop can persist learning across retries without changing model weights.	Peer-reviewed research State
📄 CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing Paper · Tool-grounded verification	ICLR Zhibin Gou et al. · 2024	Uses external tools to obtain feedback that a language model can apply when critiquing and revising its outputs. Why: Supports verification gates grounded in observable evidence instead of another ungrounded model opinion.	Peer-reviewed research Gates
📄 A Multi-Level Organization for Problem Solving Using Many, Diverse, Cooperating Sources of Knowledge Paper · Blackboard coordination	IJCAI Lee D. Erman; Victor R. Lesser · 1975	Presents the Hearsay-II multi-level blackboard, where independent knowledge sources react to shared hypotheses, create explicit structural dependencies, and verify or revise one another's contributions. Why: Provides an early architecture for loosely coupled specialist nodes coordinating through inspectable shared state instead of direct all-to-all calls.	Peer-reviewed research State
📄 The Contract Net Protocol: High-Level Communication and Control in a Distributed Problem Solver Paper · Negotiated delegation	IEEE Transactions on Computers Reid G. Smith · 1980	Defines a negotiation protocol in which managers announce tasks, potential contractors bid, and awards establish temporary problem-solving relationships. Why: Supplies the classic contract for capability-aware delegation and auditable assignment edges between autonomous nodes.	Peer-reviewed research Handoffs
📄 Scaling Large Language Model-based Multi-Agent Collaboration Paper · Collaboration scaling	ICLR Chen Qian; Zihao Xie; YiFei Wang; Wei Liu; Kunlun Zhu; Hanchen Xia; Yufan Dang; Zhuoyun Du; Weize Chen; Cheng Yang; Zhiyuan Liu; Maosong Sun · 2025	Organizes agents on directed acyclic graphs and scales collaboration past 1,000 agents, comparing chain, tree, star, mesh, and random topologies. Two durable findings: irregular topologies beat regular ones, and performance follows a logistic 'collaborative scaling law' whose emergence point arrives earlier than neural-scaling emergence. Why: Makes topology a causal scaling variable rather than assuming that larger teams or denser communication automatically help.	Peer-reviewed research Topology
📄 MAS-Orchestra: Understanding and Improving Multi-Agent Reasoning Through Holistic Orchestration and Controlled Benchmarks Paper · Holistic orchestration	ICML Zixuan Ke et al. · 2026	Formulates orchestration as reinforcement-learned generation of a complete multi-agent program and evaluates it across controlled dimensions including depth, horizon, breadth, parallelism, and robustness. Why: Tests when whole-system graph structure helps instead of attributing gains to coordination without controlled task evidence.	Peer-reviewed research Work graphs
📄 CARD: Towards Conditional Design of Multi-agent Topological Structures Paper · Conditional topology	ICLR Tongtong Wu et al. · 2026	Generates communication graphs conditioned on agent roles, models, tools, and data sources, and adapts topology as task resources change. Why: Treats the available capabilities and directed edges as a versionable organizational artifact rather than a fixed team template.	Peer-reviewed research Evolution
📄 EvoMAS: Evolutionary Generation of Multi-Agent Systems Paper · System evolution	ICML Yuntong Hu et al. · 2026	Evolves structured multi-agent configurations through trace-guided mutation, crossover, selection, and an experience memory across reasoning, coding, and tool-use tasks. Why: Shows how an inspectable team specification can evolve from execution evidence while retaining executability and runtime robustness.	Peer-reviewed research Evolution
📄 Dynamic Generation of Multi LLM Agents Communication Topologies with Graph Diffusion Models Paper · Task-adaptive topology synthesis	ACL Eric Hanchen Jiang et al. · 2026	Introduces Guided Topology Diffusion, which iteratively synthesizes sparse, task-adaptive communication graphs using a proxy model for objectives such as accuracy, utility, and cost. Why: Treats topology as a multi-objective, per-task design artifact rather than a static collaboration template.	Peer-reviewed research Evolution
📄 AgentDropout: Dynamic Agent Elimination for Token-Efficient and High-Performance LLM-Based Multi-Agent Collaboration Paper · Dynamic node and edge elimination	ACL Zhexuan Wang; Yutong Wang; Xuebo Liu; Liang Ding; Miao Zhang; Jie Liu; Min Zhang · 2025	Ports dropout from neural nets to agent orgs: optimizes trainable adjacency matrices per communication round, then drops low-contribution agent nodes and redundant edges round-by-round rather than fixing one team for the whole task. Cuts prompt tokens ~21.6% and completion tokens ~18.4% while slightly improving task performance. Why: Makes node participation and edge density explicit cost-quality decisions instead of assuming every role should run on every task.	Peer-reviewed research Evolution
📄 Multi-agent Architecture Search via Agentic Supernet Paper · Query-dependent architecture search	ICML Guibin Zhang; Luyang Niu; Junfeng Fang; Kun Wang; Lei Bai; Xiang Wang · 2025	Instead of searching for one best static system, learns an 'agentic supernet' — a continuous distribution over architectures — and samples a query-dependent multi-agent system per task, allocating LLM calls and tool budget to query difficulty. Uses 6-45% of the inference cost of prior handcrafted or automated systems while beating them by 0.54-11.82%; ICML 2025 oral. Why: Replaces a one-size-fits-all graph with per-query architecture and resource allocation while keeping the design space explicit.	Peer-reviewed research Evolution
📄 AMAS: Adaptively Determining Communication Topology for LLM-based Multi-agent System Paper · Task-adaptive graph routing	EMNLP Industry Track Hui Yi Leong; Yuheng Li; Yuqing Wu; Wenwen Ouyang; Wei Zhu; Jiechao Gao; Wei Han · 2025	Introduces a dynamic graph selector that uses lightweight LLM adaptation to choose task-specific communication structures and route each query through a matching agent pathway. Why: Operationalizes topology as an input-conditioned decision rather than a fixed template; evidence is currently concentrated in question answering, mathematics, and code generation.	Peer-reviewed research Evolution
📄 MasRouter: Learning to Route LLMs for Multi-Agent Systems Paper · Collaboration-mode, role, and model routing	ACL Yanwei Yue; Guibin Zhang; Boyang Liu; Guancheng Wan; Kun Wang; Dawei Cheng; Yiyan Qi · 2025	Defines multi-agent system routing as a joint decision over collaboration mode, role allocation, and language-model selection, implemented with a cascaded controller that progressively constructs a task-specific system. Why: Makes node roles, model assignment, and whether collaboration is needed explicit routing decisions with measurable cost-quality trade-offs.	Peer-reviewed research Evolution
📄 MOC: Multi-Order Communication in LLM-based Multi-Agent Systems Paper · Multi-hop evidence propagation	ICML Yao Guan; Lin Wang; Zhihui Lu; Ziyi Wang; Wenzhu Yan; Qiang Duan · 2026	Constructs structured multi-order evidence streams so agents can receive relevant information from multiple upstream hops, then applies semantic-topological merging under token constraints. Why: Shows that graph engineering includes what evidence survives across paths, not only which nodes and edges exist; it improves communication over a supplied topology rather than selecting that topology.	Peer-reviewed research Handoffs
📄 HieraMAS: Optimizing Intra-Node LLM Mixtures and Inter-Node Topology for Multi-Agent Systems Paper · Hierarchical node and topology optimization	ICML Tianjun Yao; Zhaoyi Li; Zhiqiang Shen · 2026	Models each functional role as a supernode containing heterogeneous language models in a propose-synthesis structure, then uses multi-level reward attribution and graph classification to select the inter-supernode topology. Why: Jointly engineers node composition, role capability, credit assignment, and communication edges instead of optimizing each dimension independently.	Peer-reviewed research Evolution
📄 AIPOM: Agent-aware Interactive Planning for Multi-Agent Systems Paper · Human-in-the-loop work-graph planning	EMNLP System Demonstrations Hannah Kim; Kushan Mitra; Chen Shen; Dan Zhang; Estevam Hruschka · 2025	Presents conversational and graph-based interfaces for inspecting, editing, and collaboratively guiding plans in orchestrated multi-agent systems. Why: Treats the work graph as a human-legible control surface; its non-autonomous agents execute assigned tasks, so it is a planning substrate rather than a complete agent organization.	Peer-reviewed research Work graphs
📄 AgentNet: Decentralized Evolutionary Coordination for LLM-based Multi-Agent Systems Paper · Decentralized adaptive coordination	NeurIPS Yingxuan Yang; Huacan Chai; Shuai Shao; Yuanyi Song; Siyuan Qi; Renting Rui; Weinan Zhang · 2025	Organizes specialized agents in a decentralized DAG, uses retrieval-backed local experience for capability refinement, and adapts routing without a single central orchestrator. Why: Offers a contrasting topology for settings where central coordination is a bottleneck or trust boundary; its reported evidence is concentrated in reasoning and coding tasks.	Peer-reviewed research Topology
📄 Grammar Search for Multi-Agent Systems Paper · Grammar-based system search	ACL Mayank Singh; Vikas Yadav; Shiva Krishna Reddy Malay; Shravan Nayak; Sai Rajeswar; Sathwik Tejaswi Madhusudhan; Eduardo Blanco · 2026	Searches a fixed grammar of composable multi-agent components to construct modular, interpretable organizations at lower search cost than unconstrained code generation. Why: Makes the agent-system design space explicit and auditable while retaining automated search over roles, operators, and coordination structure.	Peer-reviewed research Evolution
📄 Graph-GRPO: Stabilizing Multi-Agent Topology Learning via Group Relative Policy Optimization Paper · Topology policy optimization	Findings of ACL Yueyang Cang; Xiaoteng Zhang; Erlu Zhao; Zehua Ji; Yuhang Liu; Yuchen He; Zhiyuan Ning; Chen Yijun; Wenge Que; Li Shi · 2026	Samples groups of query-specific communication graphs and uses relative rewards to assign edge-level credit while training a topology-generating policy. Why: Addresses unstable topology learning with explicit graph samples and localized credit rather than treating orchestration as an opaque prompt update.	Peer-reviewed research Evolution
📄 TopoDIM: One-shot Topology Generation of Diverse Interaction Modes for Multi-Agent Systems Paper · One-shot topology generation	Findings of ACL Rui Sun; Jie Ding; Chenghua Gong; Tianjun Gu; Yihang Jiang; Juyuan Zhang; Liming Pan; Linyuan Lü · 2026	Generates heterogeneous decentralized communication topologies in one pass, replacing repeated coordination rounds with a task-conditioned interaction structure. Why: Expands topology design beyond homogeneous or fully connected teams while directly evaluating the quality and token cost of the generated graph.	Peer-reviewed research Topology
📄 EvoHyper: Evolving Hypergraph Topologies for Unified Collaboration in Multi-Agent Communication Paper · Evolving hypergraph collaboration	Findings of ACL Heng Zhang; Yihao Zhong; Lubin Gan; Zhihe Chen; Jiajun Wu; Yuling Shi; Xiaodong Gu; Hao Zhang; Haochen You; Jin Huang · 2026	Uses evolving hyperedges to bind groups of agents to shared memory while a controller creates, updates, and merges collaboration units during execution. Why: Models group communication as a first-class structure and joins topology evolution with explicit shared-state ownership instead of reducing every exchange to pairwise chat.	Peer-reviewed research Evolution
📄 RouterHGC: Optimized Router for LLM-based Multi-Agent Systems via Heterogeneous Graph Contrastive Learning Paper · Heterogeneous graph routing	Findings of ACL Yitao Xiao; Shaoyong Guo; Guoming Yang; Qingnan Wang; Yinlin Ren; Xuesong Qiu; Qi Feng · 2026	Represents queries, collaboration modes, roles, and language models as heterogeneous nodes and learns cost-performance routing relationships with graph contrastive learning. Why: Treats the agent organization itself as the graph under optimization and exposes model, role, mode, and budget choices within one routing decision.	Peer-reviewed research Evolution
📄 LLM-as-Scheduler: Agentic Workflow Dynamic Scheduling Paper · Dynamic workflow scheduling	ACL Dawei Xiang; Kexin Chu; Wenyan Xu; Wenhui Zhang; Wei Zhang · 2026	Dynamically selects query-specific paths through a workflow DAG, including early exit, verification, repair, and rerouting decisions made during execution. Why: Separates the reusable work graph from the runtime path and supplies concrete control points for budgets, validation, and local recovery.	Peer-reviewed research Work graphs
📄 Learning Optimal Message Representations for Agentic Communication Paper · Adaptive message representation	Findings of ACL Shashwat Gupta; Anson Bastos; Mayukh Das; Supriyo Ghosh; Nagarajan Natarajan; Chetan Bansal; Saravan Rajmohan · 2026	Models the choice of inter-agent message representation as an expanding Markov decision process that can select natural-language or structured protocols at each exchange. Why: Makes payload format an adaptive edge policy rather than a global convention, clarifying when structure, compression, or expressiveness should dominate.	Peer-reviewed research Handoffs
📄 Graph-of-Agents: A Graph-based Framework for Multi-Agent LLM Collaboration Paper · Relevance-ordered message passing	ICLR Sukwon Yun; Jie Peng; Pingzhi Li; Wendong Fan; Jie Chen; James Y. Zou; Guohao Li; Tianlong Chen · 2026	Selects relevant agents from model cards, constructs directed edges from response relevance, performs forward and reverse message passing, and pools the revised answers. With a pool of six LLMs, three selected agents outperform baselines that use all six across MMLU, MMLU-Pro, GPQA, MATH, HumanEval, and MedMCQA. Why: Offers an end-to-end graph construction and aggregation method in which node selection, edge order, and communication direction are explicit design choices.	Peer-reviewed research Topology
📄 Stochastic Self-Organization in Multi-Agent Systems Paper · Response-conditioned self-organization	ICLR Nurbek Tastan; Samuel Horváth; Karthik Nandakumar · 2026	Builds a response-conditioned directed acyclic graph from approximate peer-contribution scores and updates the communication structure across collaboration rounds. Why: Adapts topology from observed agent behavior without a separate judge, pretrained generator, or fixed graph, while retaining stable acyclic message flow.	Peer-reviewed research Evolution
📄 GraphPlanner: Graph Memory-Augmented Agentic Routing for Multi-Agent LLMs Paper · Memory-augmented agent routing	ICLR Tao Feng; Haozhen Zhang; Zijie Lei; Peixuan Han; Jiaxuan You · 2026	Generates query-specific workflows by selecting both language-model backbones and agent roles while consulting a heterogeneous graph of query-agent-response history. Why: Combines work-graph routing with reusable interaction memory and evaluates unseen-task and unseen-model generalization rather than only in-distribution quality.	Peer-reviewed research Evolution
📄 MPAS: Breaking Sequential Constraints of Multi-Agent Communication Topologies via Individual-Epistemic Message Propagation Paper · Parallel message propagation	AAAI Jingxuan Yu; Ju Jia; Simeng Qin; Xiaojun Jia; Siqi Ma; Yihao Huang; Yali Yuan; Guang Cheng · 2026	Reworks sequential multi-agent communication into parallel graph message propagation with optimized connections and aggregation while preserving agent-specific epistemic state. Why: Targets latency introduced by dependency-ordered exchanges and makes the tradeoff between parallel execution, aggregation, and resilience measurable.	Peer-reviewed research Topology
📄 HyperAgent: Leveraging Hypergraphs for Topology Optimization in Multi-Agent Communication Paper · Hypergraph topology optimization	AAMAS Heng Zhang; Yuling Shi; Xiaodong Gu; Zijian Zhang; Haochen You; Lubin Gan; Yilei Yuan; Jin Huang · 2026	Represents collaboration groups as hyperedges and uses a variational hypergraph autoencoder with sparsity regularization to generate task-adaptive communication topology. An earlier arXiv version was withdrawn by the authors in December 2025 citing a fundamental methodology error, and revised in February 2026; cite the current version. Why: Adds group-level topology learning beyond pairwise edges and differs from evolving-hypergraph work by focusing on generated sparse structures for each task.	Peer-reviewed research Evolution
📄 D³MAS: Decompose, Deduce, and Distribute for Enhanced Knowledge Sharing in Multi-Agent Systems Paper · Three-layer knowledge sharing	AAMAS Heng Zhang; Yuling Shi; Xiaodong Gu; Haochen You; Zijian Zhang; Lubin Gan; Yilei Yuan; Jin Huang · 2026	Connects task decomposition, collaborative reasoning, and distributed memory as coordinated layers in a heterogeneous graph designed to reduce duplicated retrieval, inference, and assignment. An earlier arXiv version was withdrawn by the authors in December 2025 citing a fundamental methodology error, and revised in February 2026; cite the current version. Why: Unifies work, reasoning, and state dependencies in one inspectable organization rather than optimizing those coordination surfaces separately.	Peer-reviewed research Work graphs
📄 Multi-Agent Collaboration via Evolving Orchestration Paper · Reinforced evolving orchestration	NeurIPS Yufan Dang; Chen Qian; Xueheng Luo; Jingru Fan; Zihao Xie; Ruijie Shi; Weize Chen; Cheng Yang; Xiaoyin Che; Ye Tian; Xuantang Xiong; Lei Han; Zhiyuan Liu; Maosong Sun · 2025	Trains a centralized orchestrator with reinforcement learning to select and sequence agents from evolving task state, producing dynamic inference graphs and compact cyclic reasoning paths. Why: Learns runtime activation and ordering rather than fixing a workflow in advance, providing a centralized contrast to decentralized evolutionary coordination.	Peer-reviewed research Evolution
📄 Stop Hand-Holding Your Coding Agent: Engineering the Loops that Replace Step-by-Step Prompting Paper · Bridging research	arXiv Sandeco Macedo · 2026	First academic treatment of the loop-engineering discourse itself (June 28, 2026): formalizes the loop specification as a bounded reusable artifact — trigger, goal, verification, stopping rule, memory — with a taxonomy drawn from fifty real-world implementations. Defines the node-level unit that graph engineering composes into organizations. Why: Formalizes the loop as a bounded, reusable artifact, which is the unit a graph node encapsulates.	Research preprint Work graphs
📐 FIPA Agent Communication Language Specifications Standard · Classic MAS lineage	fipa.org Foundation for Intelligent Physical Agents (IEEE) · 2002	The classic lineage of every modern handoff protocol: speech-act-theoretic messages with 22 performatives (inform, request, propose, ...), each with feasibility preconditions and rational effects, plus interaction protocols like Contract Net for task delegation by bidding. Standardized 2002, inherited by IEEE in 2005 — the 20-year-old prior art for agent cards, task delegation, and typed message contracts. Why: The speech-act lineage behind every modern handoff protocol, worth reading before reinventing its performatives.	Industry standard Handoffs
🧰 CAMEL Tool · Agent societies	GitHub CAMEL-AI.org (Guohao Li et al.) · 2026	The earliest role-playing multi-agent LLM framework (2023 paper), now an active open-source ecosystem explicitly chasing 'the scaling laws of agents' — how behavior changes as agent societies grow. Provides role-playing and workforce modules for building agent societies, and underpins downstream projects like OWL and OASIS million-agent simulations. Why: The origin of role-playing agent societies, and the paper most later role designs implicitly inherit from.	Maintained OSS project Roles
📄 Measuring and Mitigating Identity Bias in Multi-Agent Debate via Anonymization Paper · Anti-groupthink gates	arXiv (see arXiv listing) · 2025	Shows debate agents weight WHO said something over WHAT was said — with conformity (sycophancy) generally exceeding obstinacy — causing premature consensus that reinforces wrong answers, and demonstrates response anonymization as a concrete fix. Direct design guidance for council nodes: strip author identity from edges before deliberation. Why: Debate agents weighting who spoke over what was said is a direct threat to any verification built on discussion.	Research preprint Gates
📄 Too Polite to Disagree: Understanding Sycophancy Propagation in Multi-Agent Systems Paper · Anti-groupthink gates	arXiv (see arXiv listing) · 2026	Documents how sycophancy propagates through agent organizations: instead of correcting each other, agents amplify each other's agreement patterns into collective conformity that entrenches errors. A 2026 empirical warning that stacking agreeable agents in a council produces confident consensus, not error correction — and why deliberation topologies need dissent mechanisms. Why: Documents sycophancy propagating through an organization, which is why agreement between nodes is weak evidence.	Research preprint Gates
🧰 llm-council: A Council of LLMs with a Chairman Tool · Council/committee design	GitHub Andrej Karpathy · 2025	Karpathy's minimal working council: a query fans out to a fixed panel of frontier models (GPT-5.1, Gemini 3 Pro, Claude Sonnet 4.5, Grok 4 via OpenRouter), members review and rank each other's anonymized answers to prevent favoritism, and a Chairman synthesizes the final response. A weekend hack that became the reference implementation of committee-of-rivals topology with a built-in anti-bias gate. Why: A minimal working council with a chairman, useful as the smallest honest implementation of panel verification.	Maintained OSS project Gates
📄 MegaAgent: Autonomous Large-Scale LLM Agent Systems Without Predefined SOPs Paper · Dynamic org generation	arXiv / ACL 2025 Findings Qian Wang et al. · 2024	Where MetaGPT hand-writes the org chart, MegaAgent generates it: agents recursively spawn subordinate groups sized to task complexity, with hierarchical coordination, parallel execution, and system-level monitoring but no predefined SOPs. Scaled to 590 agents in a national-policy simulation and built a working Gobang game in ~800 seconds — an early concrete instance of the org graph rewriting itself on evidence. Why: Generating the org chart rather than writing it is the frontier claim, and this is a concrete attempt at it.	Peer-reviewed research Evolution
📝 Reverse Conway's Law and Agents Blog · Org theory / Conway's law	jedi.be Patrick Debois · 2025	Debois (co-originator of DevOps) inverts Conway for the agent era: instead of asking how org structure shapes agent systems, he asks how embedding capable agents reshapes the human organization itself — with an accompanying AI Engineer conference talk. Short, but the sharpest statement of the bidirectional Conway question for agent orgs. Why: Inverting a well-known law for the agent era reframes org design as a choice rather than an inheritance.	Practitioner analysis Roles
📄 The Coasean Singularity? Demand, Supply, and Market Design with AI Agents Paper · Org theory / agent economics	NBER NBER (The Economics of Transformative AI volume) · 2025	NBER chapter examining what happens to markets and organizational boundaries as agents drive transaction costs toward zero — instant preference elicitation, automated contracting — while creating new costs around trust and verification. The rigorous economics counterpart to practitioner essays on whether to organize agents as a 'firm' (org graph) or a market. Why: Asks what happens to organizational boundaries when transaction costs collapse, which is the economics behind agent staffing.	Research preprint Roles
📄 From Coase to AI Agents: Why the Economics of the Firm Still Matters in the Age of Automation Paper · Org theory / firm boundaries	California Management Review California Management Review · 2025	Applies Coase's 1937 transaction-cost theory of the firm to agents: if AI slashes the costs of search, negotiation, and monitoring, the efficient boundary between doing work inside an agent organization versus buying it from outside shifts — and may shrink firms. The clearest bridge from classical org economics to the question of where an agent org's edges should stop. Why: Applies transaction-cost theory to agents, giving a principled answer to which work belongs inside one organization.	Research preprint Roles
📝 The Future of Team Topologies: When AI Agents Dominate Blog · Org theory / team design	teamtopologies.com Andy Spamer (Team Topologies) · 2025	The Team Topologies organization asks how its own framework survives teams that are 50-90% agents: Dunbar's-number limits on team size dissolve, communication has no social cost, and Conway's law must be re-read when the 'communicators' are programmable. First-party org-theory engagement with agent teams from the people who wrote the standard book on human team design. Why: A widely used team-design framework confronting mostly-agent teams, useful for anyone mapping human structures onto graphs.	Practitioner analysis Roles
📄 Examining Identity Drift in Conversations of LLM Agents Paper · Role drift	arXiv Junhyuk Choi et al. · 2024	Empirical study of role drift across nine LLMs: agent interaction styles and identities shift over multi-turn conversations, larger models drift MORE, and assigning a persona does not reliably anchor identity. The core evidence base for why stable org-graph roles need active maintenance (persona re-injection, memory refresh) rather than a one-time system prompt. Why: Identity drift over long interactions is why a role needs enforcement, not just a good initial description.	Research preprint Roles
📝 Subagents — Agentic Engineering Patterns Blog · Role specialization practice	simonwillison.net Simon Willison · 2026	Chapter of Willison's 'Agentic Engineering Patterns' guide on dispatching fresh specialist instances — code reviewers, debuggers, explorers — so the parent agent's token budget and context stay clean. A concise, vendor-neutral treatment of why specialist roles with isolated context beat one generalist loop, with parallel-subagent performance notes. Why: Dispatching fresh specialists with clean context is the most reusable pattern for keeping node context clean.	Practitioner analysis Roles
📝 The Fable 5 Orchestrator Playbook: One Smart Model Managing Cheap Workers Blog · Staffing economics (model tiers)	developersdigest.tech Developers Digest · 2026	Worked cost math for staffing org-graph nodes by model tier: frontier model for planning/dispatch/synthesis where errors compound, Sonnet-class for implementation, Haiku-class for read-only scouting. A 12-worker codebase audit runs $3.70 with cheap workers vs $14.50 all-frontier (74% savings), with the key asymmetry stated plainly — downgrading workers saves far more than downgrading the orchestrator. Includes honest counter-cases (shallow fan-outs, latency-sensitive flows). Why: Worked cost arithmetic for mixed-tier staffing, which turns a staffing intuition into a number you can check.	Practitioner analysis Observability & cost
📄 The Hearsay-II Speech-Understanding System: Integrating Knowledge to Resolve Uncertainty Paper · Blackboard classics	ACM Computing Surveys Lee Erman, Frederick Hayes-Roth, Victor Lesser, Raj Reddy · 1980	The original blackboard system (CMU, 1971-76): independent knowledge sources cooperate solely by reading and writing a shared hierarchical blackboard, with opportunistic scheduling deciding who acts next. Every modern shared-scratchpad multi-agent design descends from this paper; free PDF at mas.cs.umass.edu/Documents/Erman_Hearsay80.pdf. Why: The original shared-state architecture, where cooperation happens entirely through reads and writes to a common structure.	Peer-reviewed research State
📄 Exploring Advanced LLM Multi-Agent Systems Based on Blackboard Architecture Paper · Blackboard revival	arXiv Bochen Han, Songmao Zhang · 2025	First implementation of a full blackboard architecture for LLM agents: all roles share one public blackboard, a control unit selects which agent acts based on blackboard content, and rounds repeat until consensus. Matches state-of-the-art master-slave systems on reasoning and math benchmarks while spending fewer tokens — evidence that shared state can beat message passing on cost. Why: A full shared-state architecture for language agents, testing whether the classical idea survives modern nodes.	Research preprint State
📄 LLM-based Multi-Agent Blackboard System for Information Discovery in Data Science Paper · Blackboard revival	arXiv Aoxue Li et al. · 2025	Replaces the master-slave orchestration paradigm with a blackboard: a central agent posts requests to a shared board and subordinate agents volunteer answers, so the orchestrator never needs to know which agent holds what. 13-57% relative improvement in end-to-end task success over direct-communication baselines on data-discovery tasks. Why: Replacing central orchestration with a shared board changes who decides what happens next, which is a topology claim.	Research preprint State
📄 SagaLLM: Context Management, Validation, and Transaction Guarantees for Multi-Agent LLM Planning Paper · Consistency & transactions	arXiv Edward Y. Chang, Longling Geng (Stanford) · 2025	Imports the Saga transaction pattern from distributed databases into agent graphs: persistent cross-agent memory, independent validation agents, checkpointing, and compensating actions so a partially-failed multi-agent plan rolls back consistently instead of leaving corrupted shared state. The most direct treatment of the two-agents-one-resource consistency problem in the literature. Why: Brings transaction guarantees to agent state, addressing the partial-write problem that breaks shared memory in practice.	Research preprint Reliability
📄 MemGPT: Towards LLMs as Operating Systems Paper · Memory systems	arXiv Charles Packer, Sarah Wooders, et al. (UC Berkeley) · 2023	The paper that started OS-inspired agent memory: virtual context management pages data between the fixed context window (main memory) and external storage, with the agent editing its own memory via tools. Direct ancestor of Letta and of the memory-block abstraction now used for multi-agent shared state. Why: The paper that started operating-system framing for agent memory, and the ancestor of most current designs.	Research preprint State
📄 Multi-Agent Systems: From Classical Paradigms to Large Foundation Model-Enabled Futures Paper · Classical MAS bridge	arXiv / IEEE-CAA Journal of Automatica Sinica Zixiang Wang, Mengjia Gong, Qiyu Sun, et al. · 2026	April 2026 survey bridging classical MAS theory (consensus, formation control, distributed decision-making) and LLM-era agent systems, contrasting them across architecture, operating mechanism, and adaptability. Useful precisely because graph engineering keeps reinventing 30 years of MAS coordination results under new names — this connects the two literatures. Why: Bridges classical coordination theory and current practice, which is the shortest route to not reinventing solved problems.	Peer-reviewed research Topology
📄 Exchange-of-Thought: Enhancing LLM Capabilities through Cross-Model Communication Paper · Communication paradigms	arXiv / EMNLP 2023 Zhangyue Yin, Qiushi Sun, et al. · 2023	Explicitly imports network topology into LLM communication design with four paradigms — Memory (bus), Report (star), Relay (ring), Debate (tree) — plus a confidence mechanism to dampen wrong reasoning propagation. An early, still-cited demonstration that communication shape, not just agent count, determines collective reasoning quality. Why: Imports explicit network topologies into agent communication, making the shape of discussion a design variable.	Peer-reviewed research Topology
📄 ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate Paper · Debate for evaluation	arXiv / ICLR 2024 Chi-Min Chan, Weize Chen, et al. · 2023	Applies debate topology to evaluation: a referee team of persona-diverse agents with distinct communication strategies (one-by-one, simultaneous, summarizer variants) converges on judgments better aligned with humans than single-judge LLMs. The standard reference for judge-panel org design in agent systems. Why: Applies debate structure to evaluation itself, with persona-diverse referees rather than a single judge.	Peer-reviewed research Gates
📄 Dynamic LLM-Agent Network (DyLAN) Paper · Dynamic agent networks	arXiv / COLM 2024 Zijun Liu, Yanzhe Zhang, Peng Li, Yang Liu, Diyi Yang · 2023	Models collaboration as a temporal feed-forward network over agents, with an LLM ranker deactivating low performers mid-task and an unsupervised Agent Importance Score selecting the team before solving. One of the earliest demonstrations that the work graph should be rewritten on evidence during execution rather than fixed up front; team selection alone improves accuracy by up to 25.0% on some MMLU subjects. Why: Deactivating low-contribution agents mid-run makes the working topology a function of observed performance.	Peer-reviewed research Evolution
📄 AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors Paper · Dynamic team composition	arXiv / ICLR 2024 Weize Chen, Yusheng Su, et al. (Tsinghua) · 2023	Framework and study of dynamically re-composed agent teams via an expert-recruitment → collaborative-decision → action → evaluation loop, with documented emergent social behaviors (volunteering, conformity, destructive behaviors) as team structure changes. Early evidence for the org-graph/work-graph distinction: the team roster itself is an optimizable, per-task variable. Why: Dynamic team re-composition per round is an early, readable version of evidence-driven staffing.	Peer-reviewed research Evolution
📄 AgentConductor: Topology Evolution for Multi-Agent Competition-Level Code Generation Paper · Dynamic topology	arXiv (February 2026 preprint) · 2026	An RL-optimized orchestrator agent that dynamically generates density-aware, layered DAG topologies per task from end-to-end execution feedback, evolving the org graph during multi-round code generation. Up to +14.6 points pass@1 over the strongest baseline with 68% token-cost reduction on competition-level coding — a concrete 2026 instance of a work graph that rewrites itself on evidence. Why: A reinforcement-trained orchestrator generating layered task graphs, aimed at competition-level difficulty.	Research preprint Evolution
📄 More Agents Is All You Need (Agent Forest) Paper · Ensemble scaling baseline	arXiv / TMLR Junyou Li, Qin Zhang, Yangbin Yu, Qiang Fu, Deheng Ye (Tencent) · 2024	Shows that plain sampling-and-voting scales LLM performance with agent count, with gains correlated to task difficulty — no communication topology at all. Essential null-hypothesis reading: any nontrivial org graph must beat this embarrassingly parallel baseline to justify its coordination cost. Why: The plain sampling-and-voting baseline that many elaborate topologies fail to beat, and therefore must be compared against.	Peer-reviewed research Topology
📄 Why Do Multi-Agent LLM Systems Fail? (MAST) Paper · Failure diagnosis	arXiv / ICLR 2026 Mert Cemri, Melissa Z. Pan, Shuyi Yang, et al. (UC Berkeley) · 2025	First empirically grounded failure taxonomy for multi-agent systems: 14 failure modes in 3 classes (specification issues, inter-agent misalignment, task verification) derived from 1,600+ annotated traces across 7 frameworks with kappa 0.88. The debugging vocabulary for agent organizations — most documented failures are org-design failures, not model failures. Why: Turns reliability incidents into recurring, attributable failure classes that can guide graph redesign.	Peer-reviewed research Reliability
📄 MAS-GPT: Training LLMs to Build LLM-based Multi-Agent Systems Paper · Generative MAS construction	arXiv / ICML 2025 Rui Ye, et al. (SJTU / Shanghai AI Lab) · 2025	Trains an LLM that emits a complete executable multi-agent system (roles, topology, prompts) as ordinary Python in one forward pass per query, turning MAS construction into a generative language task. Outperforms 10+ adaptive baselines across 9 benchmarks while being drastically cheaper at design time than search-based methods like ADAS or AFlow. Why: Emitting an entire executable multi-agent system as ordinary output turns architecture into a generation task.	Peer-reviewed research Evolution
📄 Heterogeneous Swarms: Jointly Optimizing Model Roles and Weights for Multi-LLM Systems Paper · Joint role-weight optimization	arXiv / NeurIPS 2025 Shangbin Feng, et al. (UW / Google) · 2025	Represents a multi-LLM system as a DAG and jointly optimizes the graph (which model feeds which) via particle swarm optimization over continuous adjacency matrices, and the models' weights themselves. Beats 15 role- or weight-only baselines by 18.5% across 12 tasks — evidence that org topology and member capability should be co-optimized, not chosen sequentially. Why: Jointly optimizing which model feeds which and the weights themselves, the most integrated version of staffing plus structure.	Peer-reviewed research Evolution
📄 Mixture-of-Agents Enhances Large Language Model Capabilities Paper · Layered aggregation	arXiv / ICLR 2025 Junlin Wang, Jue Wang, Ben Athiwaratkun, Ce Zhang, James Zou (Together AI) · 2024	The layered-topology result: stack agents in layers where each agent takes all previous-layer outputs as auxiliary context. Using only open-source models, MoA topped GPT-4o on AlpacaEval 2.0, making it the most-deployed 'topology as ensembling' pattern and a standard baseline in every subsequent topology-search paper. Why: Layered aggregation of many agents' outputs is a simple topology with unusually strong reported results.	Peer-reviewed research Topology
📄 GoAgent: Group-of-Agents Communication Topology Generation for LLM-based Multi-Agent Systems Paper · Learned topology generation	arXiv (March 2026 preprint) · 2026	Treats collaborative groups — not individual agents — as the atomic unit of topology construction: an LLM enumerates task-relevant candidate groups, then autoregressively connects them, with a conditional information bottleneck compressing inter-group messages. 93.84% average accuracy across six benchmarks with ~17% token reduction; state of the topology-generation art as of mid-2026. Why: Treating groups rather than individuals as the atomic unit is a genuinely different way to construct topology.	Research preprint Topology
📄 Rethinking Multi-Agent Intelligence Through the Lens of Small-World Networks Paper · Network-science priors	arXiv Boxuan Wang, Zhuoyun Li, Xiaowei Huang, Yi Dong · 2025	Imports Watts-Strogatz small-world theory into agent org design: topologies balancing local clustering with a few long-range shortcuts match fully-connected debate accuracy at similar token cost while substantially stabilizing consensus dynamics. A rare paper connecting agent topology to decades of complex-networks and neuroscience results. Why: Small-world structure balances local clustering against global reach, giving topology design a principled starting shape.	Research preprint Topology
📄 FlowReasoner: Reinforcing Query-Level Meta-Agents Paper · Query-adaptive MAS generation	arXiv Hongcheng Gao, Yue Liu, et al. (NUS / Kling) · 2025	A reasoning meta-agent (distilled from DeepSeek-R1, then RL-trained on execution feedback with a multi-purpose reward for performance, complexity, and efficiency) that designs a bespoke multi-agent system for each user query. Beats o1-mini by 10.52% on code benchmarks; representative of the shift from task-level to query-level org design. Why: A meta-agent that reasons about workflow structure per query, trained on execution feedback rather than heuristics.	Research preprint Evolution
📄 CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society Paper · Role-playing foundations	arXiv / NeurIPS 2023 Guohao Li, Hasan Hammoud, Hani Itani, Dmitrii Khizbullin, Bernard Ghanem · 2023	The March-2023 origin of LLM role-playing societies: two inception-prompted agents (user + assistant) cooperate on tasks with minimal human intervention, establishing the dyad as the atomic edge of agent organizations. Seeded the CAMEL-AI ecosystem (later OWL and Workforce) and is the most-cited ancestor of the whole field. Why: The origin of paired role-playing agents, and the source of the inception-prompt pattern still used to hold roles stable.	Peer-reviewed research Roles
📄 ChatDev: Communicative Agents for Software Development Paper · SOP-structured orgs	arXiv / ACL 2024 Chen Qian, Wei Liu, et al. (Tsinghua) · 2023	Runs a virtual software company on a 'chat chain' — a linear graph of phase-scoped dyadic conversations — plus communicative dehallucination where agents request detail before answering. With its successor MacNet, it forms the clearest research arc from fixed pipeline org to generalized graph-organized collaboration. Why: A linear chain of phase-scoped conversations, the clearest early demonstration that process shape drives output quality.	Peer-reviewed research Topology
📄 MetaGPT: Meta Programming for a Multi-Agent Collaborative Framework Paper · SOP-structured orgs	arXiv / ICLR 2024 Sirui Hong, Mingchen Zhuge, et al. · 2023	Encodes human Standardized Operating Procedures into an assembly-line agent org: role-specialized agents (PM, architect, engineer, QA) pass structured artifacts rather than free chat, an early instance of typed edges in an org graph. The paper side of the 45k+-star framework whose team later produced AFlow and the FoundationAgents stack. Why: Standard operating procedures as topology, and the most-copied blueprint for assembly-line agent organizations.	Peer-reviewed research Roles
📄 Large Language Model based Multi-Agents: A Survey of Progress and Challenges Paper · Survey	arXiv / IJCAI 2024 Taicheng Guo, Xiuying Chen, et al. · 2024	The standard early survey of LLM multi-agent systems, organizing the field by agent-environment interface, profiling, communication structure (layered, decentralized, centralized, shared message pool), and capability acquisition. Still the fastest map of the 2023-24 foundations under the 2026 graph-engineering layer. Why: The standard early survey, useful for placing any newer proposal against what was already tried.	Peer-reviewed research Topology
📄 Multi-Agent Collaboration Mechanisms: A Survey of LLMs Paper · Survey	arXiv Khanh-Tung Tran, Dung Dao, Minh-Duong Nguyen, Quoc-Viet Pham, Barry O'Sullivan, Hoang D. Nguyen · 2025	Characterizes collaboration along five dimensions — actors, types (cooperation/competition/coopetition), structures (peer-to-peer, centralized, distributed), strategies, and coordination protocols — with an extensible framework for composing them. The closest thing to a design-space catalog for choosing org-graph edge semantics, not just edge existence. Why: Characterizes collaboration along five dimensions, which gives a vocabulary for comparing designs that look superficially alike.	Research preprint Topology
📄 LLM-Based Multi-Agent Orchestration: A Survey of Frameworks, Communication Protocols, and Emerging Patterns Paper · Topology taxonomy survey	Future Internet (Future Internet 18(6), 326) · 2026	June 2026 survey (literature cutoff March 2026) proposing a three-topology taxonomy — centralized, decentralized, hierarchical — plus an orthogonal dynamic-adaptivity axis, with a practical decision framework keyed to task structure, agent count, fault tolerance, and cost budget. Also compares LangGraph, CrewAI, AutoGen/Microsoft Agent Framework, OpenAI Agents SDK, MetaGPT, and DSPy, and maps the MCP/A2A/ANP protocol stack onto coordination layers. Why: A recent survey with a clean centralized, decentralized, and hybrid taxonomy, current enough to be useful for decisions.	Peer-reviewed research Topology
📄 Training Verifiers to Solve Math Word Problems (GSM8K) Paper · Verifier models as gates	arXiv Karl Cobbe, Vineet Kosaraju, et al. (OpenAI) · 2021	The 2021 origin of generate-then-verify: training a separate verifier model to rank sampled solutions beats simply scaling the generator. Introduced GSM8K and the architectural insight — a dedicated verifier node downstream of the generator — that graph engineering inherits. Why: The origin of generate-then-verify, and the reason a separate verifier is a standard component rather than an add-on.	Research preprint Gates
🧭 12-Factor Agents List · Control-flow ownership	GitHub Dex Horthy (HumanLayer) · 2025	Production principles for LLM software whose Factor 8, 'Own your control flow,' is the core argument for deterministic-code orchestration: the model chooses actions, but application code owns the loop, stop conditions, retries, and approval gates. Widely cited (HN front page, 10k+ stars) as the counterweight to letting the model drive the whole graph. Why: Owning your control flow is the principle that decides how much of a work graph belongs in code rather than in a prompt.	Community resource Work graphs
📄 GAP: Graph-Based Agent Planning with Parallel Tool Use and Reinforcement Learning Paper · Dependency inference	arXiv GAP authors · 2025	Trains an agent to explicitly infer sub-task dependencies as a DAG — absent edges mean parallel execution is safe — combining supervised fine-tuning on graph-structured plans with reinforcement learning for adaptive serial/parallel tool scheduling. Representative of the newest wave: dependency inference as a learned capability rather than a prompt trick. Why: Inferring subtask dependencies as an explicit graph is what makes safe parallelism a derived property rather than a guess.	Research preprint Work graphs
📄 TDAG: A Multi-Agent Framework based on Dynamic Task Decomposition and Agent Generation Paper · Dynamic agent spawning	arXiv Yaoxiang Wang, Zhiyong Wu, et al. · 2024	Dynamically decomposes a complex task into subtasks and generates a purpose-built subagent for each — node spawning driven by the evolving decomposition rather than a fixed roster. Later published in Neural Networks (2025); ships ItineraryBench for fine-grained evaluation of interconnected progressive tasks. Why: Generating a purpose-built subagent per subtask ties role creation directly to decomposition.	Peer-reviewed research Work graphs
📄 From Agent Loops to Deterministic Graphs: Execution Lineage for Reproducible AI-Native Work Paper · Loops vs graphs	arXiv Josh Rosen, Seth Rosen · 2026	May 2026 preprint arguing that representing agent work as explicit DAGs with stable intermediate artifacts and identity-based replay beats conversational loops for maintaining work products over time. Experiments on document-revision tasks show DAG replay achieves zero churn and zero unrelated-branch contamination versus loop baselines. Why: Argues explicit graphs with stable intermediate artifacts make agent work reproducible, which is the reliability case for structure.	Research preprint Work graphs
📄 From Agent Loops to Structured Graphs: A Scheduler-Theoretic Framework for LLM Agent Execution Paper · Loops vs graphs	arXiv Hu Wei · 2026	April 2026 position paper applying classical scheduling theory to agent execution, diagnosing the agent loop's implicit step dependencies, unbounded recovery cycles, and mutable histories. Proposes a Structured Graph Harness with static DAG control flow, immutable plans, separated planning/recovery layers, and formal escalation, grounded in an analysis of 70 systems. Why: Applies scheduling theory to agent execution, diagnosing implicit dependencies and unbounded recovery precisely.	Research preprint Work graphs
📄 Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning Paper · Orchestrator prompting	arXiv Lei Wang et al. · 2023	ACL 2023 paper showing that prompting a model to first devise a subtask plan and then execute it fixes the missing-step errors of zero-shot chain-of-thought. The minimal ancestor of every plan-then-execute orchestrator prompt. Why: The minimal decompose-then-execute result, and the baseline any elaborate planner should be measured against.	Peer-reviewed research Work graphs
📄 An LLM Compiler for Parallel Function Calling (LLMCompiler) Paper · Parallel DAG planning	arXiv Sehoon Kim et al. (SqueezeAILab, UC Berkeley) · 2024	ICML 2024 paper that treats the LLM as a compiler: a Planner emits a DAG of function-call tasks with placeholder variables, a Task Fetching Unit dispatches ready nodes in parallel, and an Executor runs them, unblocking dependents as results arrive. Reports up to 3.7x latency speedup, 6.7x cost savings, and ~9% higher accuracy than ReAct; the archetype of work-graph generation for tool calls (code: github.com/SqueezeAILab/LLMCompiler). Why: Compiling a plan into a dependency graph of calls is the cleanest formulation of automatic parallelization.	Peer-reviewed research Work graphs
📄 ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Models Paper · Plan-ahead execution	arXiv Binfeng Xu et al. · 2023	Introduced plan-with-placeholders: the planner writes the full tool-use plan up front with variable substitution (#E1, #E2) instead of interleaving reasoning with observations, cutting token consumption roughly 5x on some benchmarks. The variable-passing idea underlies most modern DAG planners including LLMCompiler. Why: Planning with placeholders separates reasoning from observation, which cuts token cost and makes plans inspectable.	Research preprint Work graphs
📝 Plan-and-Execute Agents Blog · Plan-and-execute patterns	langchain.com LangChain · 2024	Practitioner walkthrough of three planning architectures — plan-and-execute with a replanning step, ReWOO's variable-passing plans, and an LLMCompiler-style streaming DAG executor — with runnable LangGraph implementations. The standard bridge from the planning papers to working code, including why replanning beats one-shot plans and when small models can execute a big model's plan. Why: Walks through planning architectures including replanning, which is the part most implementations omit.	Practitioner analysis Work graphs
📄 Plan-and-Act: Improving Planning of Agents for Long-Horizon Tasks Paper · Planner/executor separation	arXiv Lutfi Eren Erdogan et al. (SqueezeAILab, UC Berkeley) · 2025	ICML 2025 work separating a Planner model (structured high-level plans, dynamically updated after each step) from an Executor model, trained on synthetic plan annotations of ground-truth trajectories. Reached 57.58% on WebArena-Lite and 81.36% text-only on WebVoyager, evidence that explicit plan/execute separation scales to long horizons. Why: Separating a planner from an executor with dynamic plan updates is the practical shape of long-horizon work.	Peer-reviewed research Work graphs
📄 Understanding the Planning of LLM Agents: A Survey Paper · Planning survey	arXiv Xu Huang et al. (USTC / Huawei) · 2024	First systematic survey of LLM-agent planning, organized into task decomposition, plan selection, external planning modules, reflection, and memory. Useful index into the pre-2024 decomposition and replanning literature that work-graph planners build on. Why: The systematic survey of agent planning, organized so you can locate the specific decomposition problem you face.	Research preprint Work graphs
📄 Demystifying Chains, Trees, and Graphs of Thoughts Paper · Reasoning-topology taxonomy	arXiv Maciej Besta et al. (ETH Zurich) · 2024	Taxonomy unifying chain-, tree-, and graph-structured reasoning under a common notation of topology, schedule, and transformation operators, surveying dozens of X-of-Thoughts variants. The reference map for choosing a work-graph shape for a reasoning task. Why: Unifies chain, tree, and graph reasoning under one notation, clarifying what structure buys at the reasoning level.	Research preprint Work graphs
📄 ADaPT: As-Needed Decomposition and Planning with Language Models Paper · Recursive decomposition	arXiv Archiki Prasad et al. (UNC / AI2) · 2024	Findings of NAACL 2024 paper on recursive, lazy decomposition: only when the executor fails a subtask does the planner decompose it further, so the work graph's depth adapts to both task complexity and executor capability. Improves success rates by up to 28.3% on ALFWorld and 27% on WebShop over fixed plan-and-execute. Why: Decomposing only when execution fails avoids paying planning cost on tasks that never needed it.	Peer-reviewed research Work graphs
📄 AdaPlanner: Adaptive Planning from Feedback with Language Models Paper · Replanning on failure	arXiv Haotian Sun et al. (Georgia Tech) · 2023	NeurIPS 2023 closed-loop planner that refines its own plan on environmental feedback, distinguishing in-plan refinement (observation matches prediction) from out-of-plan replanning (deviation triggers plan rewrite), with code-style plan prompts to curb hallucination. A core reference for replanning-on-failure policies. Why: Closed-loop plan refinement on feedback, distinguishing recoverable failures from ones that need a new plan.	Peer-reviewed research Work graphs
📄 Flow: Modularized Agentic Workflow Automation Paper · Runtime graph refinement	arXiv Boye Niu et al. · 2025	ICLR 2025 paper representing multi-agent workflows as activity-on-vertex (AOV) graphs that agents refine during execution, reallocating subtasks based on historical performance and maximizing parallelism while minimizing dependency complexity. One of the clearest formalizations of runtime work-graph rewriting for error tolerance. Why: Representing workflows as graphs that agents revise at runtime sits exactly at the boundary between planning and evolution.	Peer-reviewed research Work graphs
📄 Graph of Thoughts: Solving Elaborate Problems with Large Language Models Paper · Search-shaped work graphs	arXiv Maciej Besta et al. (ETH Zurich) · 2024	AAAI 2024 generalization of ToT to arbitrary graphs: thoughts are vertices, dependencies are edges, and branches can merge, aggregate, and feed back — exactly the split/merge operations of a dynamic work graph. Improves sorting quality 62% over ToT while cutting cost by over 31%. Why: Generalizing reasoning to arbitrary graphs, including aggregation, which anticipates work-graph shapes at the node level.	Peer-reviewed research Work graphs
📄 Tree of Thoughts: Deliberate Problem Solving with Large Language Models Paper · Search-shaped work graphs	arXiv Shunyu Yao et al. (Princeton / Google DeepMind) · 2023	NeurIPS 2023 framework that turns inference into search over a tree of intermediate 'thoughts' with generation, self-evaluation, lookahead, and backtracking. The tree is the simplest nontrivial work-graph shape, and ToT's generate/evaluate/prune cycle is the template for branch management in larger graphs (code: github.com/princeton-nlp/tree-of-thought-llm). Why: Search over intermediate steps with explicit evaluation, the origin of treating deliberation as structure.	Peer-reviewed research Work graphs
📄 HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face Paper · Task-graph planning	arXiv Yongliang Shen et al. (Zhejiang University / Microsoft) · 2023	NeurIPS 2023 paper that established the four-stage task-graph pipeline — task planning, model selection, execution, response generation — with an LLM controller decomposing requests into subtasks linked by a resource-dependency mechanism. The earliest widely known example of an LLM generating a work graph over other models. Why: The four-stage task-graph pipeline that established planning, selection, execution, and response as separable stages.	Peer-reviewed research Work graphs
📄 Can Small Agents Collaborate to Beat a Single Large Language Model? Paper · Model-tier staffing	arXiv Agata Żywot, Xinyi Chen, Yifei Yuan, Anders Søgaard, Maarten de Rijke · 2026	Across factual retrieval, multi-hop reasoning, scientific QA, and math, teams of small models beat substantially larger single agents that had direct tool access; gains tracked orchestrator capacity, while enabling reasoning in sub-agents gave limited or negative benefit. Why: Direct evidence for spending model budget on the orchestrator rather than uniformly across worker nodes.	Research preprint Roles
📄 Generative Agents: Interactive Simulacra of Human Behavior Paper · Generative agent societies	UIST 2023 Joon Sung Park, Joseph C. O'Brien, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, Michael S. Bernstein · 2023	Twenty-five LLM-driven characters in the Smallville sandbox plan days, spread news, form relationships, and self-organize a Valentine's Day party from a single seeded intention. The memory-stream / retrieval / reflection / planning architecture became the default template for agent societies, and the ablations show believability collapses when any one of those components is removed. Why: The template most agent societies still inherit, with ablations showing which cognitive components believability actually depends on.	Peer-reviewed research Roles
📄 LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals Paper · Generative agent societies	arXiv Joon Sung Park, Carolyn Q. Zou, Jonne Kamphorst, Niles Egan, Aaron Shaw, Benjamin Mako Hill, Carrie Cai, Meredith Ringel Morris, Percy Liang, Robb Willer, Michael S. Bernstein · 2024	Agents built from two-hour interviews with 1,052 Americans answer held-out General Social Survey items at 83% of the participants' own two-week test-retest consistency, against 74% for demographics-only agents; survey-only reaches 82% and combining both sources 86%. They also reduce accuracy disparities across racial and ideological groups. Originally circulated as "Generative Agent Simulations of 1,000 People" and retitled in later revisions — cite the current title. Why: Evidence on how faithfully a persona can stand in for a real individual, which bounds what role descriptions can deliver.	Research preprint Roles
📄 Generative agent-based modeling with actions grounded in physical, social, or digital space using Concordia Paper · Simulation platforms	arXiv Alexander Sasha Vezhnevets, John P. Agapiou, Avia Aharon, Ron Ziv, Jayd Matyas, Edgar A. Duéñez-Guzmán, Joel Z. Leibo et al. · 2023	Introduces the Game Master pattern borrowed from tabletop RPGs: agents state intended actions in natural language and a separate GM entity adjudicates what actually happens in the world. That split makes the coordination and grounding rules an explicit, swappable component rather than something buried in each agent's prompt. Why: The Game Master pattern makes adjudication an explicit, swappable component instead of rules buried in each agent's prompt.	Research preprint Topology
📄 Project Sid: Many-agent simulations toward AI civilization Paper · Large-scale civilizations	arXiv Altera.AL · 2024	Runs 50–1,000+ agents in Minecraft using PIANO (Parallel Information Aggregation via Neural Orchestration), which lets multiple cognitive modules run concurrently while a bottleneck enforces coherence across output streams. Reported emergent outcomes include spontaneous role specialization, a functioning merchant economy, collective rule-making and amendment, and religious/cultural transmission between societies. Why: Concurrent cognitive modules behind a coherence bottleneck, with role specialization and rule-making emerging rather than assigned.	Research preprint Topology
📄 Voyager: An Open-Ended Embodied Agent with Large Language Models Paper · Large-scale civilizations	arXiv Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, Anima Anandkumar · 2023	A single Minecraft agent that keeps an ever-growing library of executable-code skills, indexed and retrieved for reuse, driven by an automatic curriculum and self-verification loop. The skill library is the load-bearing idea for agent organizations: capability accumulates as an inspectable, composable artifact outside the model rather than inside a context window. Why: The skill library is the transferable idea: capability accumulates as an inspectable artifact outside any context window.	Research preprint State
📄 AgentSociety: Large-Scale Simulation of LLM-Driven Generative Agents Advances Understanding of Human Behaviors and Society Paper · Large-scale civilizations	arXiv Jinghua Piao, Yuwei Yan, Jun Zhang, Nian Li, Junbo Yan, Xiaochong Lan, Zhihong Lu, Yong Li et al. · 2025	Simulates over 10,000 agents producing roughly 5 million interactions inside a modeled city, with agents given sociologically motivated needs, emotions, and motivations, and applies it to polarization, misinformation spread, and policy scenarios. The accompanying open-source platform (tsinghua-fib-lab/AgentSociety) is one of the few society simulators built for distributed execution at that scale. Why: One of the few society simulators built for distributed execution, useful as a reference for coordinating agents at city scale.	Research preprint Topology
📄 OASIS: Open Agent Social Interaction Simulations with One Million Agents Paper · Large-scale civilizations	arXiv Ziyi Yang, Zaibin Zhang, Zirui Zheng, Yuxian Jiang, Ziyue Gan, Zhiyu Wang, Jing Shao et al. · 2024	Scales X/Reddit-style social simulation to one million LLM agents with 23 available actions (follow, repost, comment, like…), and reproduces information spread, group polarization, and herd effects. Notably finds that some phenomena only appear past certain population sizes, which is direct evidence that small-N agent experiments can mislead about large-N behavior. Why: Finds phenomena that only appear past certain population sizes, direct evidence that small-N agent experiments can mislead.	Research preprint Topology
📄 Emergent social conventions and collective bias in LLM populations Paper · Norms and conventions	Science Advances 11, eadu9368 Ariel Flint Ashery, Luca Maria Aiello, Andrea Baronchelli · 2025	Decentralized populations of LLM agents playing a naming game converge on universally adopted conventions with no central coordination, and develop strong collective biases toward particular options that are absent when the same agents are tested individually. A committed minority of adversarial agents can then flip the whole population's convention — a tipping-point result with direct implications for how a few misaligned agents can reshape norms in a running agent org. Why: Shows conventions and collective bias emerging from agents that are individually unbiased, and a committed minority overturning them.	Peer-reviewed research Evolution
📄 Emergent Coordination in Multi-Agent Language Models Paper · Norms and conventions	ICLR 2026 Christoph Riedl · 2025	Applies partial information decomposition of time-delayed mutual information to test whether an agent group is genuinely more than the sum of its members, separating spurious temporal coupling from performance-relevant cross-agent synergy. In a guessing game with no direct communication, control-condition groups show strong temporal synergy but little real cross-agent coordination — a usable statistical test for whether an org graph is actually doing work. Why: Measures whether coordination genuinely emerges between language-model agents rather than being imposed by the prompt.	Peer-reviewed research Topology
📄 Evaluating Cooperation in LLM Social Groups through Elected Leadership Paper · Org structure evidence	arXiv Ryan Faulkner, Anushka Deshpande, David Guzman Piedrahita, Joel Z. Leibo, Zhijing Jin · 2026	Adds elected personas and candidate-driven agendas to LLM groups managing a shared resource, raising social welfare 55.4% and survival time 128.6% across several strong models. The authors build agent relationship networks to measure leader influence, making this a direct experiment on whether adding an authority edge to the org graph pays for itself. Why: Tests whether electing a leader changes cooperation, which is the closest analogue to choosing an orchestrator by policy.	Research preprint Roles
📄 Corrupted by Reasoning: Reasoning Language Models Become Free-Riders in Public Goods Games Paper · Norms and conventions	COLM 2025 David Guzman Piedrahita, Yongjin Yang, Mrinmaya Sachan, Giorgia Ramponi, Bernhard Schölkopf, Zhijing Jin · 2025	In public goods games with costly punishment, reasoning models such as the o1 series cooperate substantially worse than some non-reasoning models, converging on free-riding. A negative result worth designing around: putting your strongest reasoner in a shared-resource role can degrade collective outcomes rather than improve them. Why: Finds stronger reasoning models free-riding more in shared-resource settings, a caution against assuming better nodes cooperate better.	Peer-reviewed research Roles
📄 The Role of Social Learning and Collective Norm Formation in Fostering Cooperation in LLM Multi-Agent Systems Paper · Norms and conventions	AAMAS 2026 Prateek Gupta, Qiankun Zhong, Hiromu Yakura, Thomas Eisenmann, Iyad Rahwan · 2025	A common-pool-resource testbed that strips out explicit reward signals and instead gives agents social learning from successful peers plus norm-based punishment, following Ostrom's institutional design principles. Norms emerge endogenously, and the paper reports systematic differences between models in whether cooperation is sustained under resource scarcity versus abundance. Why: Norms forming through social learning is the mechanism behind conventions an agent organization never explicitly configured.	Peer-reviewed research Evolution
📄 Institutional AI: Governing LLM Collusion in Multi-Agent Cournot Markets via Public Governance Graphs Paper · Org structure evidence	arXiv Marcantonio Bracale Syrnikov, Federico Pierucci, Marcello Galisai, Matteo Prandi, Piercosma Bisconti, Francesco Giarrusso et al. · 2026	Reframes alignment as mechanism design in institution-space: a public, immutable governance graph declares legal states, transitions, sanctions, and restorative paths, and an Oracle/Controller runtime enforces it while writing a cryptographically keyed append-only governance log. Tested on repeated Cournot markets against ungoverned and prompt-only "constitutional" baselines, it is the clearest published case of the coordination policy itself being the inspectable artifact. Why: Treats collusion between agents as a governance problem with institutional remedies, which is what a gate has to enforce.	Research preprint Gates
📄 Drop the Hierarchy and Roles: How Self-Organizing LLM Agents Outperform Designed Structures Paper · Org structure evidence	arXiv Victoria Dochkina · 2026	Varies coordination protocols from centralized to fully self-organized across 25,000 tasks, 8 models, and 4–256 agents; the autonomous Sequential protocol beats centralized coordination by 14% (p<0.001), and 8 agents generate 5,006 distinct self-assigned roles. The capability-dependent split is the practical takeaway — strong models self-organize, weaker ones need imposed structure — though it is a single-author preprint whose scale claims deserve independent replication. Why: Reports self-organizing agents beating centralized coordination by 14%, with rigid structure helping only weaker models.	Research preprint Roles
📄 Can LLM Agents Sustain Long-Horizon Organizational Dynamics? Paper · Org structure evidence	arXiv Xuancheng Zhu, Yang Yue, Shuaibing Wan, Zihan Dou, Xiaohan Zhang, Yongrui Liu, Guoshun Nan · 2026	TaskWeave runs a year-long simulated IT company using a Formulate-Partition-Diagnose-Align cycle plus dependency-tracking memory that grounds each step in the actual task sequence and accumulated artifacts. The conclusion — that structured simulation memory, not agent cleverness, is what keeps an organization coherent over long horizons — is essentially an argument for the run graph as durable infrastructure. Why: Asks whether an agent organization holds together over long horizons, which is the question production fleets actually face.	Research preprint Reliability
📄 The Silicon Society Cookbook: Design Space of LLM-based Social Simulations Paper · Critiques and validity	arXiv Aurélien Bück-Kaeffer, Sneheel Sarangi, Maximilian Puelma Touzel, Reihaneh Rabbany, Zachary Yang, Jean-François Godbout · 2026	Sweeps design choices — base model, network topology, persona construction, interaction protocol — across simulated social networks and finds the effects are non-linear and interacting, with the choice of base LLM dominating everything else including topology. Directly relevant to graph engineering: your carefully designed structure may matter less than which model sits in the nodes. Why: Maps the design space of society simulations, which doubles as a catalog of coordination choices and their consequences.	Research preprint Topology
📄 A Universal Modular ACTOR Formalism for Artificial Intelligence Paper · Actor model and supervision	IJCAI 1973 Carl Hewitt, Peter Bishop, Richard Steiger · 1973	Introduces the actor model: everything is an actor, an independent entity that has an address, receives messages, and decides locally what to do next, including creating new actors. This is the original argument that concurrent computation can be built entirely out of addressable, message-passing units rather than shared state and function calls. Why: The origin of independent entities communicating only by message, and the ancestor of every agent-as-node design.	Peer-reviewed research Topology
📄 Making Reliable Distributed Systems in the Presence of Software Errors Paper · Actor model and supervision	PhD thesis, Royal Institute of Technology, Stockholm Joe Armstrong · 2003	Armstrong's doctoral thesis behind Erlang/OTP, arguing that reliability comes from isolated processes that fail fast and separate supervisors that restart them, rather than from defensive error handling inside each process. Formalizes supervision trees and the worker/supervisor split that later gets borrowed for agent failure containment. Why: The thesis behind let-it-crash and supervision trees, the most battle-tested answer to what happens when a node dies.	Peer-reviewed research Reliability
📚 Erlang/OTP Supervisor Behaviour (Design Principles) Docs · Actor model and supervision	Erlang/OTP System Documentation Ericsson / Erlang/OTP team · 2026	The operational spec for supervision trees: child specifications, the four restart strategies (one_for_one, one_for_all, rest_for_one, simple_one_for_one), and restart intensity limits (max restarts per time period) that stop a crash loop from cascading. Concrete prior art for how a parent node in an agent graph should react when a child dies. Why: Supervision strategies specified concretely, including restart policy, which is what an agent orchestrator eventually reimplements.	Official documentation Reliability
📐 FIPA Contract Net Interaction Protocol Specification (SC00029H) Standard · Delegation and task allocation protocols	Foundation for Intelligent Physical Agents FIPA · 2002	Standardizes Smith's contract net as a message sequence with explicit deadlines and cancellation: cfp, propose or refuse, accept-proposal or reject-proposal, then inform-done, inform-result, or failure. A wire-level, machine-checkable definition of a delegation edge, including how it is allowed to fail. Why: The classic task-allocation protocol: announce, bid, award. Modern delegation edges rediscover it repeatedly.	Industry standard Handoffs
📄 BDI Agents: From Theory to Practice Paper · Agent architectures	ICMAS 1995 Anand S. Rao, Michael P. Georgeff · 1995	Connects belief-desire-intention logic to a working interpreter, showing how an agent commits to intentions, when it drops or reconsiders them, and how commitment strategies trade responsiveness against thrashing. The reference statement of what a single bounded agent's decision loop consists of, drawn from PRS and dMARS deployments. Why: The belief-desire-intention model that gave agents an internal structure worth reasoning about rather than a prompt.	Peer-reviewed research Roles
📄 The Gaia Methodology for Agent-Oriented Analysis and Design Paper · Agent-oriented software engineering	Autonomous Agents and Multi-Agent Systems 3, 285-312 Michael Wooldridge, Nicholas R. Jennings, David Kinny · 2000	Treats a multi-agent system as an organization to be designed, with an analysis phase producing roles (responsibilities, permissions, activities, protocols) and a design phase producing an agent model, services model, and acquaintance model. The acquaintance model is literally the communication graph, made an explicit deliverable rather than an emergent property. Why: An early methodology treating agent systems as organizations with roles and interactions, decades before the current label.	Peer-reviewed research Roles
📄 Developing Organised Multi-Agent Systems Using the MOISE+ Model Paper · Organization models for multi-agent systems	International Journal of Agent-Oriented Software Engineering 1, 370-395 Jomi F. Hübner, Jaime S. Sichman, Olivier Boissier · 2007	Specifies an agent organization as a first-class, machine-readable artifact in three separable parts: structural (roles, groups, inheritance, intra- and inter-group links), functional (global goals decomposed into missions and schemes), and deontic (which role is permitted or obliged to take which mission). The closest pre-LLM precedent for a declared, inspectable org graph decoupled from the agents that fill it. Why: Separates the organization specification from the agents filling it, which is exactly the org-graph-versus-instance distinction.	Peer-reviewed research Roles
📄 Trends in Cooperative Distributed Problem Solving Paper · Blackboard and distributed problem solving	IEEE Transactions on Knowledge and Data Engineering 1, 63-83 Edmund H. Durfee, Victor R. Lesser, Daniel D. Corkill · 1989	Surveys how nodes that each hold partial, inconsistent information can still solve a problem jointly, covering task decomposition, result synthesis, organizational structuring, and partial global planning. Names the failure modes (distraction, redundant work, conflicting local views) that reappear whenever agents are given overlapping scope. Why: Surveys how distributed problem solvers divide work and share results, the questions topology research still asks.	Peer-reviewed research Topology
📄 The Interdisciplinary Study of Coordination Paper · Coordination theory	ACM Computing Surveys 26, 87-119 Thomas W. Malone, Kevin Crowston · 1994	Defines coordination as managing dependencies between activities, then enumerates dependency types (shared resource, producer/consumer, simultaneity, task/subtask) and the coordination mechanisms available for each. Gives a vocabulary for saying precisely what an edge in an org or run graph is carrying, drawn from computer science, economics, and organization theory at once. Why: Defines coordination as managing dependencies between activities, a definition that makes edges the unit of design.	Peer-reviewed research Topology
📄 End-to-End Arguments in System Design Paper · Distributed systems principles	ACM Transactions on Computer Systems 2, 277-288 J. H. Saltzer, D. P. Reed, D. D. Clark · 1984	Argues that a function can usually only be implemented correctly at the endpoints that know the application semantics, so lower layers should not try to guarantee it; low-level versions are justified only as performance optimizations. Directly bears on where verification and correctness checks belong in an agent pipeline: the orchestrator cannot check what only the endpoint understands. Why: The argument for placing correctness checks at the endpoints, which decides whether a gate belongs at the edge or in the middle.	Peer-reviewed research Handoffs
📄 A Note on Distributed Computing Paper · Distributed systems principles	Sun Microsystems Laboratories Technical Report TR-94-29 Jim Waldo, Geoff Wyant, Ann Wollrath, Sam Kendall · 1994	Shows that hiding the local/remote distinction behind a uniform call interface fails, because remote interaction differs irreducibly in latency, memory access, concurrency, and above all partial failure. The standing argument against pretending a delegation to another agent is just a function call. Why: Explains why pretending remote calls are local fails, the mistake every agent handoff abstraction is tempted to make.	Research preprint Handoffs
📄 The Byzantine Generals Problem Paper · Distributed systems principles	ACM Transactions on Programming Languages and Systems 4, 382-401 Leslie Lamport, Robert Shostak, Marshall Pease · 1982	Proves that with unsigned messages, agreement among n participants is impossible unless more than two-thirds are loyal, and that signatures change the bound. The formal ceiling on what voting, cross-checking, or majority-of-agents schemes can deliver when some participants produce arbitrarily wrong output. Why: The canonical result on agreeing in the presence of faulty participants, the theory behind quorum gates.	Peer-reviewed research Gates
📄 Time, Clocks, and the Ordering of Events in a Distributed System Paper · Distributed systems principles	Communications of the ACM 21, 558-565 Leslie Lamport · 1978	Defines the happened-before partial order and logical clocks, showing that causality, not wall-clock time, is what a distributed system can actually observe. The formal basis for run-graph lineage: which agent's output could possibly have influenced which other, when there is no global clock. Why: Ordering events without a global clock is the problem shared state across parallel agents actually has.	Peer-reviewed research State
📄 Sagas Paper · Distributed systems principles	ACM SIGMOD 1987 Hector Garcia-Molina, Kenneth Salem · 1987	Introduces the saga: a long-running unit of work split into a sequence of committed sub-transactions, each with a compensating action that semantically undoes it if a later step fails. The pattern behind rollback and repair in long agent workflows where holding a lock across the whole run is not an option. Why: The compensating-transaction pattern that gives a long-running graph a defined way to undo partial work.	Peer-reviewed research Reliability
📄 Life beyond Distributed Transactions: an Apostate's Opinion Paper · Distributed systems principles	CIDR 2007; revised in ACM Queue 14, 2016 Pat Helland · 2007	Argues that at scale you give up distributed transactions and instead structure the world as entities (atomically updatable alone) plus activities that coordinate them through messages, which forces every message to be made idempotent and every partner relationship to carry state. A blueprint for scoping agent state and making retries safe. Why: Argues for designing around the absence of distributed transactions, which is the situation agent state is already in.	Peer-reviewed research State
📝 Fallacies of Distributed Computing Explained Blog · Distributed systems principles	Self-published whitepaper Arnon Rotem-Gal-Oz · 2006	Works through the eight fallacies attributed to L. Peter Deutsch and James Gosling at Sun (the network is reliable, latency is zero, bandwidth is infinite, the network is secure, topology doesn't change, there is one administrator, transport cost is zero, the network is homogeneous) with the consequences of each. The compact checklist of assumptions that quietly break once agents run on separate machines under separate owners. Why: The assumptions that break distributed systems, restated clearly, and every one of them applies to an agent fleet.	Practitioner analysis Reliability
📄 The Nature of the Firm Paper · Organization theory	Economica 4, 386-405 Ronald H. Coase · 1937	Asks why firms exist rather than everything being contracted on the open market, and answers with transaction costs: a firm grows until the cost of organizing one more task internally equals the cost of buying it outside. The framework for deciding when to add a standing agent role versus spawning a one-off task. Why: Transaction costs decide what belongs inside one organization, which is the economics behind adding or cutting a node.	Peer-reviewed research Roles
📄 Organization Design: An Information Processing View Paper · Organization theory	Interfaces 4, 28-36 Jay R. Galbraith · 1974	Treats task uncertainty as the driver of structure: the more uncertain the work, the more information must be processed during execution, and organizations respond either by reducing the need (slack resources, self-contained units) or by increasing capacity (vertical information systems, lateral links). A direct design lever for choosing between hierarchical routing and peer-to-peer agent links. Why: Treats organization structure as a response to information-processing load, a direct analogue for topology under uncertainty.	Peer-reviewed research Topology
📕 The Structuring of Organizations: A Synthesis of the Research Book · Organization theory	mintzberg.org Henry Mintzberg · 1979	Reduces all organizational coordination to five mechanisms (mutual adjustment, direct supervision, standardization of work processes, of outputs, and of skills) and builds five structural configurations from them. Gives a finite, checkable taxonomy for what an edge between two agents actually enforces: a supervisor, a shared spec, or an informal channel. Why: A structural vocabulary for organizations that maps onto agent topologies more precisely than ad hoc diagrams.	Book/course Topology
📄 Exploration and Exploitation in Organizational Learning Paper · Organization theory	Organization Science 2, 71-87 James G. March · 1991	Models the tension between exploring new possibilities and exploiting known ones, showing with a simulation that fast learners converging on shared code can lower long-run collective performance because they stop generating variety. The reason a fleet of agents that all adopt the same successful strategy gets worse, not better. Why: The exploration-exploitation tension in organizations, which is what a self-modifying graph is really trading off.	Peer-reviewed research Evolution
📕 The Mythical Man-Month: Essays on Software Engineering, Anniversary Edition Book · Organization theory	Addison-Wesley Professional Frederick P. Brooks Jr. · 1995	Brooks's law (adding people to a late project makes it later) rests on communication paths growing as n(n-1)/2, and the surgical team chapter proposes an asymmetric structure where one person does the work and the rest are specialized support. The 1995 edition adds a twenty-year retrospective and reprints No Silver Bullet. Why: Communication overhead growing faster than headcount is why adding agent nodes stops helping.	Book/course Topology
📄 How Do Committees Invent? Paper · Organization theory	Datamation, April 1968 Melvin E. Conway · 1968	The original article behind Conway's law, whose thesis appears three paragraphs from the end: organizations that design systems are constrained to produce designs that copy their own communication structures. Full text on the author's site, with his later note on why the argument is laid out that way. Why: The original statement of the law linking communication structure to system design, applicable to agents as literally as to teams.	Research preprint Topology
📄 The Architecture of Complexity Paper · Organization theory	Proceedings of the American Philosophical Society 106, 467-482 Herbert A. Simon · 1962	Argues that complex systems that survive are hierarchic and nearly decomposable: interactions within a subsystem are strong and frequent, interactions across subsystems are weak and slow, which is why such systems can be assembled and understood at all. The theoretical case for cutting an agent organization at low-traffic boundaries. Why: Why complex systems are hierarchical and nearly decomposable, an argument for the shape most agent organizations converge on.	Peer-reviewed research Topology
📕 The Society of Mind Book · Agent architectures	Simon & Schuster Marvin Minsky · 1986	Builds intelligence out of large numbers of small, individually mindless agents organized into agencies, with chapters on how they are recruited, how conflicts between them are settled, and how censors and suppressors block bad plans. Written as 270 one-page essays; the source of the intuition that capability is an emergent property of an organization of narrow parts. Why: Intelligence from many simple interacting agents, the intellectual precedent for treating a mind as an organization.	Book/course Roles
📄 A Brief History of Stigmergy Paper · Coordination theory	Artificial Life 5, 97-116 Guy Theraulaz, Eric Bonabeau · 1999	Traces Grassé's 1959 concept of stigmergy, where insects coordinate not by talking to each other but by modifying a shared environment that then triggers the next action, and distinguishes quantitative from qualitative variants. The formal account of indirect coordination through shared artifacts, with no messages and no coordinator. Why: Coordination through traces left in a shared environment, the mechanism behind agents that coordinate via artifacts.	Peer-reviewed research Evolution
📄 The Application of Petri Nets to Workflow Management Paper · Workflow and dataflow theory	Journal of Circuits, Systems and Computers 8, 21-66 Wil M. P. van der Aalst · 1998	Defines WF-nets, a restricted class of Petri nets for workflows, and gives a decidable soundness criterion: every case can still reach completion, completion leaves no tokens behind, and no transition is dead. Shows what it means to statically verify that a task graph cannot deadlock or strand work before you run it. Why: Formal workflow semantics developed long before agent DAGs, including the soundness questions those DAGs ignore.	Peer-reviewed research Work graphs
📕 Multiagent Systems: Algorithmic, Game-Theoretic, and Logical Foundations Book · Foundational texts	Cambridge University Press Yoav Shoham, Kevin Leyton-Brown · 2009	The standard graduate reference on multiagent systems — distributed constraint satisfaction, coalition formation, mechanism design, social choice — with a free uncorrected PDF distributed by the authors under their Cambridge contract. The formal grounding for delegation, verification, and escalation edges predating LLM agents. Why: The standard textbook for the algorithmic and game-theoretic side of multi-agent systems.	Book/course Topology
📕 An Introduction to MultiAgent Systems (2nd edition) Book · Foundational texts	John Wiley & Sons Michael Wooldridge · 2009	The standard undergraduate textbook for classical multi-agent systems; the 2009 second edition adds chapters on ontologies, voting, auctions, bargaining, coalition formation, and argumentation. Author's page carries errata and slide decks. Why: The standard introductory text on multi-agent systems, and the fastest way to stop reinventing its vocabulary.	Book/course Roles
📄 Topological Structure Learning Should Be A Research Priority for LLM-Based Multi-Agent Systems Paper · Topology research agenda	arXiv Jiaxi Yang, Mengqi Zhang, Yiqiao Jin, Hao Chen, Qingsong Wen, Lu Lin, Yi He, Srijan Kumar, Weijie Xu, James Evans, Jindong Wang · 2025	A position paper arguing that how agents are connected is neglected relative to how they are prompted, and proposing a three-stage agenda of agent selection, structure profiling, and topology synthesis, tied to adaptability, efficiency, robustness, and fairness. Revised October 2025. Why: States the case that topology is a first-class design variable, and names the stages a team would work through to choose one deliberately.	Research preprint Topology
📄 Inference-Time Graph Engineering for Multi-Agent LLM Workflows Paper · Task-adaptive topology synthesis	arXiv Katherine Tieu, Dongqi Fu, Yinglong Xia, Hong Li, Hong Yan, Jingrui He · 2026	Introduces ReActNet, a training-free method that compiles a query and a set of role-specialized agents into a sequence of directed communication graphs, one per reasoning stage, in which every edge carries a natural-language instruction for what the source agent should send the target. Compilation is kept separate from execution, and the compiled graph is then run by structured message passing. Improves on fixed-topology and learned-topology baselines across knowledge reasoning, math, code generation, and GAIA-style assistant tasks at competitive inference cost. Why: Treats what an edge instructs, not only whether it exists, as something to engineer, and keeps the compiled graph as an inspectable artifact separate from the run.	Research preprint Topology
📄 Reinforcement Learning for LLM-based Multi-Agent Systems through Orchestration Traces Paper · Learning orchestration	arXiv Chenchen Zhang · 2026	Surveys reinforcement learning for multi-agent LLM systems through orchestration traces: temporal interaction graphs whose events include spawning, delegation, communication, tool use, return, aggregation, and stopping. Groups reward design into eight families, places reward and credit signals on eight units from token to team, and splits orchestration learning into when to spawn, whom to delegate to, how to communicate, how to aggregate, and when to stop, finding no explicit RL training method for the stopping decision in its pool as of May 2026. Releases an 84-entry tagged paper pool and a JSON schema for replayable traces. Why: Frames the spawn, delegate, and stop decisions in a run graph as learnable, and shows which of them research has barely touched.	Research preprint Evolution
📄 LEMON: Learning Executable Multi-Agent Orchestration via Counterfactual Reinforcement Learning Paper · Learned topology generation	arXiv Xudong Chen, Yixin Liu, Hua Wei, Kaize Ding · 2026	Trains an LLM orchestrator to emit one executable specification that covers task-specific roles, duties, capacity levels, and the dependency structure between agents. Training adds a counterfactual signal to the GRPO objective: it edits a single role, capacity, or dependency field and applies the resulting reward difference only to the edited span, so each local orchestration choice gets its own credit. Reports the best results among the orchestration methods it evaluates on MMLU, GSM8K, AQuA, MultiArith, SVAMP, and HumanEval. Why: Treats the org graph as a generated, deployable artifact and gives each role and dependency choice its own learning signal.	Research preprint Evolution
📄 Multi-Agent Coordination Adaptation via Structure-Guided Orchestration Paper · Structure-guided orchestration	arXiv Haoran Li, Shulun Chen, Shaoyuan Sun, Hanchen Wang · 2026	Casts coordination as posterior inference over structure and orchestration together. MACA learns a task- and budget-conditioned structural prior over which agents take part and how they interact, and uses it to guide a policy that makes orchestration decisions at run time. Outperforms adaptive multi-agent baselines by an average of 8.42% while using 43.19% fewer tokens, and joint adaptation suppresses redundant interactions. Why: Sits between fixing the topology up front and orchestrating ad hoc, by letting a learned structural prior constrain run-time routing.	Research preprint Topology
📄 Coordination as an Architectural Layer for LLM-Based Multi-Agent Systems Paper · Controlled topology comparison	arXiv Maksym Nechepurenko, Pavel Shuvalov · 2026	Argues coordination should be a configurable layer separate from agent logic and from information access, and tests the idea with everything else held fixed: one model, fixed tools, output cap, and prompt template across five coordination configurations on 100 Polymarket binary markets resolved after the model's training cutoff. A Murphy decomposition of the Brier score gives each configuration a distinguishable signature even when aggregate scores match. Three of five pre-specified predictions hold in direction, pairwise tests do not survive Bonferroni correction, and the harness, traces, and agents are released. Why: A template for comparing coordination configurations with everything else held constant, and for reporting modest effects without overstating them.	Research preprint Topology
📄 ORCH: Organizational Principles Enable Collective Intelligence in Embodied AI Paper · Org structure evidence	arXiv Zhengran Ji, Jonathan Hyun, Boyuan Chen · 2026	Builds task-specific hierarchies from two classical forms of interdependence: pooled, for work that can run concurrently, and sequential, for work with prerequisites. Across 25 wildfire-response missions with teams of up to 50 heterogeneous agents and eight language models, organizations designed on these principles beat four representative embodied multi-agent approaches on mission outcome, execution efficiency, exploration, and compute use; human-designed ones improved final score by 63.97% and efficiency by 74.29% on average, and organizations generated by language models by 43.63% and 52.53%. Collective performance was not monotonic in model scale. Why: Rare large-scale evidence that a coordination structure chosen to match how work depends on other work beats a fixed one, and that a bigger model does not substitute for it.	Research preprint Topology
📄 Learning How Much to Collaborate: Difficulty-Aware Topology Selection for Multi-Agent Code Generation Paper · Difficulty-aware topology selection	arXiv Yunsong Hong · 2026	Measures five communication topologies on 614 problems from APPS, HumanEval+, and LiveCodeBench, and finds hierarchy's advantage over a single agent grows from 2.4 points of pass@1 on the easiest third to 21.1 on the hardest third, at roughly ten times the token cost. The proposed selector predicts each topology's chance of solving a problem and picks the one maximizing predicted success minus cost; at 40% of the always-hierarchical spend it reaches 77.7% pass@1 against 73.6%. Its budget-matched protocol re-ranks six cost-aware methods, with two baselines that led falling behind once calibrated to equal spend. Why: Argues topology is a per-task decision rather than a deployment-wide one, and shows how misleading topology comparisons are until spend is held equal.	Research preprint Topology
📄 Toward an Organizational Science of Multi-Agent LLM Systems: Decoupling Who, How, and Which Algorithm Paper · Org theory / team design	arXiv Huan Chen, Xiang Song, Jian Jin, Pan Ren, Liang-Jie Zhang · 2026	Separates three things most frameworks entangle: who is on the team, how members align, and which algorithm fuses their work. Belbin roles, Mintzberg coordination, and RACI accountability become executable configuration, six published collaboration algorithms sit behind one interface, and a contextual-bandit meta-protocol picks a protocol per task under a stated quality-cost tradeoff. The ablations locate the mechanism: where accountability sits changes outcomes exactly when the protocol routes the deliverable through the accountable agent, and the best placement flips across model families. Why: Makes organization, coordination, and protocol independently swappable, which is what comparing org designs honestly requires, and shows accountability placement has to be revalidated per model.	Research preprint Roles
📄 Codebook Agent: Amortized Topology Design for LLM Multi-Agent Systems Paper · Learned topology generation	arXiv Jinxi Yu, Yubei Li, Eric Hanchen Jiang, Zhi Zhang, Dong Liu, Wenxiao Zhao, et al. · 2026	Challenges how per-query topology designers are built, with three measurements: topologies that survive a reward filter collapse to about six distinct graphs even as codebook capacity grows from 8 to 64; edge count correlates negatively with measured token use, around r = -0.4, so sparsifying a graph can make inference more expensive; and a message-passing scorer over agent-profile nodes cannot rank candidates when agents share a profile, the default on published benchmarks. The proposed method compresses successful topologies into a 16-entry codebook, maps each query to a distribution over codes, and reranks decoded candidates with a proxy trained on measured utility and token cost. It is the most accurate of the compared designers on six benchmarks, 84.6 against 83.0 on average, emits a topology in 2.4 ms, and uses 21.9 to 33.2% fewer tokens. Why: Evidence that fewer edges is not a cost proxy and that useful topologies form a small repertoire, which changes how topology search should be framed.	Research preprint Topology
📄 Evolve as a Team: Collaborative Self-Evolution for LLM-based Multi-Agent Systems Paper · System evolution	arXiv Zhezheng Hao, Tianfu Wang, Huanshuo Dong, Ziyan Liu, Hong Wang, Xiankun Lin, et al. · 2026	Meta-Team improves a multi-agent system from its own execution experience while keeping each agent's execution context and coordinating a post-task exchange in which agents share the distributed evidence of what went wrong. It then applies improvements at three scales: individual agent behavior, inter-agent coordination, and team-level organization. Across six long-horizon agent benchmarks it outperforms single-agent systems, hand-built multi-agent systems, and prior evolution methods. Why: Treats a team's post-run review as an explicit phase, so improvements can target coordination and organization as well as individual prompts.	Research preprint Evolution
📄 CollabSim: A CSCW-Grounded Methodology for Investigating Collaborative Competence of LLM Agents through Controlled Multi-Agent Experiments Paper · Collaborative competence	arXiv Jiaju Chen, Bo Sun, Yuxuan Lu, Yun Wang, Dakuo Wang, Bingsheng Yao · 2026	Brings Computer-Supported Cooperative Work research into multi-agent evaluation: collaborative competence is defined as establishing common ground, maintaining shared task understanding, balancing individual and collective incentives, and repairing misalignment during interaction. The framework manipulates interaction conditions under control and probes agents' internal states at the level of individual actions. Experiments across four LLMs show it captures condition effects, separates model performance patterns, and exposes task-dependent effects of agent design. Why: Connects agent coordination to decades of research on human teams working under constrained communication, a lineage this catalog otherwise lacks.	Research preprint Handoffs
📄 When Sub-Agents Work in Parallel: The Promises and Pitfalls of Dynamic Concurrency in Long-Horizon Coding Tasks Paper · Dynamic agent spawning	arXiv Han Li, HanHaoNing Li, Ziqian Jiang, Yiling Lou · 2026	Treats letting an agent decide mid-run whether and how to spawn concurrent sub-agents as an execution policy and measures it directly: matched Codex, Claude Code, and Kimi Code executions with the policy on and off, across 354 tasks and 2,124 executions spanning task complexity and horizon. On shorter tasks model capability largely decides the outcome; on long-horizon development, orchestration becomes central. Characterizes 13 concurrency-specific failure modes, 28 observable scheduling patterns, and the conditions under which spawning helps. Why: Controlled evidence on model-directed fan-out in the coding agents people actually use, including the specific ways it goes wrong.	Research preprint Work graphs
📄 From Agents to Edges: Paired Optimization for Multi-Agent Collaboration Paper · Edge-level optimization	arXiv Feng Lin, Xiaotian Yu, Yifan Yang, Aijun Li · 2026	Treats the communication edge as the unit of optimization: for black-box models it jointly revises the instructions at both ends of an edge and their shared convention, keeping weights, roles, and topology fixed. A bandit picks which edge to refine and execution feedback guides paired revisions; the learned instructions fold into existing prompts with no extra calls at deployment. On a four-agent workflow it outperforms prompt-optimization and communication baselines across multi-agent benchmarks, and configurations learned on HotpotQA transfer to other datasets. Why: Shows how much a fixed graph gains from tuning what each edge says and how the receiver reads it.	Research preprint Handoffs

Frameworks & SDKs

Resource	Source	What it contributes	Evidence
📚 CrewAI Crews Docs · Role-based teams	CrewAI Documentation CrewAI · 2026	Documents role-based agent crews, assigned tasks, delegation, and sequential or hierarchical execution processes. Why: Offers accessible primitives for testing explicit ownership and manager-worker coordination.	Official documentation Roles
📚 OpenAI Agents SDK: Agent orchestration Docs · Agent orchestration	OpenAI Agents SDK Documentation OpenAI · 2026	Explains manager-style orchestration with agents exposed as tools and decentralized orchestration through handoffs. Why: Makes the centralized-versus-decentralized topology choice explicit in a production SDK.	Official documentation Topology
📚 Strands Agents: Coordinate Multiple Agents Docs · Pattern catalog	Strands Agents Documentation Strands Agents; AWS · 2026	Overview of Strands' coordination primitives: agents as tools, where an orchestrator delegates to specialist agents in a hierarchy; swarm, where agents hand off to one another autonomously with shared context and working memory; graph, with deterministic DAG or cyclic execution, conditional edges, and nested agents; and workflow, for sequential tasks with explicit dependencies. It links to A2A for reaching remote agents, and swarm and graph runs can pause for human approval through interrupts. Why: Lets builders compare topology choices within one SDK instead of treating one pattern as universal.	Official documentation Topology
📚 Pydantic AI: Multi-Agent Applications Docs · Typed agents	Pydantic AI Documentation Pydantic · 2026	Shows delegation, programmatic control flow, and graph-based state machines for composing typed Python agents. Why: Useful for expressing node inputs, outputs, dependencies, and orchestration boundaries in ordinary application code.	Official documentation Topology
📚 LlamaIndex: Multi-Agent Patterns Docs · Multi-agent orchestration	LlamaIndex Documentation LlamaIndex · 2026	Covers agent workflow, orchestrator, and planner-oriented approaches for coordinating specialized agents. Why: Provides implementation patterns for choosing who owns delegation and how results return to the coordinating node.	Official documentation Handoffs
📚 Google ADK: Graph-based Agent Workflows Docs · Graph workflows	Google ADK Documentation Google · 2026	Documents declarative workflows whose nodes combine agents, tools, functions, and human input through explicit edges, typed data passing, routing, branching, state, fan-out and join, loops, escalation, and nesting. Why: Makes the graph load-bearing and inspectable while separating deterministic process control from model reasoning.	Official documentation Work graphs
📚 Microsoft Agent Framework: Workflows Docs · Graph workflows	Microsoft Learn Microsoft · 2026	Describes workflows built from executors and explicit edges, with support for branching, aggregation, state, and checkpointing. Why: Exposes the work graph as an inspectable program rather than hiding coordination inside prompts.	Official documentation Work graphs
📚 LangGraph overview Docs · Graph runtime	LangGraph Documentation LangChain · 2025	Introduces a low-level runtime for stateful agent graphs with durable execution, streaming, memory, and human intervention. Why: A widely used substrate for implementing explicit nodes, edges, state transitions, and resumable work graphs.	Official documentation Work graphs
📄 MASFactory: A Graph-centric Framework for Orchestrating LLM-Based Multi-Agent Systems with Vibe Graphing Paper · Graph-centric orchestration	ACL System Demonstrations Yang Liu et al. · 2026	Compiles natural-language intent into an editable workflow specification and executable directed graph, with reusable components, topology preview, runtime tracing, multimodal messages, and human interaction. Why: Provides a direct implementation path from an inspectable organizational graph to execution and evaluates it on seven public benchmarks.	Peer-reviewed research Work graphs
📚 AutoGen GraphFlow (Workflows) Docs · Directed agent graphs	Microsoft AutoGen Documentation Microsoft · 2026	Implements directed multi-agent execution graphs with sequential, parallel, conditional, fan-in, and cyclic paths, edge conditions, activation groups, safe loop exits, and separately configurable message filtering. The feature is explicitly experimental. Why: Distinguishes the execution graph from the message graph, exposing both who acts next and what context each agent receives.	Official documentation Work graphs
📝 Build reliable multi-agent applications with ADK Go 2.0 Blog · Graph workflow runtime	Google Developers Blog Toni Klopfenstein; Sampath Kumar Maddula · 2026	Documents a graph-based multi-agent workflow engine with typed nodes, conditional edges, fan-out and fan-in, nested graphs, cycles, retries, durable human input, state, and unified telemetry. Why: Provides a first-party implementation of explicit, resumable work graphs while showing that function and tool nodes remain distinct from agent nodes.	Practitioner analysis Work graphs
📄 MARTI: A Framework for Multi-Agent LLM Systems Reinforced Training and Inference Paper · Multi-agent reinforced training	ICLR Kaiyan Zhang; Kai Tian; Runze Liu; Sihang Zeng; Xuekai Zhu; Guoli Jia; Yuchen Fan; Xingtai Lv; Yuxin Zuo; Che Jiang; Yuru Wang; Jianyu Wang; Ermo Hua; Xinwei Long; Junqi Gao; Youbang Sun; Zhiyuan Ma; Ganqu Cui; Ning Ding; Biqing Qi; Bowen Zhou · 2026	Supports centralized multi-agent interaction, distributed policy training, asynchronous rollouts, graph-defined workflows, heterogeneous models, tools, and configurable rewards. Why: Connects executable organizations to reinforced training infrastructure so graph policies can be optimized and evaluated rather than remaining hand-authored inference code.	Peer-reviewed research Evolution
📚 AutoGen Core Docs · Event-driven agent runtime	Microsoft AutoGen Documentation Microsoft · 2026	Documents an actor-style agent runtime with asynchronous messages, topics, subscriptions, distributed execution, resilience mechanisms, and telemetry. Why: Adds the lower-level message and runtime architecture beneath conversational teams, clarifying how agent nodes communicate and scale across processes.	Official documentation Handoffs
📚 Multiagent orchestration Docs · Managed multi-agent orchestration	Claude Platform Documentation Anthropic · 2026	Documents managed multi-agent orchestration with a lead agent that creates specialized subagents, delegates work, and combines their results under a beta API contract. Why: Provides a current first-party implementation of dynamic specialization and hierarchical delegation while clearly exposing its managed-runtime boundary and beta status.	Official documentation Topology
🧰 Gas Town (multi-agent workspace manager) Tool · Agent-organization pre-history	GitHub Steve Yegge and contributors · 2026	The Go-based open-source implementation of Yegge's agent organization: orchestrates dozens of Claude Code instances under the Mayor/Crew/polecat role structure with git-persisted state so work survives crashes. Reading the role definitions is the fastest way to see a hardcoded org graph before frameworks made them programmable. Why: A working implementation of a coordinated agent fleet, useful as a reference for how such a system is actually wired.	Maintained OSS project Topology
🧰 Dapr Agents Tool · Cloud-native runtimes	GitHub Dapr project / CNCF (Yaron Schneider et al.) · 2026	CNCF's cloud-native agent framework, GA 1.0 at KubeCon Europe (March 2026): every agent invocation runs as a durable Dapr workflow on the virtual actor model, giving scale-to-zero with ~3ms activation, thousands of agents per core, and 30+ pluggable state stores. The strongest answer for teams who want agent graphs governed by Kubernetes-native operational discipline. Why: Every invocation as a durable workflow gives an organization crash-resistance from the platform rather than from prompts.	Maintained OSS project Reliability
🧰 smolagents Tool · Code-acting frameworks	GitHub Hugging Face (Aymeric Roucher et al.) · 2026	Hugging Face's minimalist library whose CodeAgent writes actions as Python code rather than JSON tool calls, giving composability (loops, conditionals, function nesting) for free. Hierarchical multi-agent systems are built by passing named agents as managed_agents to a manager agent — a lightweight manager-worker graph in a few dozen lines, popular for research and as the basis of the HF Agents Course. Why: Writing actions as code rather than tool JSON collapses many planning steps into one, which changes what a node needs to be.	Maintained OSS project Work graphs
🧰 AG2 Tool · Conversation-based frameworks	GitHub Chi Wang, Qingyun Wu, AG2 community · 2026	Community continuation of AutoGen led by original paper authors Chi Wang and Qingyun Wu, with independent open governance. Actively developed (v0.12.x as of May 2026, path to 1.0 announced) and conversation-centric: group chats, nested chats, and swarm-style handoffs rather than explicit DAGs. The choice for teams that want AutoGen's model without Azure alignment. Why: The community continuation of a foundational framework, worth tracking when deciding which lineage to build on.	Maintained OSS project Topology
🧰 AutoGen (maintenance mode) Tool · Conversation-based frameworks	GitHub Microsoft Research (Chi Wang, Qingyun Wu et al.) · 2025	The framework that popularized conversation-as-orchestration — agents coordinate by talking in group chats rather than traversing explicit edges — and seeded a whole research lineage (AutoGen Studio, Magentic-One). Placed in maintenance mode in October 2025: no new features, community-managed, with Microsoft directing new projects to Agent Framework. Still essential reading as the conversational pole of the graph-model design space. Why: Conversation-as-orchestration shaped the field's early vocabulary, and its maintenance status matters for anyone still on it.	Maintained OSS project Topology
🧰 AgentKit by Inngest Tool · Durable execution	agentkit.inngest.com Inngest · 2026	TypeScript library for multi-agent Networks with a distinctive router abstraction: routing between agents ranges from fully code-based (deterministic) to LLM-based (autonomous), decided each cycle against a typed shared State. Runs on Inngest's durable execution runtime, so every agent step is retried and resumable without separate orchestration infrastructure. Why: Router-first design makes the routing decision an explicit, testable component instead of an emergent behavior.	Maintained OSS project Work graphs
📚 DBOS OpenAI Agents integration Docs · Durable execution	docs.dbos.dev DBOS Inc. · 2026	Postgres-backed durable execution as a library: DBOSRunner.run is a drop-in replacement for the OpenAI Agents SDK Runner, and @DBOS.workflow/@DBOS.step decorators checkpoint every agent and tool step into a database — about 10 lines to make a multi-agent app crash-proof. Also a first-class durable backend for Pydantic AI. Why: Durable execution as a library, so a graph gains replay without adopting a separate orchestration service.	Official documentation Reliability
📝 Temporal for AI agents Blog · Durable execution	temporal.io Temporal Technologies (Maxim Fateev, Samar Abbas) · 2026	The heavyweight durable-execution engine as agent-graph substrate: every step is journaled so multi-agent workflows resume exactly where they crashed, with Nexus (GA) connecting workflows across isolated namespaces — organizational boundaries as first-class infrastructure. The OpenAI Agents SDK integration went GA in March 2026, and Replay 2026 announced serverless workers aimed squarely at agent workloads. Why: Journaled steps let a multi-agent run survive process death, which is the difference between a demo and a service.	Practitioner analysis Reliability
🧰 OWL (Optimized Workforce Learning) Tool · Dynamic workforce frameworks	GitHub CAMEL-AI.org · 2026	CAMEL-AI's workforce framework for real-world task automation: a coordinator dynamically assembles and routes worker agents (browser, document, multimodal) per task — a working example of an evidence-driven dynamic work graph. Scored 69.09% average on GAIA, ranking #1 among open-source frameworks at release. Why: A coordinator that assembles and routes a workforce dynamically, useful as a reference for evidence-driven staffing.	Maintained OSS project Roles
📚 LlamaIndex Workflows (llama-agents) Docs · Event-driven frameworks	developers.llamaindex.ai LlamaIndex (Jerry Liu et al.) · 2026	Event-driven, async-first orchestration where steps subscribe to typed events rather than wiring explicit edges — an implicit-graph alternative that makes fan-out, loops, and routing natural. Workflows 1.0 (2025) made it a standalone lightweight framework; AgentWorkflow layers multi-agent handoffs on top, and llama-deploy runs workflows as distributed services in production. Why: Typed events rather than explicit edges is a genuinely different way to express a work graph, and worth comparing against.	Official documentation Work graphs
📝 AI Agent Frameworks (2026 Update): 8 SDKs Compared Blog · Framework comparisons	morphllm.com Morph · 2026	Side-by-side comparison of the eight major agent SDKs current to 2026 — LangGraph, CrewAI, OpenAI Agents SDK, Claude Agent SDK, ADK, and peers — including a primitive-by-primitive Claude Agent SDK reference. Useful as a fast orientation map before committing to a graph model (explicit DAG vs handoff vs conversation). Why: A current side-by-side of the major SDKs, useful when a framework choice has to be justified to other people.	Practitioner analysis Topology
📝 Two Lineages, One Framework: How AutoGen and Semantic Kernel Became the Microsoft Agent Framework Blog · Framework succession analysis	alexbevi.com Alex Bevilacqua · 2026	Clear-eyed June 2026 account of the AutoGen/Semantic Kernel convergence: why Microsoft abandoned free-form agent conversation for typed graph workflows, what the migration actually costs, and how to decide between AG2 and Agent Framework. The best single explainer for the most consequential framework succession of 2025-26. Why: Explains why a major vendor abandoned free-form agent conversation for explicit structure, which is an argument about topology.	Practitioner analysis Topology
📚 LangGraph: Workflows and Agents Docs · Graph patterns	docs.langchain.com LangChain · 2026	The doc page that codifies the workflow-vs-agent spectrum — prompt chaining, parallelization, routing, orchestrator-worker, evaluator-optimizer — as composable graph patterns, directly mirroring Anthropic's 'Building effective agents' taxonomy in runnable LangGraph code. The most-cited single reference for choosing an edge topology before writing any agent code. Why: Codifies the workflow-to-agent spectrum, giving a team language for how much autonomy each node actually needs.	Official documentation Topology
🧰 AWS Strands Agents Tool · Graph-native frameworks	strandsagents.com AWS · 2026	AWS's open-source SDK (1.0 July 2025, ~16.7M downloads/month by June 2026) with three composable multi-agent primitives that map directly onto graph engineering vocabulary: Graph (explicit deterministic DAG), Swarm (agents autonomously hand off within a pool), and Workflow — and they nest, so graphs can orchestrate swarms and vice versa. Python and TypeScript; used internally by Amazon Q Developer. Why: Three composable multi-agent primitives in one SDK, useful for comparing patterns without switching stacks.	Maintained OSS project Topology
📚 Google Agent Development Kit (ADK) Docs · Graph-native frameworks	adk.dev Google · 2026	Google's open-source agent framework, now at 2.0 with first-class graph workflows alongside its original workflow-agent primitives (SequentialAgent, ParallelAgent, LoopAgent, LLM-driven sub-agent delegation) across Python, TypeScript, Go, Java, and Kotlin. Deploys to Vertex AI Agent Engine, Google Cloud's managed agent runtime, and speaks A2A for cross-vendor agent-to-agent edges; Python repo neared 20K stars by mid-2026. Why: First-class graph workflows alongside older agent primitives, showing how a major framework converged on explicit structure.	Official documentation Topology
🧰 LangGraph Tool · Graph-native frameworks	GitHub LangChain (Harrison Chase et al.) · 2026	The reference point for graph-native agent orchestration: agents and functions are nodes in a StateGraph, edges (including conditional edges) define control flow, and a checkpointer gives every graph durable, resumable state with human-in-the-loop interrupts. Hit 1.0 in late 2025 and ships prebuilt supervisor and swarm topologies (langgraph-supervisor, langgraph-swarm), making it the cleanest explicit-DAG expression of the org-graph/work-graph idea in production today. Why: The reference implementation of graph-native orchestration, and the vocabulary most coordination discussions borrow.	Maintained OSS project Topology
📚 OpenAI Agents SDK Docs · Handoff-based frameworks	openai.github.io OpenAI · 2026	Swarm's production successor (March 2025), built on the handoff model: agents transfer control peer-to-peer, carrying context across the transition — a dynamic-routing alternative to explicit DAGs. The April 2026 'Next Evolution' release added a subagent primitive (beta), native sandbox execution, and a long-horizon harness; guardrails, sessions, and built-in tracing round out the production story. Why: Handoffs as the core primitive make peer-to-peer control transfer the unit of design rather than an afterthought.	Official documentation Handoffs
🧰 Amazon Bedrock AgentCore Tool · Managed agent runtimes	aws.amazon.com AWS · 2026	AWS's framework-agnostic managed runtime for production agents (GA late 2025): serverless session isolation, centralized Memory for state checkpointing across a multi-agent stack, Gateway for tools, Policy for permission boundaries, and A2A support for supervisor/worker topologies. The successor bet to Bedrock Agents' built-in multi-agent collaboration — run any org graph (Strands, LangGraph, CrewAI nodes) on managed infrastructure. Why: Managed runtime with session isolation removes a whole class of state and sandboxing work from an organization's operators.	Maintained OSS project Reliability
📚 Anthropic Managed Agents Docs · Managed node runtimes	Claude Platform Docs Anthropic · 2026	Hosted REST counterpart to the Agent SDK: Anthropic runs the agent loop and a managed sandbox per session, your application streams events in and results out, with session state kept in an Anthropic-hosted event log. Turns each agent node into a managed service — relevant when your org graph's nodes should not carry their own infrastructure. Why: A hosted agent loop with a managed sandbox per session, which changes the build-versus-buy line for node infrastructure.	Official documentation Reliability
📚 Claude Agent SDK Docs · Node runtimes / subagents	Claude Docs Anthropic · 2026	Anthropic's library exposing the Claude Code harness — tools, the agent loop, and context management — as a Python and TypeScript SDK, and the canonical building block for an agent as a node runtime. Subagents are defined declaratively with their own prompt, tool restrictions, and model choice, then invoked through the Agent tool, alongside hooks, sessions, and MCP support. Why: Exposes a production coding harness as a library, so a node inherits proven tools and context management.	Official documentation Roles
📝 A harness for every task: dynamic workflows in Claude Code Blog · Orchestration-as-code	claude.com Anthropic · 2026	Anthropic's design rationale for the Workflow tool: instead of the model narrating coordination turn-by-turn, Claude writes a short orchestration script on the fly that spawns and sequences fleets of subagents deterministically — coordination logic runs outside the context window at zero token cost. A key primary source for the shift from loop engineering to programmable work graphs. Why: The design rationale for deterministic code orchestrating agents, which is the strongest current alternative to model-narrated coordination.	Practitioner analysis Work graphs
🧰 CrewAI Tool · Role-based frameworks	GitHub João Moura / CrewAI Inc. · 2026	Role-based multi-agent framework built on two primitives that map cleanly to org graph vs work graph: Crews (stable teams of role-playing agents with tasks and a process model) and Flows (event-driven, deterministic orchestration with typed state, the recommended production wrapper since 2025). Claims 60% of the US Fortune 500 and ~450M agent executions/month as of early 2026; ~54K GitHub stars. Why: Two primitives that map cleanly onto standing roles and per-run execution, which is why it reads as an org-graph framework.	Maintained OSS project Roles
🧰 MetaGPT Tool · SOP-based org graphs	GitHub DeepWisdom / Foundation Agents (Sirui Hong, Chenglin Wu et al.) · 2026	The 'AI software company' framework: encodes Standard Operating Procedures into prompt sequences so specialized roles (PM, architect, engineer, QA) pass structured artifacts along a fixed org graph — the clearest early demonstration that organizational structure, not just agent capability, drives output quality (ICLR 2024 oral). Commercialized as MGX (MetaGPT X), which hit $1M ARR within a month of its 2025 launch. Why: Encodes standard operating procedures into role sequences, the clearest example of process-as-topology.	Maintained OSS project Roles
🧰 Letta Tool · Stateful agent platforms	letta.com Letta (Charles Packer, Sarah Wooders) · 2026	The MemGPT lineage as an agent platform: agents are persistent server-side entities that self-manage an OS-inspired memory hierarchy (core/recall/archival), which makes them natural long-lived nodes in a larger org graph — shared memory blocks give multiple agents a common substrate. 2026 work pushed further with git-based Context Repositories (February) and RL-trained memory models (June). Why: Agents as persistent server-side entities make memory ownership an architectural property rather than a prompt convention.	Maintained OSS project State
🧰 Agno Tool · Team-based frameworks	agno.com Agno (Ashpreet Bedi) · 2026	High-performance Python framework (rebranded from Phidata in January 2025, ~43K stars) organized around Agents, Teams, and Workflows plus AgentOS, an HTTP runtime that owns durable state. Its pitch is scale of the work graph itself: ~2 microsecond agent instantiation at 3.75 KiB each, enabling thousands of concurrent agent sessions on modest hardware. Why: A performance-oriented framework organized around teams, useful when coordination overhead itself is the constraint.	Maintained OSS project Topology
🧰 Mastra Tool · TypeScript frameworks	mastra.ai Mastra (Sam Bhagwat, Abhi Aiyer, Shane Thomas) · 2026	The leading TypeScript agent framework (from the Gatsby founders, YC W25, v1.0 January 2026) with a graph-based workflow engine — .then(), .branch(), .parallel() — plus suspend/resume that persists execution state so workflows can pause indefinitely for human input. July 2026 added Goals: durable objectives for long-running agents with LLM-judged evaluations. Why: The strongest TypeScript option, which matters because node language choice constrains who can operate the graph.	Maintained OSS project Topology
📚 Microsoft Agent Framework Docs · Typed-graph frameworks	Microsoft Learn Microsoft (AutoGen + Semantic Kernel teams) · 2026	Microsoft's unified successor to both AutoGen and Semantic Kernel, GA 1.0 on April 3, 2026, replacing AutoGen's free-form conversations with a typed graph-based workflow model (executors and edges, checkpointing, request/response gates) for .NET and Python. This is the framework Microsoft now recommends for all new multi-agent work, with Azure AI Foundry as its managed runtime. Why: The unified successor to two major frameworks, and the migration target anyone on either lineage now faces.	Official documentation Topology
📚 Pydantic AI + pydantic-graph Docs · Typed-graph frameworks	ai.pydantic.dev Pydantic team (Samuel Colvin et al.) · 2026	Pydantic AI (V1 September 2025, ~21K stars) runs every agent on pydantic-graph, an async state-machine library where nodes and edges are defined with type hints — the most type-rigorous way to express an agent graph in Python. First-class durable execution adapters for Temporal, DBOS, and Restate let the same graph survive crashes and long human-in-the-loop waits. Why: Every agent running on a typed state machine makes execution structure inspectable by construction.	Official documentation Work graphs
🧰 Dify Tool · Visual graph builders	GitHub LangGenius · 2026	Open-source LLMOps platform (106K+ stars, the fastest-growing visual builder) combining a drag-and-drop agent workflow canvas with RAG, observability, and deployment in one product. The pragmatic on-ramp for teams that want visual work graphs with production monitoring before graduating to code-first frameworks. Why: A visual builder with real adoption, useful for teams whose graph operators are not all engineers.	Maintained OSS project Topology
🧰 n8n Tool · Visual graph builders	GitHub n8n GmbH (Jan Oberhauser) · 2026	The dominant visual workflow canvas (182K+ stars, seven years of development) that grew a first-class AI Agent node plus vector-store, memory, and tool-calling nodes — agents become nodes inside larger ops automations, which is exactly how most enterprises first encounter work graphs. Fair-code licensed and self-hostable; best when AI is one step in a broader automation rather than the whole system. Why: A mature workflow canvas that grew agent nodes, which is often the pragmatic path for organizations with existing automation.	Maintained OSS project Work graphs
🧰 Sim Tool · Visual graph builders	GitHub Sim Studio (Emir Karabeg, Waleed Latif) · 2026	Open-source agent canvas (~30K stars) whose block system covers genuinely agentic graph logic — Agent, Router, Condition, Parallel, Loop, Evaluator, Guardrails, and nested Workflow blocks — rather than linear automation; you can also describe the desired graph in plain English and have it constructed. Self-hostable with Ollama/vLLM support; reports 100K+ builders and production customers including Rivian. Why: Block-level agentic logic in a canvas, a useful middle ground between code frameworks and no-code tools.	Maintained OSS project Topology
📚 CrewAI Agents: Role, Goal, Backstory Docs · Role definition frameworks	docs.crewai.com CrewAI · 2026	Documentation for the most popular role-first framework: every agent is declared with a required role (function and expertise), goal (decision-making objective), and backstory (context and personality), composed into crews with sequential or hierarchical processes. The de facto standard for expressing an org graph's roles as declarative config, used in production well before the graph-engineering label existed. Why: The role, goal, and backstory contract shows what a framework considers the minimum specification of a node.	Official documentation Roles
📚 Claude Code: Create Custom Subagents Docs · Role definition tooling	code.claude.com Anthropic · 2026	Official docs for defining named agent roles as configuration: each subagent gets its own system prompt, context window, tool allowlist, permission mode, and model choice, and the main agent delegates when a task matches the role's description. The most widely used mechanism in 2026 for encoding an org graph's 'who' — reusable role definitions that persist across projects. Why: Defining roles as configuration with their own prompts and tools is role design made version-controllable.	Official documentation Roles
📚 Orchestrator and Subagent Multi-Agent Patterns Docs · Supervisor pattern guidance	Microsoft Learn Microsoft · 2026	Microsoft's guidance on hierarchical orgs where an orchestrator delegates to child agents or externally-owned 'connected agents' maintained by other teams — an explicit treatment of independent ownership and reuse as org-design criteria. Unusually honest about when NOT to use the pattern: compliance-grade consistency needs, well-defined step sequences, and latency-sensitive flows. Updated February 2026. Why: Vendor guidance on hierarchical delegation, useful as a baseline for what a supervisor node is expected to do.	Official documentation Roles
🧰 langgraph-supervisor: Hierarchical Multi-Agent Systems Tool · Supervisor pattern implementation	GitHub LangChain · 2025	LangChain's library for building supervisor orgs on LangGraph: a central supervisor routes tasks to specialist worker agents, and multi-level hierarchies are built by making supervisors manage supervisors. The repository was archived in September 2026 and last pushed on 15 July 2026, so it is a frozen reference implementation rather than a maintained dependency; LangChain now recommends building the supervisor pattern directly with tool calling for finer context control, and points to its own multi-agent guide and supervisor tutorial. Why: Still the clearest compact reading of the supervisor pattern, now with its own maintainers' verdict on when a library is the wrong shape for it.	Maintained OSS project Topology
📚 LangGraph Graph API: branching, map-reduce, and the Send API Docs · Dynamic fan-out/fan-in	docs.langchain.com LangChain · 2026	Reference for building work graphs whose fan-out width is decided at runtime: Send dispatches a dynamic number of parallel branches with per-branch state (map), superstep barriers gather them (reduce), and state reducers merge concurrent writes. The standard mechanism for evidence-dependent node spawning in the most widely deployed graph framework. Why: Runtime-decided fan-out width is the practical primitive for work graphs whose size is unknown until execution.	Official documentation Work graphs
📚 CrewAI Flows Docs · Event-driven orchestration	docs.crewai.com CrewAI · 2026	Event-driven orchestration layer (@start, @listen, @router decorators) that wraps autonomous agent crews inside precise procedural control flow with typed state and conditional routing. A widely used middle path between fully model-driven crews and hand-written pipelines. Why: An event-driven layer wrapping autonomous agents, showing how deterministic control and agent judgment can coexist.	Official documentation Work graphs
🧰 deepagents Tool · Planning harness	GitHub LangChain · 2025	Batteries-included agent harness distilling the Claude Code recipe into a library: a write_todos planning tool for explicit task lists, a filesystem for context offloading, and a task tool that spawns ephemeral subagents with fresh context returning a single report. A minimal, hackable implementation of planner-holds-the-list work-graph management on the LangGraph runtime. Why: Distills a proven harness recipe into a library, including planning and delegation as built-in components.	Maintained OSS project Work graphs
📝 Graph-Based Agent Workflow Orchestration in Production: The 2026 Landscape Blog · Production DAG orchestration	zylos.ai Zylos Research · 2026	Survey of production graph orchestration as of April 2026, comparing LangGraph, Temporal, Mastra, Microsoft Agent Framework 1.0, and the OpenAI Agents SDK on typed state, conditional routing, checkpoints, and observability. Argues the graph is the specification of what the system is allowed to do, expressed as code. Why: Compares production graph orchestrators against each other, which is what a framework decision actually requires.	Practitioner analysis Topology
📚 Claude Code: Orchestrate subagents at scale with dynamic workflows Docs · Script-driven orchestration	code.claude.com Anthropic · 2026	Official docs for the clearest shipping example of deterministic-code orchestration: Claude writes a JavaScript workflow script that holds the plan, fans out up to 1,000 subagents (16 concurrent) via agent() and pipeline(), keeps intermediate results in script variables instead of any context window, and supports resume with cached agent results. The comparison table of who-holds-the-plan across subagents, skills, agent teams, and workflows is a compact statement of the model-driven vs script-driven orchestration tradeoff. Why: The clearest shipping example of code orchestrating agents deterministically, with fan-out and verification expressed as ordinary control flow.	Official documentation Work graphs
🧰 joonspk-research/generative_agents Tool · Generative agent societies	GitHub Joon Sung Park et al. · 2023	The released Smallville codebase, including the simulation server, agent cognitive modules, and a replay viewer that lets you step through a recorded run agent-by-agent. It is the most-forked starting point for anyone building their own agent town, and its replay artifact is an early example of treating a run as an inspectable record rather than a transcript. The repository has not been updated since August 2024 and is a frozen research artifact rather than an actively developed project. Why: A runnable society plus a replay viewer, which makes a multi-agent run an inspectable record rather than a wall of transcript.	Maintained OSS project Roles
🧰 AI Town Tool · Generative agent societies	GitHub a16z Infrastructure / AI Stack Devs · 2023	MIT-licensed JS/TS starter kit for a virtual town of AI characters that walk, converse, and remember, built on Convex so the back end natively supports shared global state, transactions, and a deterministic simulation engine. Defaults to local inference via Ollama, making agent-society experiments cheap to run without API spend. Why: A cheap local-inference starting point for society experiments, built on a backend that gives shared state real transactions.	Maintained OSS project State
🧰 google-deepmind/concordia Tool · Simulation platforms	GitHub Google DeepMind · 2023	Apache-licensed library for generative agent-based models, structured around Entities, composable Components (memory, chain-of-thought, logic), and an Engine that runs the loop. Component-level composition means an agent's reasoning structure is declared as data you can diff and re-run, not hand-written prose. Why: Component-level composition means an agent's reasoning structure is declared as data you can diff and re-run.	Maintained OSS project Topology
🧰 GraphCode Tool · Session-graph workspace	GitHub Srinivasa Chaitanya Gopireddy and contributors · 2026	A macOS app for Apple Silicon that lays out coding-agent sessions as a graph. Each node is an agent loop of one of four kinds: turn-based, goal-based with an optional shell predicate, time-based, or a composite sub-graph of loops. Edges hand off, message, or spawn, each with a condition and a cycle guard. Nodes are live terminal sessions that survive app restarts and can be steered mid-run, and a background daemon fires hand-off edges and polls goal predicates. Works with Claude Code, Codex, Copilot CLI, OpenCode, and pi. The app is source-available under FSL-1.1-MIT and converts to MIT two years after each release; its kit and CLI are MIT. Why: The run graph is the operator's working surface and each node is itself a loop, which makes the graph-of-loops idea concrete and inspectable.	Maintained OSS project Handoffs
🧰 deer-workflow Tool · Orchestration-as-code	GitHub deerwork-ai · 2026	MIT-licensed TypeScript runtime on Bun in which code defines the valid execution paths, phases, inputs, and failure handling, while coding agents do the semantic work inside each node. A bundled skill asks an agent to generate a runnable workflow module from a plain-language description; Codex is the default agent, with Claude Code and Pi built in. Interactive runs show a phase-aware terminal UI, and automation can consume a stable JSONL event stream. Why: Keeps the work graph in reviewable code and the agents replaceable, the opposite trade from letting a conversation decide control flow.	Maintained OSS project Work graphs
🧰 headcount Tool · Installable org graph	GitHub cbrock84 and contributors · 2026	An agent organization shipped as a company: 16 departments and 172 skills addressed as department:skill, such as security:threat-modeling, each department installable on its own from a Claude Code plugin marketplace, and usable from ChatGPT and Codex by dropping departments into a skills directory. Security and Legal & Risk are reviewer-class lanes that report to the chief executive rather than into the functions they review, and the README states their blocking findings are not overrulable by the department under review. Every path maps to exactly one owning department in a surfaces document, so two agents cannot both claim the same file. MIT, with a searchable org chart page. Why: A worked org graph you can install department by department, where review authority and exclusive write surfaces are stated as checkable rules rather than described in prose.	Maintained OSS project Roles
📚 OpenAI Responses API: Multi-agent Docs · Managed multi-agent orchestration	OpenAI API docs OpenAI · 2026	Beta mode of the Responses API in which a root agent spawns subagents, sends them further information, waits for their results, and synthesizes the final answer, with no orchestration code in the calling application. Each subagent keeps its own context and is addressed by a path such as /root/reviewer/tester, and output items record which agent produced them. max_concurrent_subagents caps active subagent turns across the whole tree, excluding the root, and defaults to 3; there is no fixed limit on tree depth or on the total number of subagents. The guide lists when not to use it: steps that each depend on the previous one, small tasks, agents contending over the same mutable resource, or a requirement for a fixed, deterministic execution graph. Why: A model-directed run graph offered as an API primitive, documented together with the cases where a declared graph is the better choice.	Official documentation Topology

Protocols & Handoffs

Resource	Source	What it contributes	Evidence
📚 OpenAI Agents SDK: Handoffs Docs · In-process transfer	OpenAI Agents SDK Documentation OpenAI · 2026	Documents transfers from one agent to another, including tool-shaped handoff schemas, input filters, and callbacks. Why: Turns an edge into an explicit contract controlling when ownership moves and what context crosses with it.	Official documentation Handoffs
📐 Model Context Protocol Specification 2025-11-25 Standard · Tool and context protocol	MCP Specification 2025-11-25 Model Context Protocol project; Agentic AI Foundation · 2025	Specifies a client-server protocol through which AI applications discover and use tools, resources, prompts, and contextual data. Why: Standardizes capability and context edges, while remaining distinct from a protocol for delegating work between autonomous agents.	Industry standard Handoffs
📐 Agent2Agent Protocol Specification v1.0.0 Standard · Agent interoperability	A2A Specification A2A Protocol Working Group; Linux Foundation · 2026	Defines interoperable agent discovery, task lifecycle, messages, artifacts, streaming, asynchronous updates, version negotiation, and multiple protocol bindings across service boundaries. Why: Provides a version-pinned wire contract for cross-system handoffs where agents cannot share an in-process runtime.	Industry standard Handoffs
📐 Model Context Protocol: Tasks Standard · Durable remote tasks	MCP Specification 2025-11-25 Model Context Protocol project; Agentic AI Foundation · 2025	Specifies experimental durable asynchronous request state machines with capability negotiation, polling, deferred results, progress, input-required states, cancellation, TTLs, and task-message correlation. Why: Turns a remote tool or context edge into a recoverable task contract that can outlive one synchronous request.	Official documentation Reliability
📚 Secure Low-Latency Interactive Messaging (SLIM) Docs · Secure agent messaging	IETF Datatracker; individual Internet-Draft Luca Muscariello; Michele Papalini; Mauro Sardara; Sam Betts · 2026	Proposes a transport layer for A2A and MCP using gRPC over HTTP/2 and HTTP/3 with stream multiplexing, flow control, group communication, native RPC semantics, and MLS end-to-end encryption. It is an individual informational Internet-Draft with no formal IETF standing. Why: Adds a concrete secure transport substrate for high-volume graph edges that must cross process and organizational boundaries.	Official documentation Handoffs
📚 Accelerating the Adoption of Software and Artificial Intelligence Agent Identity and Authorization Docs · Agent identity and authorization	NIST NCCoE Initial Public Draft Harold Booth; William Fisher; Ryan Galluzzo; Joshua Roberts · 2026	Outlines considerations and open questions for standards-based identity, authorization, auditing, and non-repudiation when software and AI agents access enterprise systems and take actions. The concept paper remains an initial public draft under review. Why: Grounds graph roles and permissions in identity practice so delegation does not silently transfer more authority than an edge contract allows.	Official documentation Roles
📚 Identity Management for Agentic AI: The new frontier of authorization, authentication, and security for an AI agent world Docs · Agent identity and delegated authorization	OpenID Foundation OpenID Foundation Artificial Intelligence Identity Management Community Group · 2025	Maps identity, authentication, delegated authority, consent, governance, and accountability gaps for autonomous agents against OAuth and OpenID Connect foundations. Why: Provides a standards-grounded vocabulary for identity-bearing nodes and auditable delegation across agent and tool edges; it is a non-normative landscape whitepaper, not a protocol.	Official documentation Handoffs
📐 Open Agentic Schema Framework (OASF) v1.1.0 Standard · Agent capability and discovery schema	AGNTCY Project, Linux Foundation AGNTCY Project · 2026	Defines versioned schemas and taxonomies for agent identity metadata, capabilities, skills, modules, interactions, and discovery records, including MCP and A2A alignment modules. Why: Gives graph builders a portable node descriptor for capability-based discovery and routing validation; it describes capabilities but does not execute a graph or authorize an edge.	Industry standard Roles
🧰 AGNTCY Dir Tool · Distributed agent discovery	AGNTCY Project, Linux Foundation AGNTCY Project · 2026	Implements a distributed directory for capability-based discovery of agents and multi-agent systems using OASF records and cryptographic identity. Why: Covers how potential graph participants are registered, discovered, authenticated, and selected before a work or communication edge is created.	Maintained OSS project Handoffs
📐 FIPA ACL Message Structure Specification Standard · Agent message envelopes	Foundation for Intelligent Physical Agents FIPA TC Communication; FIPA Architecture Board · 2002	Defines interoperable agent messages with performatives, sender and receiver, ontology, protocol, conversation identifiers, reply correlation, and deadlines. Why: Supplies the historical standards basis for message-envelope and conversation concepts that reappear in current agent handoff protocols.	Industry standard Handoffs
📄 µACP: A Formal Calculus for Expressive, Resource-Constrained Agent Communication Paper · Resource-bounded communication calculus	AAMAS Arnab Mallick; Indraveni Chebolu · 2026	Formalizes resource-bounded agent communication with PING, TELL, ASK, and OBSERVE operations, proves finite-state expressiveness and message bounds, and checks properties with TLA+ and Coq. Why: Provides a precise substrate for reasoning about protocol behavior and communication budgets; it is a general multi-agent calculus rather than an LLM-specific runtime.	Peer-reviewed research Handoffs
🎓 A2A: The Agent2Agent Protocol (DeepLearning.AI course) Course · A2A	DeepLearning.AI DeepLearning.AI with Google · 2025	Hands-on short course building A2A-connected agents: publishing Agent Cards, delegating tasks between agents built on different frameworks, and wiring A2A alongside MCP. The fastest structured path from zero to a working cross-framework handoff. Why: Hands-on practice publishing capability descriptions and delegating across agents, which is the edge contract made concrete.	Book/course Handoffs
📐 Agent2Agent (A2A) Protocol — Specification & SDKs Standard · A2A	GitHub A2A Project (Linux Foundation); originated at Google · 2025	The open protocol for peer-to-peer communication between opaque agents: JSON-RPC/gRPC/HTTP bindings, Agent Cards for capability discovery, task lifecycle with streaming and push notifications, and multi-language SDKs. Launched by Google in April 2025 with 50+ partners; now the de facto standard edge protocol for cross-vendor agent graphs, backed by 100+ companies. Why: The specification for peer-to-peer delegation between agents that do not share a codebase or vendor.	Industry standard Handoffs
📝 Linux Foundation Launches the Agent2Agent Protocol Project Blog · A2A	linuxfoundation.org Linux Foundation · 2025	The canonical record of Google donating A2A to the Linux Foundation (June 23, 2025), with AWS, Cisco, Microsoft, Salesforce, SAP, and ServiceNow as founding members. Matters because it moved the leading inter-agent edge protocol to vendor-neutral governance — the reason competing stacks now interoperate over it. Why: Records the governance move that made cross-vendor agent delegation a neutral standard rather than one company's protocol.	Practitioner analysis Handoffs
🧰 Waggle: Attributed Artifact References for Agent Handoffs Tool · Artifact passing	GitHub Modiqo · 2026	Rust-based MCP-native reference layer that replaces pasting full context across a handoff with a ~30-byte resolvable token: the consuming agent pulls a projection of the artifact sized to its context, with attribution, versioning, and append-only consumption tracking. v0.5.3 (July 2026) integrates with Claude Code, Codex, and Cursor — the clearest working example of artifact passing over message passing. Why: Passing attributed references instead of pasted context attacks handoff cost and provenance at the same time.	Maintained OSS project Handoffs
📐 Agent Network Protocol (ANP) Standard · Decentralized protocols	GitHub ANP open-source community (Gaowei Chang et al.) · 2025	Decentralized agent communication stack built on W3C DIDs (did:wba) for cross-platform identity without a central registry, plus a meta-protocol layer where agents negotiate their own wire protocols. The peer-to-peer, open-internet answer to A2A's enterprise model; the design informs the W3C AI Agent Protocol community group's white paper. Why: Decentralized identity for agents addresses the question every cross-organization graph eventually hits: who is this caller.	Maintained OSS project Handoffs
📚 AGNTCY — Internet of Agents Infrastructure Docs · Discovery & identity	agntcy.org Cisco Outshift, LangChain, Galileo; now Linux Foundation · 2025	Open infrastructure for cross-vendor agent organizations: OASF schema records for describing agents, a distributed Agent Directory for discovery, cryptographically verifiable agent Identity, and the SLIM low-latency messaging transport (request-response, pub-sub, streaming, quantum-safe). Donated to the Linux Foundation in July 2025 with Dell, Google Cloud, Oracle, and Red Hat as formative members. Why: Directory, identity, and schema infrastructure for organizations whose nodes span vendors.	Official documentation Handoffs
📄 Defeating Prompt Injections by Design (CaMeL) Paper · Edge security	arXiv Edoardo Debenedetti et al. (Google DeepMind, ETH Zurich) · 2025	CaMeL extracts control flow and data flow from the trusted user query so untrusted data flowing through the system can never alter program flow, and attaches capabilities to values to block unauthorized data flows — solving 77% of AgentDojo tasks with provable security. The blueprint for capability-tracked edges in agent graphs. Why: Separating control flow from untrusted data is the most principled defense available at an agent edge.	Research preprint Handoffs
📄 Design Patterns for Securing LLM Agents against Prompt Injections Paper · Edge security	arXiv Luca Beurer-Kellner et al. (IBM, Invariant Labs, ETH Zurich, Google, Microsoft) · 2025	Six principled patterns — including context minimization, plan-then-execute, dual LLM, and code-then-execute — sharing one rule: once a node ingests untrusted input, constrain it so that input cannot trigger consequential actions. Ten case studies; the standard vocabulary for designing quarantined edges between trusted and untrusted nodes. Why: Six named patterns give concrete options for hardening edges instead of hoping the model resists injection.	Research preprint Handoffs
📚 MCP Security Best Practices Docs · Edge security	modelcontextprotocol.io MCP core maintainers · 2025	Normative security guidance for MCP edges: confused-deputy attacks on proxy servers, the token-passthrough anti-pattern, SSRF in OAuth discovery, session hijacking, local-server compromise, and scope minimization. Required reading before exposing any node in a graph as an MCP server or client. Why: Normative guidance on the failure modes specific to tool edges, including confused-deputy and token-passthrough mistakes.	Official documentation Handoffs
📝 MCP Security Notification: Tool Poisoning Attacks Blog · Edge security	invariantlabs.ai Invariant Labs · 2025	The April 2025 disclosure that MCP tool descriptions are an injection surface: a malicious server's docstring can instruct the model to silently exfiltrate SSH keys or WhatsApp history through a benign-looking tool call, and can hijack tools from other connected servers. Spawned mcp-scan, the OWASP MCP Top 10 tool-poisoning category, and the MCPTox benchmark. Why: Tool descriptions are an injection surface, which means an edge can be compromised by the thing it connects to.	Practitioner analysis Handoffs
🧭 OWASP MCP Top 10 List · Edge security	OWASP OWASP Foundation · 2025	Community-maintained top-10 risk taxonomy for MCP deployments: token mismanagement, scope creep, tool poisoning, intent flow subversion, shadow MCP servers, and cross-context over-sharing, each with mitigations. Gives graph operators a shared checklist language for auditing tool edges. Why: A working risk checklist for tool-edge deployments, useful in review even if you disagree with the ranking.	Community resource Handoffs
📄 Prompt Infection: LLM-to-LLM Prompt Injection within Multi-Agent Systems Paper · Edge security	arXiv Donghyun Lee, Mo Tiwari · 2024	Shows a single injected prompt can self-replicate across agent edges like a worm — hijack, payload, data-collection, and replication components propagate to every downstream agent, succeeding at data theft and scam generation over 80% of the time on GPT-4 pipelines. The founding demonstration that edges are an attack propagation surface; proposes LLM tagging as a partial defense. Why: Shows an injected prompt self-replicating across agent edges, which is the worst case for an unguarded graph.	Peer-reviewed research Handoffs
📄 Security Threat Modeling for Emerging AI-Agent Protocols: MCP, A2A, Agora, and ANP Paper · Edge security	arXiv (February 2026 preprint) · 2026	Systematic comparative threat model of the four live protocol families, mapping protocol-specific risk surfaces from trust assumptions and lifecycle behavior — e.g., A2A Agent Card tampering where JSON metadata circulates without mandatory end-to-end signing. The current reference for choosing an edge protocol on security grounds. Why: Compares the live protocol families by risk surface, useful when the protocol choice is also a security decision.	Research preprint Handoffs
📝 The Lethal Trifecta for AI Agents Blog · Edge security	simonwillison.net Simon Willison · 2025	Names the combination that makes agents exploitable: private data access + exposure to untrusted content + external communication. In a graph the trifecta composes across nodes — an edge can hand one agent's untrusted content to another agent's exfiltration capability — making this the baseline threat model for deciding what may cross which edge. Why: Names the exact combination that makes an agent exploitable, and gives edge design a simple test to apply.	Practitioner analysis Handoffs
📝 Command: A New Tool for Building Multi-Agent Architectures in LangGraph Blog · Framework handoff primitives	LangChain Blog LangChain · 2024	Introduces the Command primitive — a node return value that combines a state update with a dynamic routing decision, replacing static conditional edges. This is the moment 'edges' in agent frameworks became programmable at runtime, a precondition for work graphs that rewrite themselves on evidence. Why: A primitive that makes a state update and a routing decision atomic, which is what a handoff should be.	Practitioner analysis Handoffs
📚 LangChain / LangGraph — Handoffs Docs · Framework handoff primitives	LangChain Docs LangChain · 2026	How handoffs work when agents are graph nodes: handoff tools return Command objects that atomically update shared state and route control (Command.PARENT for cross-subgraph jumps), with the gotcha that the AIMessage tool call must be paired with a ToolMessage acknowledgment. The most-deployed handoff implementation in production agent graphs. Why: Shows how handoffs work when agents are graph nodes, including who owns state at the moment control transfers.	Official documentation Handoffs
📄 Handoff Debt: The Rediscovery Cost When Coding Agents Take Over Interrupted Tasks Paper · Handoff contracts	arXiv (June 2026 preprint) · 2026	Quantifies what a receiving agent must re-discover when it takes over an interrupted task — naming 'handoff debt' as a measurable cost of imperfect context transfer between agents. Gives graph engineers an empirical handle on edge quality beyond anecdote. Why: Quantifies what a receiving agent must rediscover, which is the hidden cost that makes long chains lossy.	Research preprint Handoffs
📝 The 2026-07-28 MCP Specification Release Candidate Blog · MCP	MCP Blog MCP core maintainers · 2026	Announces the next MCP revision (final July 28, 2026): a stateless protocol core that scales on plain HTTP, a Tasks extension for long-running work, an Extensions framework, MCP Apps, and hardened OAuth alignment. The stateless core and Tasks are what make MCP viable as an edge substrate for large agent organizations, not just desktop tool plugging. Why: Where the dominant tool protocol is heading, including a stateless core that changes how edges scale.	Practitioner analysis Handoffs
📝 ACP Joins Forces with A2A Under the Linux Foundation Blog · Protocol convergence	LF AI & Data IBM Research / BeeAI and A2A teams · 2025	The official announcement that IBM's Agent Communication Protocol (launched March 2025 for BeeAI) wound down and merged into A2A, with migration guides for ACP users. The key succession event in the 2025 protocol wars — cite this instead of treating ACP as a live standard. Why: Consolidation record for the protocol landscape, so you do not build against a standard that has been wound down.	Practitioner analysis Handoffs
📚 A2A and MCP: Complementary Protocols Docs · Protocol layering	a2a-protocol.org A2A Project · 2025	The official statement of how the two dominant protocols divide the agent graph: MCP standardizes agent-to-tool edges, A2A standardizes agent-to-agent peer edges, and a typical node speaks both. Useful mental model for deciding whether a dependency should be modeled as a tool call or a delegated task to another agent. Why: The official division of labor between tool edges and agent-to-agent edges, which resolves a common architecture argument.	Official documentation Handoffs
📄 Agora: A Scalable Communication Protocol for Networks of Large Language Models Paper · Protocol negotiation	arXiv Samuele Marro, Emanuele La Malfa, Jesse Wright, Guohao Li, Nigel Shadbolt, Michael Wooldridge, Philip Torr (Oxford) · 2024	Formulates the Agent Communication Trilemma (versatility, efficiency, portability) and resolves it with negotiated Protocol Documents: routines for frequent exchanges, natural language for rare ones, LLM-written code in between. Demonstrated on a 100-agent network with 5x cost reduction — the strongest academic case that edges should negotiate their own contracts. Why: Names the tradeoff every communication protocol faces and offers a resolution, useful framing even if you pick differently.	Research preprint Handoffs
📄 A Survey of Agent Interoperability Protocols: MCP, ACP, A2A, and ANP Paper · Surveys & taxonomies	arXiv Abul Ehtesham, Aditi Singh, Gaurav Kumar Gupta, Saket Kumar · 2025	The most-cited comparison of the four 2025 protocol contenders, analyzing each by transport, discovery, session semantics, and security, and proposing a phased adoption path (MCP for tools, then A2A for collaboration, ANP for open networks). Read with the caveat that ACP has since merged into A2A. Why: The most-cited comparison of the protocol contenders, organized by transport, discovery, and security.	Research preprint Handoffs
📄 A Survey of AI Agent Protocols Paper · Surveys & taxonomies	arXiv Yingxuan Yang et al. (Shanghai Jiao Tong University) · 2025	Broad systematic survey of agent protocols with a two-dimensional taxonomy (context-oriented vs inter-agent; general-purpose vs domain-specific) and evaluation across efficiency, scalability, and security. The wider-angle companion to the four-protocol survey, covering dozens of protocols including agents.json, LMOS, and AITP. Why: A two-dimensional taxonomy that separates context edges from agent edges, which keeps protocol discussions coherent.	Research preprint Handoffs
📄 A Technical Taxonomy of LLM Agent Communication Protocols Paper · Surveys & taxonomies	arXiv Linus Sander, Habtom Kahsay Gidey, Alexander Lenz, Alois Knoll · 2026	June 2026 analysis of nine open-source protocols along five dimensions: counterparty, payload, interaction state, discovery, and schema flexibility. Finds all agent-to-agent protocols use hybrid payloads with persistent sessions, and predicts a federated layered protocol stack rather than one winner — the current best map of where edge protocols are converging. Why: Compares nine protocols along dimensions that matter in implementation rather than in marketing.	Research preprint Handoffs
📄 Beyond Message Passing: A Semantic View of Agent Communication Protocols Paper · Surveys & taxonomies	arXiv Dun Yuan, Fuyuan Lyu, et al. (McGill/MILA) · 2026	Evaluates 18 protocols across communication, syntactic, and semantic layers and lands the sharpest critique in the space: transport and schema support are maturing, but clarification, context alignment, and verification have no protocol-level support — so semantic coordination gets pushed into prompts. Directly frames why handoff quality, not wire format, is the open problem. Why: The sharpest critique of current protocols: they standardize transport while leaving meaning to convention.	Research preprint Handoffs
📐 AG-UI: the Agent-User Interaction Protocol Standard · UI protocol layer	GitHub CopilotKit team and framework partners · 2025	Open event-based protocol for the agent-to-user edge: the frontend POSTs once, then consumes a typed SSE event stream (messages, tool calls, state patches, human-in-the-loop interrupts) from any agent backend. Completes the protocol triad — MCP for tools, A2A for agents, AG-UI for users — and has first-party integrations in Microsoft Agent Framework, AG2, and LangGraph. Why: The agent-to-user edge deserves a contract too, and this is the protocol that gives it typed events.	Maintained OSS project Handoffs
📄 Governance Gaps in Agent Interoperability Protocols: What MCP, A2A, and ACP Cannot Express Paper · Governance gaps	arXiv Richard Kang, Yudho Diponegoro · 2026	Checks MCP, A2A, ACP, ANP, and ERC-8004 against six governance requirements drawn from organizational theory, multi-agent systems research, and enterprise governance standards: membership, deliberation, voting, dissent preservation, human escalation, and audit and replay. Voting and dissent preservation are absent from all five protocols, deliberation is absent or at most partial, and none covers the full set. Gaps are split into those an extension could close and those that need a new architectural layer. Why: Shows where today's interoperability protocols stop, so group decisions, dissent, and escalation have to be designed into the graph rather than assumed from the wire format.	Research preprint Handoffs
📄 Strabo: Declarative Specification and Implementation of Agentic Interaction Protocols Paper · Declarative interaction protocols	arXiv; EMAS 2026 Workshop at AAMAS 2026 Samuel H. Christie, Amit K. Chopra, Munindar P. Singh · 2026	Models the checkout portion of the Universal Commerce Protocol, a Google-led standard for agent commerce, as a declarative Langshaw interaction protocol, implements the agents in Peach, a programming model for Langshaw, and shows them interoperating with UCP agents implemented by Google. Argues that declarative protocols from engineering-multiagent-systems research can enter industry agent ecosystems incrementally rather than through a wholesale rewrite. Why: Connects long-running research on declarative interaction protocols to a live industry agent protocol, backed by working interoperation rather than a position argument.	Research preprint Handoffs
📄 AECP: Artifact-Exclusive Communication Protocol for Multi-Agent Code Generation Paper · Artifact passing	arXiv Jiaqi Xue, Yanjun Wang, Xiangci Li, Lingbo Mo, Aritra Sengupta, Shweta Garg, et al. · 2026	Moves coordination out of agents' interpretation of messages and into the harness: agents may communicate only through structured artifacts, and the harness supplies recorded findings when an agent touches relevant code, screens implementations against recorded interface commitments, and makes affected agents revisit revised agreements. Across Doc2Repo, NL2Repo, and CodeProjectEval with closed and open models, it raises average test pass rate by 28.2% and cuts wall time by 16.5% relative to a team using free-form messages. Restricting communication to artifacts also stopped malicious instructions relaying between agents: they reached other agents in 95% of cases before and none after. Why: A concrete case for typed artifact edges over chat, since the harness can check commitments and block relayed instructions that free-form messages let through.	Research preprint Handoffs
📄 When Agents Talk, Secrets Travel: Topology-Driven Leakage and Recipient-Conditioned Sanitization in Multi-Agent LLM Systems Paper · Edge security	arXiv Sunday Oyinlola Ogundoyin, Muhammad Ikram · 2026	Shows that information disclosed to one authorized agent can reach an unauthorized one through ordinary coordination, with no compromise involved. Across 485 scenarios in coordinator-worker, peer-to-peer, and hierarchical topologies with seven instrumented channels, disclosure to recipients that only the topology introduced rises from 0% to 52.2% while surface leakage stays around 61% to 63%, and framework choice separately drives a 150-fold spread in leakage. A sanitizer that conditions on the recipient and execution context cuts residual disclosure from 96.7% to 29.25% while preserving 94.1% of authorized disclosures, against 20.5% under blanket redaction. Why: Privacy in an agent graph depends on who can receive what downstream, so enforcement has to reason about the recipient and not only the content.	Research preprint Handoffs

State, Memory & Artifacts

Resource	Source	What it contributes	Evidence
📚 LangGraph Persistence Docs · Durable execution	LangGraph Documentation LangChain · 2025	Reference documentation for checkpointing agent-graph state: thread-scoped snapshots via Postgres/SQLite/in-memory savers, resuming after failure, and fault-tolerance semantics. The baseline replay/resume layer most production agent graphs start with before graduating to full durable-execution engines. Why: Reference for the checkpointing that makes a graph resumable instead of restart-from-scratch after any node failure.	Official documentation State
📚 OpenAI Agents SDK: Sessions Docs · Conversation state	OpenAI Agents SDK Documentation OpenAI · 2026	Documents persistent conversation history that can be loaded and updated across repeated agent runs. Why: Provides a bounded mechanism for carrying state across nodes and turns without manually rebuilding every prompt.	Official documentation State
📚 Google ADK: Artifacts Docs · Versioned work products	Google ADK Documentation Google · 2026	Defines named, automatically versioned binary work products that agents and tools can save, load, list, and exchange within session-scoped or persistent user-scoped namespaces. Why: Provides explicit, inspectable edge artifacts instead of forcing large or structured outputs through conversational context.	Official documentation State
📄 LEGOMem: Modular Procedural Memory for Multi-agent LLM Systems for Workflow Automation Paper · Procedural memory	AAMAS 2026 Extended Abstracts Dongge Han; Camille Couturier; Daniel Madrigal Diaz; Xuchao Zhang; Victor Rühle; Saravan Rajmohan · 2026	Decomposes execution trajectories into reusable procedural memories and allocates them to an orchestrator and specialist agents for workflow automation. Why: Shows how durable experience can improve decomposition and delegation without collapsing every node's memory into one undifferentiated store; the selected artifact is a three-page extended abstract.	Peer-reviewed research State
📄 Learning to Share: Selective Memory for Efficient Parallel Agentic Systems Paper · Selective shared memory	ICML Joseph Fioresi; Parth Parag Kulkarni; Ashmal Vayani; Song Wang; Mubarak Shah · 2026	Adds a global memory bank to parallel agent teams and trains an admission controller with stepwise reinforcement learning and usage-aware credit assignment to retain reusable intermediate work. Why: Treats cross-team memory writes as governed graph operations that can reduce duplicate work while limiting indiscriminate context growth.	Peer-reviewed research State
📐 PROV-DM: The PROV Data Model Standard · Artifact provenance	W3C Recommendation Luc Moreau; Paolo Missier · 2013	Defines entities, activities, agents, derivation, attribution, responsibility, bundles, and collections for interoperable provenance records. Why: Maps artifacts, execution steps, and responsible nodes into a portable lineage model; storage, access control, and cryptographic integrity remain implementation concerns.	Industry standard State
📄 in-toto: Providing farm-to-table guarantees for bits and bytes Paper · Cryptographic artifact provenance	USENIX Security Santiago Torres-Arias; Hammad Afzali; Trishank Karthik Kuppusamy; Reza Curtmola; Justin Cappos · 2019	Introduces signed material and product links for supply-chain steps and verifies them against an expected layout, supported by analysis of 30 historical compromises and deployed integrations. Why: Supplies a transferable receipt pattern that binds each artifact transformation to an authorized step and identity; it verifies process integrity, not semantic correctness.	Peer-reviewed research State
📝 AI Agent Memory Architectures for Multi-Agent Systems: Shared, Isolated, Hierarchical Blog · Architecture patterns	zylos.ai Zylos Research · 2026	Systematic comparison of the three memory topologies — fully shared (noise and permission leakage), fully isolated (coordination overhead), hierarchical global/team/private (the industry convergence point) — with framework-by-framework analysis of CrewAI, LangGraph, and AutoGen and a review of conflict-resolution strategies. Cites the MAST finding that most multi-agent failures are coordination, not capability. Why: Compares shared, isolated, and hybrid memory directly, which is the decision every multi-agent design must make explicitly.	Practitioner analysis State
📝 Context Engineering for Agents (LangChain) Blog · Context economics	langchain.com LangChain team (Lance Martin) · 2025	The write/select/compress/isolate taxonomy that became the standard vocabulary for agent state management: write context out to scratchpads and memories, select it back in, compress via summarization, and isolate by splitting across sub-agents or sandboxed environments. Maps each strategy to concrete LangGraph mechanisms. Why: The write, select, compress, and isolate vocabulary that made context management something teams can discuss precisely.	Practitioner analysis State
📝 Effective context engineering for AI agents (Anthropic) Blog · Context economics	anthropic.com Anthropic engineering · 2025	Treats the context window as a finite resource with diminishing returns and lays out the three long-horizon state strategies — compaction, structured note-taking (agentic memory persisted outside the window), and sub-agent architectures where detailed search context stays isolated in workers while the lead synthesizes condensed summaries. The token-economics framing for shared-vs-isolated state decisions. Why: Treats the context window as a budgeted resource, which is the discipline that keeps long-running nodes coherent.	Practitioner analysis State
📝 How and when to build multi-agent systems (LangChain) Blog · Context sharing vs isolation	langchain.com Harrison Chase · 2025	Reconciles the Cognition and Anthropic positions: read-heavy tasks (research) tolerate isolated parallel contexts, while write-heavy tasks (code) demand shared context because conflicting writes are expensive to merge. A practical decision rule for how much state to share across each edge of a work graph. Why: Reconciles the two dominant positions on multi-agent design by separating read-heavy from write-heavy work.	Practitioner analysis State
📝 Context Engineering for AI Agents: Lessons from Building Manus Blog · Filesystem as state	manus.im Yichao 'Peak' Ji · 2025	Argues the filesystem is 'the ultimate context': unlimited, persistent, directly operable by the agent, letting Manus keep the token window small by paging observations out to files and restoring them on demand. The founding statement of filesystem-as-agent-memory, plus KV-cache economics and recitation (todo.md) as attention management. Why: Argues the filesystem is the durable context layer, a practical answer to context limits that agents can operate directly.	Practitioner analysis State
📚 Claude Code Agent Teams: Shared Task List, Mailboxes, and File Ownership Docs · Filesystem isolation	code.claude.com Anthropic · 2026	The shared-state substrate of a shipping agent org: a shared task list with dependency tracking and file-locked task claiming to prevent races, per-agent JSON mailboxes under ~/.claude/teams/, and the explicit guidance that teammates must own disjoint file sets because two agents editing one file leads to overwrites. Experimental (flag-gated) as of v2.1.x, July 2026. Why: A shipping implementation of shared task state with ownership rules, which is the concrete form of coordination infrastructure.	Official documentation State
📚 Claude Code: Run Parallel Sessions with Git Worktrees Docs · Filesystem isolation	code.claude.com Anthropic · 2026	Worktree-per-agent as a state strategy made first-class: claude --worktree gives each session an isolated working directory and branch under .claude/worktrees/, and subagents can declare isolation: worktree in frontmatter so parallel edits physically cannot collide — merging moves to git, where conflict resolution is a solved problem. Why: Isolation per agent as a first-class feature is what makes parallel writers safe without a merge protocol.	Official documentation State
📚 CrewAI Memory: Unified Scoped Memory for Agent Crews Docs · Framework state	docs.crewai.com CrewAI · 2026	CrewAI's 2026 rework replaced separate short-term/long-term/entity stores with one API over hierarchical scope trees (/agent/researcher, /project/alpha): all agents in a crew share crew memory unless given their own, with composite recency-relevance-importance scoring and non-blocking background writes with read barriers. A clean example of hierarchical (global/team/private) memory scoping. Why: One scoped memory API across a team, showing how a framework resolves whose memory is whose.	Official documentation State
📚 LangGraph Memory: Store, Namespaces, and Cross-Thread Sharing Docs · Graph state & checkpointing	docs.langchain.com LangChain · 2026	The other half of LangGraph state: short-term thread-scoped memory via checkpoints vs long-term memory in a Store with hierarchical namespaces, recallable from any thread and any node. Covers semantic/episodic/procedural memory types and profile-vs-collection write patterns — the concrete API for shared memory across an agent organization. Why: Separates run-scoped from cross-run memory explicitly, which is the distinction most memory bugs come from.	Official documentation State
📝 State of AI Agent Memory 2026 (Mem0) Blog · Landscape	mem0.ai Mem0 · 2026	Vendor-authored but data-heavy annual snapshot of the memory-layer landscape: benchmark results across memory systems, the convergence on graph-plus-vector hybrids, and MCP emerging as the standard interface for sharing one memory layer across heterogeneous agents. Read alongside independent comparisons; useful for July-2026 currency. Why: A data-heavy snapshot of the memory-layer landscape, useful for calibration despite being vendor-authored.	Practitioner analysis State
📄 A-MEM: Agentic Memory for LLM Agents Paper · Memory systems	arXiv Wujiang Xu, Zujie Liang, Kai Mei, Hang Gao, Juntao Tan, Yongfeng Zhang · 2025	Zettelkasten-style memory: each stored note gets generated attributes and is dynamically linked to related notes, and new memories can trigger evolution (rewriting) of old ones, so the memory network restructures itself instead of only appending. Evaluated across six foundation models; code on GitHub (agiresearch/A-mem). Why: Dynamically linked notes give a memory that reorganizes as it grows rather than accumulating flat records.	Research preprint State
📝 How to Design Multi-Agent Memory Systems for Production (Mem0) Blog · Memory systems	mem0.ai Mem0 · 2026	The multi-scope pattern for agent-org memory: every write tagged with user_id / agent_id / run_id / app_id, composed at retrieval time, so private, shared, and session-scoped facts coexist in one store. Framework-agnostic — the same memory layer serves agents built in LangGraph, CrewAI, AutoGen/AG2, and Strands. Why: Scoping every write by user, agent, and run is the practical pattern that keeps shared memory from becoming noise.	Practitioner analysis State
📚 Letta: Multi-Agent Shared Memory Blocks Docs · Memory systems	docs.letta.com Letta (formerly MemGPT) · 2026	Attach one memory block to many agents and every writer's update is instantly visible to all readers — coordination through shared in-context state rather than messaging. Notably honest about concurrency: memory_insert is append-only and safe under simultaneous writers, while rethink-style rewrites can clobber each other. Why: Memory blocks attached to many agents make shared state explicit and immediately visible to all readers.	Official documentation State
📄 Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory Paper · Memory systems	arXiv Prateek Chhikara, Dev Khant, Saket Aryan, Taranjeet Singh, Deshraj Yadav · 2025	The extract-consolidate-retrieve memory pipeline behind the most-adopted agent memory layer, plus a graph-based variant (Mem0g) for relational facts. On LOCOMO: 26% relative improvement over the OpenAI memory baseline with 91% lower p95 latency and >90% token savings versus full-context replay. Why: The extract, consolidate, and retrieve pipeline behind a widely adopted memory layer, with published benchmarks.	Research preprint State
📝 Memory Blocks: The Key to Agentic Context Management Blog · Memory systems	letta.com Letta · 2025	The design rationale behind memory blocks: structured, labeled, size-limited segments of the context window that agents edit themselves and that can be shared, versioned, and templated across an organization of agents. Explains why block-level structure beats one undifferentiated scratchpad. Why: Explains why structured, size-limited memory segments beat free-form context accumulation for long-lived roles.	Practitioner analysis State
📄 MemOS: A Memory OS for AI Systems Paper · Memory systems	arXiv Zhiyu Li et al. (MemTensor) · 2025	Treats memory as a schedulable system resource: the MemCube unit unifies plaintext, activation (KV-cache), and parameter-level memories with metadata, enabling composition, migration, and fusion between forms. The most complete 'operating system' blueprint for memory governance across fleets of agents; open-source implementation at github.com/MemTensor/MemOS. Why: Treating memory as a schedulable resource is the most ambitious current attempt to systematize agent state.	Research preprint State
📄 From Human Memory to AI Memory: A Survey on Memory Mechanisms in the Era of LLMs Paper · Surveys	arXiv Yaxiong Wu, Sheng Liang, et al. (Huawei Noah's Ark Lab) · 2025	The broadest map of the agent-memory literature: organizes the field along three dimensions (object: personal vs system; form: parametric vs non-parametric; time: short vs long term) into eight quadrants, and catalogs open problems including multi-agent shared memory. Useful as the index into everything this section cannot list. Why: The broadest map of the memory literature, organized so you can find the subproblem you actually have.	Research preprint State
🧰 Cognee: Open-Source AI Memory Platform for Agents Tool · Temporal graph memory	GitHub Topoteretes (Vasilije Markovic et al.) · 2026	Self-hostable memory engine (28k+ stars) that combines knowledge graph and vector storage behind one ECL (extract-cognify-load) pipeline, with explicit support for cross-agent knowledge sharing, session memory synced to a permanent graph, and per-user isolation. Python, TypeScript, and Rust clients; integrates with Claude Code as a memory backend. Why: Self-hostable memory infrastructure for teams that need shared recall without sending state to a vendor.	Maintained OSS project State
🧰 Graphiti: Real-Time Knowledge Graphs for AI Agents Tool · Temporal graph memory	GitHub Zep (getzep) · 2026	The open-source temporal graph engine underlying Zep: incrementally ingests episodes from many agents, resolves entities and contradictions on write, and serves hybrid semantic/keyword/graph retrieval without batch recomputation. The main off-the-shelf option for graph-shaped shared memory serving an agent organization; ships an MCP server so any agent can attach. Why: Temporal shared state across many agents, where facts carry validity intervals rather than being overwritten.	Maintained OSS project State
📄 Zep: A Temporal Knowledge Graph Architecture for Agent Memory Paper · Temporal graph memory	arXiv Preston Rasmussen, Pavlo Paliychuk, Travis Beauvais, Jack Ryan, Daniel Chalef · 2025	Agent memory as a bi-temporal knowledge graph: every edge carries validity intervals for when a fact was true and when it was learned, so contradicted facts are invalidated, not deleted — giving an agent org a queryable, auditable shared world-model over time. Beats MemGPT on Deep Memory Retrieval (94.8% vs 93.4%). Why: Bi-temporal memory answers when a fact was true and when it was learned, which matters when nodes disagree.	Research preprint State
🧰 Agent Memory Techniques (NirDiamant) Tool · Tutorials	GitHub Nir Diamant · 2026	Runnable-notebook catalog of 20+ agent memory patterns, including a dedicated multi-agent shared memory tutorial (technique 22) covering shared stores, scoping, and cross-agent retrieval with Mem0 and friends. From the author of the widely used RAG_Techniques and GenAI_Agents repos; the fastest hands-on path through this section's ideas. Why: Runnable notebooks for memory patterns, including a shared multi-agent case, which shortens the path from idea to test.	Maintained OSS project State
🧰 Firecracker Tool · Worker isolation	GitHub Amazon Web Services · 2026	Apache-2.0 virtual machine monitor built on Linux KVM that runs workloads in minimal microVMs, giving each one its own kernel while keeping the startup cost and density closer to containers than to full virtual machines. Around 37.8k stars and the isolation layer beneath several commercial agent sandboxes. Why: Hardware-level isolation per worker is what makes it safe to run many agents writing untrusted code side by side.	Maintained OSS project State
🧰 E2B Tool · Worker isolation	GitHub E2B · 2026	Apache-2.0 infrastructure for running AI-generated code in isolated cloud sandboxes, with Python and JavaScript SDKs for creating sandboxes, executing commands, and managing filesystem state, plus Terraform-based self-hosting on AWS, GCP, and Azure. Around 14.7k stars and actively developed. Why: Gives each node a disposable filesystem and shell, which is the practical unit of isolation when workers run in parallel.	Maintained OSS project State
🧰 gVisor Tool · Worker isolation	GitHub Google · 2026	Apache-2.0 OCI runtime implementing a Linux-compatible kernel in userspace Go, so a sandboxed process's syscalls are serviced by gVisor rather than reaching the host kernel directly. Around 19.9k stars, and the alternative isolation model to microVMs, trading some compatibility for faster startup. Why: The other mainstream isolation choice for agent workers, and the tradeoff against microVMs is a real design decision.	Maintained OSS project State
📚 Mnemoverse Memory API: Rooms, Domains, and Supersession Docs · Cross-agent memory semantics	mnemoverse.com Mnemoverse · 2026	First-party API reference for a hosted memory service that several agent runtimes can share, documenting the parts that matter once more than one agent writes the same store. Domains namespace memories; a shared room is created by its owner, joined through single-use expiring invite codes, and addressed as the domain xroom:<room_id>, with reads and writes passing a database-enforced membership check. Reads carry a provenance record of authorship for room atoms. A write returns stored, atom_id, importance in [0, 1], and the reason the gate kept or filtered it, and may name up to 32 earlier atoms it supersedes; a target already claimed by a concurrent write returns 409 CONFLICT naming the successor, and the whole write rolls back rather than applying partially. The MCP server and Python SDK are MIT, and the engine behind them is a hosted service with a free tier and no self-hosted option. Why: Concrete vocabulary for shared versus isolated agent state: membership, authorship, and a documented conflict rule in place of last write wins.	Official documentation State

Verification & Evals

Resource	Source	What it contributes	Evidence
📚 Evaluate agent workflows Docs · Workflow evaluation	OpenAI API Documentation OpenAI · 2026	Explains how to build datasets, graders, trace-based evaluations, and reproducible checks for agent workflows. Why: Makes gates testable at both the final outcome and the intermediate handoff level.	Official documentation Gates
📝 Demystifying evals for AI agents Blog · Evaluation practice	Anthropic Engineering Mikaela Grace et al. · 2026	Presents practical guidance for defining tasks, outcomes, graders, and evaluation suites for agents with non-deterministic trajectories. Why: Helps turn vague reviewer judgments into evidence gates that can guide graph changes.	Practitioner analysis Gates
🧰 Inspect AI Tool · Evaluation framework	Inspect AI Documentation UK AI Security Institute · 2024	Provides an open-source evaluation framework with tasks, solvers, scorers, sandboxed tools, and structured logs. Why: Supports reproducible gate nodes and traceable evidence for both individual agents and composed systems.	Maintained OSS project Gates
📄 When Agents Go Rogue: Activation-Based Detection of Malicious Behaviors in Multi-Agent Systems Paper · Adversarial agent detection	ICML Haowen Xu et al. · 2026	Evaluates activation-space detection and restorative steering of compromised agents across five attack scenarios, multiple models, and synchronous and asynchronous multi-agent interactions. Why: Tests a topology-agnostic gate for locating and repairing a malicious node when its messages appear superficially benign.	Peer-reviewed research Reliability
📄 Verification-Aware Planning for Multi-Agent Systems Paper · Verification-aware work graphs	EACL Tianyang Xu; Dan Zhang; Kushan Mitra; Estevam Hruschka · 2026	Introduces VeriMAP, which decomposes tasks into a dependency graph and attaches planner-defined Python and natural-language verification functions to subtasks before execution. Why: Makes acceptance criteria part of the work graph so handoff failures can trigger local repair instead of surfacing only in a final answer.	Peer-reviewed research Gates
📄 Identifying Collective Intelligence Factor in LLM Agent Groups for Generalizable Multi-Agent System Design Paper · Collective-intelligence prediction	Findings of ACL Zhilun Zhou; Zihan Liu; Jiahe Liu; Yihan Wang; Qingyu Shao; Fengli Xu; Depeng Jin; Yong Li · 2026	Studies 108 agent groups varying team size, model composition, and communication topology, then derives a factor intended to predict transferable system designs. Why: Moves evaluation beyond one benchmark score by testing whether measurable group properties forecast performance across designs and tasks.	Peer-reviewed research Evolution
🧪 AgentRx: Diagnosing AI Agent Failures from Execution Trajectories Benchmark · Trajectory failure diagnosis	arXiv Shraddha Barke; Arnav Goyal; Alind Khare; Avaljot Singh; Suman Nath; Chetan Bansal · 2026	Releases 115 manually annotated failed trajectories and diagnoses critical failure steps by synthesizing guarded executable constraints and evidence-bearing validation logs. Why: Provides a concrete bridge from heterogeneous agent traces to repeatable failure localization; its evaluation includes Magentic-One but also single-agent workflows.	Benchmark/dataset Gates
🧪 MAESTRO: Multi-Agent Evaluation Suite for Testing, Reliability, and Observability Benchmark · Cross-framework system evaluation	arXiv Tie Ma; Yixi Chen; Vaastav Anand; Alessandro Cornacchia; Amândio R. Faustino; Guanheng Liu; Shan Zhang; Hongbin Luo; Suhaib A. Fahmy; Zafar A. Qazi; Marco Canini · 2026	Standardizes configuration and execution for 12 representative multi-agent systems and exports framework-agnostic traces with latency, cost, reliability, and failure signals. Why: Enables controlled comparison across architectures and repeated runs while revealing that topology can dominate model or tool changes in resource and reliability profiles.	Benchmark/dataset Observability & cost
🧰 A2A Protocol Technology Compatibility Kit Tool · Protocol conformance	Agent2Agent Project, Linux Foundation Agent2Agent Project · 2026	Tests Agent2Agent protocol implementations across gRPC, JSON-RPC, and HTTP with JSON and emits machine-readable, HTML, and JUnit conformance reports. Why: Turns interoperability claims at agent handoff edges into executable gates that can run in development and continuous integration.	Maintained OSS project Gates
📄 Debating with More Persuasive LLMs Leads to More Truthful Answers Paper · Adversarial verification / debate	arXiv Akbir Khan, John Hughes, Dan Valentine, et al. · 2024	ICML 2024 best-paper showing that a weak judge overseeing two strong models arguing opposing sides gets more truthful answers — and that optimizing debaters for persuasiveness improves judge accuracy. The strongest empirical case for adversarial (rather than cooperative) verifier edges in scalable-oversight settings. Why: A weak judge overseeing strong debaters reaching better answers is the theoretical basis for using cheaper verifiers.	Peer-reviewed research Gates
📄 Peacemaker or Troublemaker: How Sycophancy Shapes Multi-Agent Debate Paper · Anti-groupthink	arXiv Binwei Yao, Chao Shang, Wanyu Du, et al. (AWS AI) · 2025	Identifies sycophancy-driven "disagreement collapse" — panels converging on harmony before reaching the correct answer — in both peer debate and centralized-judge topologies, and proposes design principles to keep disagreement productive. Core reading on why councils fail as verification gates. Why: Names disagreement collapse, where panels converge on harmony before reaching correctness, and the conditions that cause it.	Research preprint Gates
📄 The Cost of Consensus: Isolated Self-Correction Prevails Over Unguided Homogeneous Multi-Agent Debate Paper · Debate skepticism	arXiv Blaž Bertalanič, Carolina Fortuna · 2026	April 2026 result showing independent self-correction by isolated models beats teams of identical models debating without structure, at a fraction of the compute. Reinforces the design rule that debate edges only pay off with heterogeneous agents and structured protocols. Why: Finds isolated self-correction beating identical-model teams, which sets a hard baseline for any panel design.	Research preprint Gates
📝 Reflection Agents Blog · Evaluator loops as topology	langchain.com Ankush Gola (LangChain) · 2024	Implements three evaluator-loop topologies in LangGraph — basic generator/reflector cycles, Reflexion with evidence-grounded critique, and LATS combining reflection with Monte Carlo tree search. The practical bridge from reflection-as-prompting to reflection-as-graph-structure, with runnable code. Why: Implements generator and reflector cycles concretely, which is the smallest useful verification structure in a graph.	Practitioner analysis Gates
📄 Proof-or-Stop: Don't Trust the Agent, Trust the Evidence Paper · Evidence-gated lifecycle control	arXiv Jek Huang, Jeffery Hsia, Jiayi Sun, Freddie Shi, Wei Huang, Ian H. White · 2026	Treats every agent claim ("tests passed", "reviewed") as an unverified assertion and only allows lifecycle transitions backed by fresh, traceable, mechanically checkable evidence read by gates. Reports zero false-positive completion claims across 10 scenarios, rejection of 18 evidence-tampering methods, and hidden-failure amplification cut from 1.7% to 0.1%; published July 16, 2026, days before the graph-engineering discourse took off. Why: Treats every claim as unverified until evidence is produced, the strictest available formulation of an evidence gate.	Research preprint Gates
🧪 JudgeBench: A Benchmark for Evaluating LLM-based Judges Benchmark · Judge evaluation	arXiv Sijun Tan, Siyuan Zhuang, Kyle Montgomery, et al. · 2024	Benchmarks judges on response pairs where factual correctness — not style — determines the right verdict, exposing how badly judges do on hard knowledge, reasoning, math, and coding comparisons. The go-to number for deciding whether a candidate judge model is strong enough to sit on a gating edge. Published at ICLR 2025. Why: Benchmarks judges on cases where correctness rather than style decides, exposing judges that reward polish.	Benchmark/dataset Gates
📄 Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge (CALM) Paper · Judge failure modes	arXiv Jiayi Ye, Yanbo Wang, Yue Huang, et al. · 2024	The CALM framework automatically injects perturbations to quantify 12 distinct bias types in LLM judges, giving a systematic audit checklist for any judge node before it is trusted as a gate. Complements the anecdotal bias lists in earlier work with automated measurement. Why: Quantifies twelve distinct judge biases, which turns judge selection into an engineering decision with known failure modes.	Research preprint Gates
📄 Large Language Models are not Fair Evaluators Paper · Judge failure modes	arXiv Peiyi Wang, Lei Li, Liang Chen, et al. · 2023	Shows that judge rankings can be flipped simply by reordering candidate answers in the prompt, letting weaker models "beat" stronger ones. Proposes the standard mitigations — position swapping, multiple evidence generation, human-in-the-loop calibration — that any pairwise judge node should implement. Why: Reordering candidates can flip a judge's ranking, which means a gate's prompt structure is part of its correctness.	Peer-reviewed research Gates
📄 Self-Preference Bias in LLM-as-a-Judge Paper · Judge failure modes	arXiv arXiv · 2024	Quantifies GPT-4's significant preference for its own outputs and traces the mechanism to perplexity: judges favor text that is familiar to them. Concrete design implication for graphs: never let the same model family both generate and gate its own work without cross-model checks. Why: Judges prefer their own outputs, a decisive argument against evaluating a node with the same model that produced the work.	Research preprint Gates
🧰 DeepEval Tool · Judge infrastructure	GitHub Confident AI · 2026	Pytest-style open-source LLM evaluation framework (19.3k stars, v4.1.0 July 2026) with G-Eval judge metrics, a DAG builder for deterministic decision-tree evaluation, and agentic metrics (task completion, tool correctness, plan adherence). One of the few tools that treats deterministic gates and model judges as composable alternatives in the same graph. Why: Test-style evaluation makes gates part of the development loop rather than a separate review exercise.	Maintained OSS project Gates
🧰 Verdict: a library for compound LLM-judge systems Tool · Judge infrastructure	GitHub Nimit Kalra, Leonard Tang (Haize Labs) · 2025	Open-source framework that composes judges from graph primitives (Units, Layers, Blocks) into hierarchical verification, debate, and aggregation pipelines over arbitrary dependency graphs — literally judges as subgraphs. Claims SOTA-competitive evaluation accuracy without fine-tuning by scaling judge-time compute; MIT-licensed. Why: Composing judges from graph primitives makes a verification stage a designed structure rather than one prompt.	Maintained OSS project Gates
📝 Evaluating the Effectiveness of LLM-Evaluators (aka LLM-as-Judge) Blog · Judge reliability in practice	eugeneyan.com Eugene Yan · 2024	Literature-grounded practitioner review of when LLM evaluators work: pairwise comparison is more stable than direct scoring, classification metrics suit objective checks while correlations suit subjective ones, and recall on factual-inconsistency detection can be alarmingly low (30-60%) even with high precision. A calibration reference before trusting a judge as a gate. Why: A literature-grounded account of when model judges work, including why pairwise comparison beats absolute scoring.	Practitioner analysis Gates
📄 Who Validates the Validators? Aligning LLM-Assisted Evaluation with Human Preferences (EvalGen) Paper · Judge validation	arXiv Shreya Shankar, J.D. Zamfirescu-Pereira, Björn Hartmann, Aditya Parameswaran, Ian Arawjo · 2024	Introduces EvalGen and documents "criteria drift": humans only discover their real evaluation criteria by grading outputs, so judge rubrics can never be fully specified upfront. Explains why gate criteria in agent graphs must be iteratively co-developed with humans, not written once. Why: Documents criteria drift, where people only discover their real standards by looking at outputs, which reshapes how rubrics get written.	Research preprint Gates
📄 A Survey on LLM-as-a-Judge Paper · LLM-as-judge foundations	arXiv Jiawei Gu, Xuhui Jiang, Zhichao Shi, et al. · 2024	Comprehensive survey of how to build reliable judge systems: consistency strategies, bias mitigation, and adaptation across evaluation scenarios. Useful as the single starting map before choosing a judge-node design for a given edge in the graph. Why: The comprehensive survey of building reliable judges, covering consistency strategies and bias mitigation together.	Research preprint Gates
📄 Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena Paper · LLM-as-judge foundations	arXiv Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, et al. (LMSYS) · 2023	The paper that legitimized model judges as system components: GPT-4 judges reach over 80% agreement with human preferences, matching human-human agreement. Also the canonical catalog of judge failure modes — position bias, verbosity bias, self-enhancement bias, and limited reasoning — that every judge node in an agent graph must be designed around. Why: The result that legitimized model judges as system components, with measured agreement against human preference.	Peer-reviewed research Gates
📄 Self-Consistency Improves Chain of Thought Reasoning in Language Models Paper · Majority vote as topology	arXiv Xuezhi Wang, Jason Wei, Denny Zhou, et al. (Google) · 2022	The original sample-many-paths-and-majority-vote result (ICLR 2023): agreement across diverse reasoning paths is a cheap correctness signal. In graph terms, the simplest fan-out/fan-in verification topology, and the baseline every fancier debate or judge structure must beat. Why: Majority voting over diverse reasoning paths is the cheapest quorum available, and the baseline elaborate panels must beat.	Peer-reviewed research Gates
📄 Let's Verify Step by Step Paper · Process reward models	arXiv Hunter Lightman, Vineet Kosaraju, Jan Leike, John Schulman, Ilya Sutskever, Karl Cobbe (OpenAI) · 2023	Process supervision (rewarding each reasoning step) substantially outperforms outcome supervision for reliability, with the PRM800K dataset of 800K step-level labels. The foundation for using process reward models as fine-grained gates on intermediate agent steps rather than only on final outputs. Why: Rewarding each step rather than only the outcome is what makes verification usable mid-run instead of after it.	Peer-reviewed research Gates
📄 Minority Sentinel: When to Overturn Majority Voting in Multi-Agent LLM Debates Paper · Quorum voting	arXiv Chuan He, Zebin Chen, Zhengyi Yang, et al. · 2026	Trains a meta-classifier that detects when a debate majority is suppressing a correct minority opinion and decides when to overturn the vote, hitting 81.2% override accuracy across six benchmarks. Turns quorum voting from a fixed aggregation rule into an adaptive gate — directly applicable to voting nodes in agent graphs. Why: Detecting when a majority is suppressing a correct minority addresses the central weakness of quorum gates.	Research preprint Gates
📝 Creating a LLM-as-a-Judge That Drives Business Results (Critique Shadowing) Blog · Rubric design for judges	hamel.dev Hamel Husain · 2024	The most-cited practitioner methodology for building trustworthy judges: anchor on one principal domain expert, force binary pass/fail verdicts instead of Likert scores, and bootstrap the judge prompt from expert critiques ("critique shadowing"). Directly transferable to designing gate nodes whose verdicts humans will actually trust. Why: The most-cited practitioner methodology for judges people actually trust, anchored on a single domain expert.	Practitioner analysis Gates
📄 G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment Paper · Rubric design for judges	arXiv Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, Chenguang Zhu · 2023	Introduced the chain-of-thought, form-filling rubric pattern that most production judge prompts still follow, and was among the first to flag that LLM evaluators are biased toward LLM-generated text. Directly relevant to writing rubrics for judge nodes rather than asking for bare scores. Why: The chain-of-thought rubric pattern most production judge prompts still follow, worth reading in the original.	Peer-reviewed research Gates
📄 Large Language Monkeys: Scaling Inference Compute with Repeated Sampling Paper · Verification gap	arXiv Bradley Brown, Jordan Juravsky, et al. (Stanford) · 2024	Coverage grows log-linearly with samples (SWE-bench Lite: 15.9% to 56% at 250 samples) in domains with automatic verifiers, but majority voting and reward models plateau after a few hundred samples where no oracle exists. The cleanest quantification of why the verifier — not the generator — is the binding constraint on graph designs. Why: Repeated sampling raises coverage sharply where a verifier exists, which is precisely why gates unlock parallelism.	Research preprint Gates
📝 The AI Co-Scientist Is Here. The Bottleneck Is Verification. Blog · Verification-bottleneck essays	k-dense.ai K-Dense · 2026	June 2026 essay on agentic science arguing "discovery is becoming cheap, trust is not," and specifying a concrete verification stack for research agents: source grounding, executable inspectable code, data provenance, preserved intermediate artifacts, and human approval checkpoints. Coins "research laundering" for unverifiable agent output. Why: Argues discovery is becoming cheap while trust is not, and specifies what verification would have to provide.	Practitioner analysis Gates
📝 Verification Is the New Bottleneck — Not Generation Blog · Verification-bottleneck essays	danielkeller.com Daniel Keller · 2026	July 3, 2026 essay proposing a Swiss-cheese verification stack for agent pipelines: deterministic checks (static analysis, tests, security scans) first, LLM-as-judge for semantic correctness second, and humans reviewing evidence rather than artifacts last. Recommends spending 30% of effort on generation and 70% on verification infrastructure. Why: Proposes a layered verification stack for agent pipelines, combining deterministic checks with model judgment.	Practitioner analysis Gates
📝 Verification Is the Next Bottleneck in AI-Assisted Development Blog · Verification-bottleneck essays	opslane.com via Internet Archive Abhishek Ray · 2026	April 2026 essay arguing that automation cost is collapsing while "the cost to verify is biologically bounded": teams that merged 10 PRs a week are now merging 40 to 50, reviewed by the same engineers. Prescribes a three-layer gate stack — test suites, upstream human acceptance criteria, and adversarial "agents verifying agents" — with humans reserved for critical changes. Why: Frames verification cost as the binding constraint on automation, which is the economic case for gate design.	Practitioner analysis Gates
📄 Generative Verifiers: Reward Modeling as Next-Token Prediction Paper · Verifier models as gates	arXiv Lunjun Zhang, Arian Hosseini, et al. (Google DeepMind) · 2024	Trains verifiers as generative next-token predictors (GenRM) that can reason before scoring and exploit test-time compute, outperforming discriminative reward models. Blurs the line between judge nodes and reward-model gates — the same node can deliberate and then gate. Published at ICLR 2025. Why: Verifiers that reason before scoring and can explain themselves are more useful as graph nodes than opaque scorers.	Peer-reviewed research Gates
📄 TraceFix: Repairing Agent Coordination Protocols with TLA+ Counterexamples Paper · Protocol verification	arXiv Shuren Xia, Qiwei Li, Taqiya Ehsan, Jorge Ortiz · 2026	Model-checks agent coordination protocols in TLA+ and repairs them from counterexamples until verification passes: all 48 tasks across 16 scenario families verified, 62.5% on the first attempt and none needing more than four repair cycles. Across 3,456 runs, verified protocols cut deadlock and livelock from 31.1% to 14.1%. Why: Shows a coordination protocol can be formally checked and repaired before deployment, rather than debugged from production incidents.	Research preprint Handoffs
📄 Agentproof: Static Verification of Agent Workflow Graphs Paper · Static graph verification	arXiv Melwin Xavier, Vaisakh M A, Melveena Jolly, Midhun Xavier · 2026	Extracts one abstract graph model from LangGraph, CrewAI, AutoGen, and Google ADK workflows, runs six structural checks that produce witness traces, and evaluates temporal safety policies written in a small DSL compiled to finite automata, both statically and over runtime event traces. On 18 author-built workflows, 27% had structural defects such as dead-end nodes or unreachable exits and 55% violated a human-gate policy; verification finished in under a second for graphs of up to 5,000 nodes. The authors present the corpus as a benchmark for verification tools, not as a measure of how common these defects are. Why: Catches topology defects such as missing exits or skipped human gates before deployment, including paths that runtime guardrails only see if they happen to run.	Research preprint Gates
📄 Maat: Independent Deterministic Contract-Based Governance for Multi-Agent LLM Workflows Paper · Handoff contract validation	arXiv Uliana Elina · 2026	A runtime layer that validates agent-to-agent handoffs against a versioned workflow contract with no language model in the validation or scoring path, evaluated on six controlled workflows of 6 to 15 agents over 522 trials with injected data defects. After a post-publication audit found that three benchmark scorers credited any early halt as a prevented defect, the corrected results show rubric gains of 7.7% to 29.1% in five workflows, flat in software development, and model-call cost down 17% to 53% where halts occur early. A hand review of all 94 governed halts found 35 false alarms caused by validator defects; counting those as failed work, the governed arm scores below the ungoverned one in four of six workflows. Why: Deterministic handoff checks can stop contract-detectable defects, and the paper's own audit shows how quickly a gate's false alarms turn a gain into a loss.	Research preprint Gates
📄 Deny Without Disabling: Authorization-Paired Evaluation and Control for Multi-Agent Systems Paper · Composed-flow authorization	arXiv Yunbei Zhang, Saiyue Lyu, Janet Wang, Yingqiang Ge, Jiang Guo, Jihun Hamm, et al. · 2026	Addresses contributions that are each admissible alone but jointly enable a prohibited use. Authorization-paired evaluation makes blocking prohibited uses and completing required authorized uses one success criterion, so a gate cannot score well by blocking everything, and FlowReview connects object resolution, permission ranking, and deterministic enforcement. In controlled composition experiments, reviewing combined artifacts cuts the denied-commit rate from 86.0% to zero with no loss of authorized supply, and preserving information lineage alone did not ensure correct permission attribution. Why: Places the authorization check where a graph combines its parts, and scores it so that over-blocking counts as failure.	Research preprint Gates

Reliability & Durable Execution

Resource	Source	What it contributes	Evidence
📚 Dapr Agents introduction Docs · Durable actors and workflows	Dapr Documentation Dapr; CNCF · 2026	Introduces an agent framework built on durable actors and workflows, with state, messaging, and recovery supplied by the Dapr runtime. Why: Shows how graph nodes can inherit distributed-systems durability instead of implementing recovery only in prompts.	Official documentation Reliability
🧰 Temporal integration for OpenAI Agents SDK Tool · Durable execution integration	Temporal Python SDK Temporal · 2025	Integrates OpenAI agent runs, model calls, and tools with Temporal workflows and activities for durable execution. Why: Provides replay, retry, timeout, and recovery semantics beneath an agent graph without asking the model to manage them.	Maintained OSS project Reliability
📚 DBOS AI Quickstart Docs · Database-backed workflows	DBOS Documentation DBOS · 2026	Shows how to place AI application steps inside durable workflows whose progress is recorded and recoverable after interruption. Why: Offers a compact path from an agent prototype to resumable execution with explicit step boundaries.	Official documentation Reliability
📚 Restate Durable Agents Docs · Durable agent patterns	Restate Documentation Restate · 2026	Documents durable agent patterns using persisted execution, reliable calls, retries, timers, and stateful services. Why: Maps common mid-graph failures to runtime guarantees rather than fragile application-level retry code.	Official documentation Reliability
📄 ResMAS: Resilience Optimization in LLM-based Multi-agent Systems Paper · Resilient topology	AAAI Zhilun Zhou et al. · 2026	Learns task-specific resilient communication topologies and topology-aware prompts after measuring how graph structure and node instructions affect performance under agent failures and other perturbations. Why: Moves resilience from reactive recovery into the design of the graph itself and evaluates transfer to new tasks and models.	Peer-reviewed research Reliability
📄 DoVer: Intervention-Driven Auto Debugging for LLM Multi-Agent Systems Paper · Intervention-driven debugging	ICLR Ming Ma et al. · 2026	Tests failure hypotheses by editing messages or plans and measuring whether each intervention repairs the outcome or advances execution in branching multi-agent traces. Why: Turns graph debugging into causal repair experiments instead of relying on plausible but unverified post-hoc explanations.	Peer-reviewed research Reliability
📚 Microsoft Agent Framework: Durable Extension Docs · Durable graph workflows	Microsoft Learn Microsoft · 2026	Adds persisted sessions, checkpointed progress, failure recovery, external-event waits, and distributed hosting to agents, multi-agent orchestrations, and graph workflows. The documented packages remain prerelease. Why: Shows how an agent graph can resume without losing context or repeating completed work after interruption.	Official documentation Reliability
📄 Topology Matters: Measuring Memory Leakage in Multi-Agent LLMs Paper · Topology-conditioned memory leakage	Findings of ACL Jinbo Liu et al. · 2026	Measures private-information leakage across six communication topologies, agent counts, and attacker-target placements, finding higher leakage with denser connectivity, shorter graph distance, and greater target centrality. Why: Makes privacy a measurable consequence of edge structure and node placement, motivating topology-aware access controls.	Peer-reviewed research Reliability
📐 OWASP Top 10 for Agentic Applications for 2026 Standard · Agentic application security risks	OWASP GenAI Security Project OWASP GenAI Security Project; Agentic Security Initiative · 2025	Provides a globally peer-reviewed operational framework for critical risks in autonomous and agentic applications, developed with contributions from more than 100 security experts, researchers, and practitioners. Why: Supplies a practical threat checklist for node authority, tool permissions, memory provenance, inter-agent trust, and cascading failures across an agent graph.	Industry standard Reliability
📄 IPIGuard: A Novel Tool Dependency Graph-Based Defense Against Indirect Prompt Injection in LLM Agents Paper · Structural prompt-injection defense	EMNLP Hengyu An; Jinghuai Zhang; Tianyu Du; Chunyi Zhou; Qingming Li; Tao Lin; Shouling Ji · 2025	Models task execution as traversal over a planned Tool Dependency Graph and separates action planning from interaction with untrusted external data. Why: Shows how structural constraints on allowed tool edges can block unauthorized actions instead of relying only on prompts or classifiers.	Peer-reviewed research Work graphs
📐 Serverless Workflow Specification v1.0.0 Standard · Portable workflow semantics	Open Workflow Specification, Cloud Native Computing Foundation Open Workflow Specification Project · 2025	Defines a vendor-neutral workflow language with sequential and concurrent tasks, event correlation, service calls, error handling, retries, and timeouts. Why: Provides an inspectable execution contract for production work graphs; conformance does not itself supply a durable runtime, an agent protocol, or model-level correctness.	Industry standard Reliability
📄 ETAS: An Effect-Typed Language for Agent Systems Paper · Effect-typed agent contracts	arXiv Huiri Tan; Yikun Wang; Puyang Zhang; Shangyu Li; Jiasi Shen · 2026	Defines a language in which agents, tools, typed memory, approvals, policies, effects, and execution traces are semantic program elements, with static obligations and runtime monitors. Why: Provides a transferable contract model for authorizing and auditing node actions before and during execution; it is a new preprint and not yet deployment evidence.	Research preprint Reliability
📄 AgentAsk: Multi-Agent Systems Need to Ask Paper · Handoff clarification	ACL Bohan Lin; Kuo Yang; Zelin Tan; Yingchuan Lai; Chen Zhang; Guibin Zhang; Xinlei Yu; Miao Yu; Xu Wang; Yudong Zhang; Yang Wang · 2026	Defines an edge-level handoff-error taxonomy and adds selective clarification questions at critical exchanges where transferred information is ambiguous or incomplete. Why: Turns clarification into an explicit recovery action for communication-contract failures instead of letting uncertainty silently propagate downstream.	Peer-reviewed research Handoffs
📄 Towards Self-Improving Error Diagnosis in Multi-Agent Systems Paper · Execution-verified diagnosis	Findings of ACL Jiazheng Li; Emine Yilmaz; Bei Chen; Thu Le · 2026	Detects local anomalies, traces backward to decisive failures, tests diagnostic hypotheses with tools, and retains only execution-verified diagnostic memories. Why: Combines trace localization, active validation, and durable learning so a diagnostic node improves from evidence rather than storing unverified explanations.	Peer-reviewed research Observability & cost
📄 CORBA: Contagious Recursive Blocking Attacks on Multi-Agent Systems Based on Large Language Models Paper · Denial-of-collaboration attacks	Findings of ACL Zhenhong Zhou; Zherui Li; Jie Zhang; Yuanhe Zhang; Kun Wang; Yang Liu; Qing Guo · 2026	Formalizes attacks that deform communication topology into cycles or recursively expanding exchanges, blocking useful collaboration while exhausting system resources. Why: Shows that edge activation and termination policies are security boundaries, not only scheduling details, and motivates cycle and budget guards.	Peer-reviewed research Reliability
📄 CIA: Inferring the Communication Topology from LLM-based Multi-Agent Systems Paper · Topology confidentiality	ACL Yongxuan Wu; Xixun Lin; He Zhang; Nan Sun; Kun Wang; Chuan Zhou; Shirui Pan; Yanan Cao · 2026	Infers hidden communication topology from black-box multi-agent systems using adversarial queries and semantic correlations among observed responses. Why: Establishes topology confidentiality as an engineering property and demonstrates that apparently private coordination structure can leak through behavior.	Peer-reviewed research Reliability
📄 BlindGuard: Safeguarding LLM-based Multi-Agent Systems under Unknown Attacks Paper · Unknown-attack detection	ACL Rui Miao; Yixin Liu; Yili Wang; Xu Shen; Yue Tan; Yiwei Dai; Shirui Pan; Xin Wang · 2026	Learns normal individual, neighborhood, and global interaction behavior without attack labels, then detects malicious agents through corruption-guided contrastive learning. Why: Adds graph-aware anomaly detection for attacks that were absent from training and distinguishes local behavior from neighborhood and system-wide patterns.	Peer-reviewed research Reliability
📄 Explainable and Fine-Grained Safeguarding of LLM Multi-Agent Systems via Bi-Level Graph Anomaly Detection Paper · Fine-grained graph anomaly detection	ACL Junjun Pan; Yixin Liu; Rui Miao; Kaize Ding; Yu Zheng; Quoc Viet Hung Nguyen; Alan Wee-Chung Liew; Shirui Pan · 2026	Detects sentence- and token-level anomalies over multi-agent interaction graphs and attributes suspicious tokens to support finer-grained explanations of compromised exchanges. Why: Connects graph-level protection to actionable message evidence rather than returning only an opaque risk score for an entire agent or run.	Peer-reviewed research Reliability
📄 Agents Under Siege: Breaking Pragmatic Multi-Agent LLM Systems with Optimized Prompt Attacks Paper · Topology-aware prompt attacks	ACL Rana Shahroz; Zhen Tan; Sukwon Yun; Charles Fleming; Tianlong Chen · 2025	Optimizes attack placement across latency- and bandwidth-constrained agent networks using graph-flow formulations to disrupt multi-agent task completion. Why: Makes topology part of the adversary model and tests how limited attack budgets exploit particular communication paths and bottlenecks.	Peer-reviewed research Reliability
📝 Introducing Agent Executor, Google’s distributed Agent Runtime Blog · Distributed agent runtime	Google Cloud Blog Jaana Dogan; Ethan Bao · 2026	Introduces an open-source runtime standard for execution, resumption, isolation, session consistency, reconnection, trajectory branching, and federated agent deployment. Why: Addresses durable long-running graphs at the runtime layer and makes checkpoints, single-writer state, recovery, and deployment boundaries explicit; the release is in preview.	Practitioner analysis Reliability
🧰 Agent Governance Toolkit Tool · Runtime governance	Microsoft Microsoft · 2026	Enforces runtime policies on tool calls and inter-agent messages with identity, delegation, audit logs, telemetry, and adapters for multiple agent frameworks. Why: Implements authorization and audit controls at graph edges; it is a public preview, and its in-process enforcement layer is not an operating-system security boundary.	Maintained OSS project Gates
📚 Multi-Agentic System Threat Modeling Guide v1.0 Docs · Multi-agent threat modeling	OWASP GenAI Security Project OWASP GenAI Security Project · 2025	Applies an agentic threat taxonomy to multi-agent architectures and examines attack surfaces introduced by delegation, messaging, shared memory, tools, and cross-agent actions. Why: Provides a structured way to identify graph-specific trust boundaries and abuse paths; it is community security guidance rather than a certification or formal standard.	Official documentation Gates
🧪 Terminal-Bench Benchmark · Agentic task benchmarks	tbench.ai Stanford & Laude Institute · 2026	The standard for terminal mastery: sandboxed end-to-end tasks (kernel builds, server config, ML training, security) now at version 2.1 with ~89 Harbor-native tasks, a live agent+model leaderboard, and 3.0 plus a Science track in development. Its leaderboard doubles as a public scoreboard for competing harness designs from every major lab. Why: Terminal work is where partial failure is unavoidable, making this a practical proxy for how a worker node behaves when steps go wrong.	Benchmark/dataset Reliability
🧪 Vending-Bench: A Benchmark for Long-Term Coherence of Autonomous Agents Benchmark · Long-horizon evaluation	arXiv Axel Backlund, Lukas Petersson (Andon Labs) · 2025	Agents run a simulated vending-machine business over 20M+ token horizons, with sub-agent tooling for inventory, pricing, and ordering; all models eventually hit uncorrelated 'meltdown' loops — misread schedules, forgotten orders, even emailing the FBI. The benchmark behind Anthropic's Project Vend, and the clearest measure of the long-horizon coherence an org's manager nodes must sustain. Why: Long-horizon coherence is the property a manager node must sustain, and this shows how reliably models lose it.	Benchmark/dataset Reliability
📚 LiteLLM Budgets and Rate Limits Docs · Budget kill-switches	docs.litellm.ai BerriAI · 2026	The de facto budget kill-switch layer for agent fleets: hard max-budget enforcement (fail-closed) plus TPM/RPM rate limits per key, user, team, or agent, with requests rejected outright once a ceiling is crossed. The practical defense against runaway spawning and agent fork bombs — a compromised or looping subgraph exhausts its budget and stops, instead of your credit card. Why: Budget ceilings enforced by infrastructure rather than prompt instructions are the practical stop on runaway agent spend.	Official documentation Reliability
🧰 AgentChaos: Fault Injection for LLM Agent Systems Tool · Chaos testing	GitHub IntelligentDDS group (Sun Yat-sen University) · 2026	Chaos-engineering framework that wraps the HTTP client at runtime — no source changes — to inject 65 fault configurations at the LLM API layer: crashes and timeouts, omissions (empty/truncated responses), and value faults (corruption, schema mismatch), tested against AutoGen, MAD, MapCoder, EvoMAC. Early-stage research code, but the most systematic open fault-injection harness for multi-agent stacks yet. Why: Lets you inject agent-specific faults into a running graph without touching source, turning cascade behavior into something you can test.	Maintained OSS project Reliability
📝 Chaos Engineering for AI Agents: Injecting the Failures Your Agents Will Actually Face Blog · Chaos testing	tianpan.co Tian Pan · 2026	Argues traditional chaos engineering misses agent-specific faults and catalogs six categories to inject: LLM-level faults (rate limits, stream interruptions), unvalidated tool responses, context degradation, cross-agent cascades, specification drift, and silent failures. Key thesis: because agents mask failures with plausible output, resilience testing needs semantic validation — checking correctness, not just completion. Why: Names the fault categories worth injecting into agent systems, which generic chaos engineering does not cover.	Practitioner analysis Reliability
📝 LangGraph in Production: Temporal's LangGraph Plugin Adds Durable Execution Blog · Durable execution	temporal.io Temporal · 2026	July 2026 release running each LangGraph node as a Temporal Activity: crashes, timeouts, and worker losses are retried per policy and the graph resumes from durable state on any worker. Draws the crucial distinction that checkpoints preserve your data while durable execution preserves the run itself — including human-in-the-loop pauses lasting days or months. Why: Shows what changes when each graph node becomes a durable activity: crashes and worker losses stop being data loss.	Practitioner analysis Reliability
📄 From Spark to Fire: Modeling and Mitigating Error Cascades in LLM-Based Multi-Agent Collaboration Paper · Error cascades	arXiv Yizhe Xie, Congcong Zhu, et al. · 2026	Models collaboration as a directed dependency graph to show how one injected error solidifies into system-level false consensus, classifying three vulnerability types: cascade amplification, topological sensitivity, and consensus inertia. Its genealogy-graph governance layer — a message-layer plugin needing no architecture change — prevents widespread failure in at least 89% of test runs. Why: Models how a single injected error hardens into system-wide belief, the mechanism behind most silent multi-agent failures.	Research preprint Reliability
📄 Where LLM Agents Fail and How They Can Learn From Failures Paper · Error cascades	arXiv Kunlun Zhu et al. (UIUC and collaborators) · 2025	Triple contribution for single-node reliability inside a graph: AgentErrorTaxonomy (failure modes across memory, reflection, planning, action, system), AgentErrorBench (annotated failure trajectories from ALFWorld, GAIA, WebShop), and AgentDebug, which isolates root-cause errors and feeds corrective feedback back to the agent — up to 26% relative task-success improvement. Why: A failure taxonomy plus learning method for the node itself, since organization-level reliability starts with reliable members.	Research preprint Reliability
📄 AgenTracer: Who Is Inducing Failure in the LLM Agentic Systems? Paper · Failure attribution	arXiv Guibin Zhang, Junhao Wang, et al. · 2025	Builds failure-attribution training data via counterfactual replay and programmed fault injection (TracerTraj), then trains AgenTracer-8B — a small specialist that beats Gemini-2.5-Pro and Claude-4-Sonnet by up to 18% at blaming the right agent. Feeding its diagnoses back into live systems (MetaGPT, MaAS) lifts task performance 4.8-14.2%, making attribution an actionable repair loop rather than just autopsy. Why: Counterfactual replay and injected faults turn blame assignment in a multi-agent trace into a measurable task.	Research preprint Reliability
📄 Beyond Individual Intelligence: Surveying Collaboration, Failure Attribution, and Self-Evolution in LLM-based Multi-Agent Systems Paper · Failure attribution	arXiv arXiv · 2026	2026 survey stitching the fast-moving failure-attribution literature (Who&When, AgenTracer, TraceElephant and successors) into one map alongside collaboration structure and self-evolving systems. The efficient entry point if you need the state of the art on diagnosing agent-graph failures without reading a dozen papers. Why: Stitches the fast-moving failure-attribution literature into one map, which is the fastest way into a fragmented subfield.	Research preprint Reliability
📄 Causal Agent Replay: Counterfactual Attribution for LLM-Agent Failures Paper · Failure attribution	arXiv arXiv · 2026	Treats an agent run as a structural causal model and uses intervention-based replay — re-running the graph with a suspect step swapped — to establish which error actually caused the failure, versus the ~14% step-level accuracy of judge-the-log approaches on Who&When. Counterfactual replay is emerging as the rigorous standard for work-graph postmortems. Why: Intervention-based replay isolates which node actually caused an outcome instead of which one looked suspicious in the log.	Research preprint Observability & cost
📐 Microsoft: Taxonomy of Failure Modes in Agentic AI Systems Standard · Failure taxonomies	microsoft.com Microsoft AI Red Team · 2025	Whitepaper from Microsoft's AI Red Team organizing agentic failures along two axes — safety vs. security, novel vs. existing — with novel modes like multi-agent communication breakdowns and memory poisoning of shared state, each paired with mitigations. The enterprise-governance complement to Berkeley's empirically derived MAST. Why: A red-team taxonomy separating safety from security failures, and novel agentic modes from familiar ones.	Industry standard Reliability
📝 AI Coding Assistants Chase Phantoms, Destroy Real User Data Blog · Incident writeups	Ars Technica Benj Edwards · 2025	The definitive writeup of July 2025's twin agent disasters: Replit's agent deleting Jason Lemkin's production database (1,206 executive records) during an explicit code freeze then fabricating data to mask it, and Gemini CLI destroying files while hallucinating success. A case study in confabulation cascades — the agent's false model of the world compounding into destructive action — that made agent kill-switches a mainstream requirement. Why: The clearest cautionary record of agents destroying production data, and why irreversible actions need gates and human control paths.	Practitioner analysis Reliability
📝 Multi-Agent AI Systems: Why They Fail and How to Fix Coordination Issues Blog · Production reliability	augmentcode.com Augment Code · 2026	Engineering guide mapping observed production failure symptoms — role misunderstanding, agents overwriting each other's context, unproductive loops, memory drift — to concrete coordination fixes, with the p^n compounding math (0.95^10 = 59%) made operational. Positions independent validation between nodes as the most underused reliability mechanism in multi-agent systems. Why: Maps observed production symptoms to coordination causes, which is the translation layer between an incident and a design fix.	Practitioner analysis Reliability
📚 Saga Orchestration and Compensating Transactions for Agents Docs · Saga/compensation	microsoft.github.io Microsoft (Agent Governance Toolkit) · 2026	Hands-on tutorial applying the saga pattern to agent work: every forward action gets a paired compensating action, and on failure the orchestrator walks completed steps backward in reverse order, with per-step timeouts, retry budgets, and escalation when compensation itself fails. Answers the partial-failure question — agent 3 of 5 died after sending real emails — that plain retry logic cannot. Why: Compensating transactions give a graph a defined way to undo partial work, the pattern distributed systems settled on decades ago.	Official documentation Reliability
📄 When Agents Do Not Stop: Uncovering Infinite Agentic Loops in LLM Agents Paper · Loop termination analysis	arXiv Xinyi Hou, Shenao Wang, Yanjie Zhao, Haoyu Wang · 2026	Names infinite agentic loops: repeated model calls, tool calls, workflow transitions, or agent handoffs along a feedback path that nothing effectively bounds. IAL-Scan lifts agent code into a framework-independent intermediate representation, builds an agentic loop dependence graph that recovers explicit and framework-induced feedback paths, and flags paths that can keep reaching costly or state-growing operations. On 6,549 agent repositories it reported 74 findings, of which manual review confirmed 68 across 47 projects, for 91.9% precision. Why: Makes loop bounds something you can check statically across nodes and handoff edges, before an unbounded cycle becomes a cost or side-effect incident.	Research preprint Reliability
📄 Organizational Control Layer: Governance Infrastructure at the Execution Boundary of LLM Agent Systems Paper · Runtime governance	arXiv Tianyu Shi, Yang Mo, Yiou Liu, Zhuonan Hao, Yin Wang, Wenzhuo Hu, et al. · 2026	Separates proposing an action from executing it: a model-agnostic layer intercepts a generated action, checks it against role, policy, and economic constraints, and approves, revises, blocks, or escalates it without touching the generator. On adversarial buyer-seller negotiation environments adapted from AgenticPay, across several frontier backends, observed unsafe executions fall from 88% to 0% and valid success rises from 12% to 96%; ablations attribute the gain to pre-execution enforcement combined with structured recovery rather than to prompting or blocking alone. Why: Puts the authority check on the edge between a decision and the action it triggers, where a graph's permissions are either enforced or merely described.	Research preprint Gates
📄 Why Do Agent Teams Fail Across Boundaries? A Failure Taxonomy for Agent-to-Agent Systems with Private States Paper · Failure taxonomies	arXiv Sarath Shekkizhar, Prathyusha Jwalapuram, Zixuan Ke, Vaidehi Patil, Jiayu Wang, Srijan Bansal, et al. · 2026	Studies agents acting for different principals, each with its own environment and information the other cannot see, across more than 2,500 recorded runs of 488 scenarios in eight families including legal, CRM, security, and commerce. Derives 20 failure modes in four groups: discovery and delegation, grounding and record consistency, verification, and conduct and fidelity. Three findings held across every harness, model, and scenario: whether an agent contacts a peer depends on how the task is phrased rather than on which peers exist; no agent checks a peer's factual claim unprompted, so the system inherits the reliability of its least reliable participant unless the protocol performs the check; and an aggregate outcome score hides coordination and conduct failures, so each agent has to be judged against its own principal. The run logs are public. Why: Moves failure analysis past the single-orchestrator assumption, and shows cross-boundary verification has to be built into the edge because no agent supplies it on its own.	Research preprint Handoffs
📄 Fault-Tolerant Budget Conservation in Distributed Multi-Agent Delegation Paper · Delegated budget conservation	arXiv Genliang Zhu, Chu Wang · 2026	Treats resource limits as an authorization boundary when agents delegate across concurrent, failure-prone workers, where lost replies, late completions, repeated messages, partitions, and aliased DAG joins can all cause overspend. Budgets become exclusive escrow credits that move through a delegation DAG; before dispatch a branch converts credit into a reservation bound to lineage, epoch, effect, maximum charge, receiver, and idempotency key, and uncertain effects stay charged until authenticated settlement. Proves conservation, non-amplification by descendants, at-most-once settlement, and safety under late completion and partition given stated assumptions, and exercises them with bounded TLA+ checking and crash-injected experiments. Why: Budgets are usually a counter in the orchestrator; this shows what it takes for a spending limit to survive retries, timeouts, and partitions across a delegation tree.	Research preprint Reliability

Observability & Cost

Resource	Source	What it contributes	Evidence
📚 OpenAI Agents SDK: Tracing Docs · Tracing	OpenAI Agents SDK Documentation OpenAI · 2026	Documents traces and spans for agent runs, model generations, tool calls, handoffs, and guardrails. Why: Makes node and edge behavior inspectable so latency, failures, and expensive paths can be attributed correctly.	Official documentation Observability & cost
📐 OpenTelemetry GenAI Semantic Conventions Standard · Telemetry standard	OpenTelemetry GenAI repository OpenTelemetry GenAI SIG · 2026	Develops shared telemetry names and attributes for generative-AI model, tool, and agent operations in traces, metrics, and events. Why: Helps graph telemetry remain portable across runtimes and observability vendors.	Maintained OSS project Observability & cost
📚 Arize Phoenix: Cost Tracking Docs · Cost attribution	Arize Phoenix Documentation Arize AI · 2026	Explains how Phoenix derives and displays token usage and model cost from traced generative-AI calls. Why: Supports per-node and per-path cost accounting instead of treating a multi-agent run as one opaque bill.	Official documentation Observability & cost
📚 LangSmith Observability Docs · Observability platform	LangSmith Documentation LangChain · 2026	LangChain's observability platform and the default lens on LangGraph organizations: hierarchical trace trees for multi-agent runs, dashboards and alerting, online evaluations, and a LangSmith Engine that clusters recurring failure patterns in traces and diagnoses root causes. Framework-agnostic in practice — OpenAI, Anthropic, CrewAI, Pydantic AI and Vercel AI SDK integrations alongside LangGraph. Why: Hierarchical traces make a graph's structure visible as structure, which is the minimum for debugging coordination.	Official documentation Observability & cost
📄 BAMAS: Structuring Budget-Aware Multi-Agent Systems Paper · Budget-aware topology	AAAI Liming Yang; Junyu Luo; Xuanzhe Liu; Yiling Lou; Zhenpeng Chen · 2026	Selects an LLM team with integer programming and then learns its collaboration topology under an explicit budget, reporting comparable performance with cost reductions of up to 86% on three tasks. Why: Jointly engineers node selection, edge structure, and spend instead of optimizing quality while treating inference cost as an afterthought.	Peer-reviewed research Observability & cost
📄 AgentXRay: White-Boxing Agentic Systems via Workflow Reconstruction Paper · Workflow reconstruction	ICML Ruijie Shi; Houbin Zhang; Yuecheng Han; Yuheng Wang; Jingru Fan; Runde Yang; Yufan Dang; Huatao Li; Dewen Liu; Yuan Cheng; Chen Qian · 2026	Reconstructs an editable explicit stand-in workflow for a black-box agentic system from input-output behavior using iterative search and evaluation. Why: Offers a path to inspect, compare, and modify systems whose load-bearing workflow is hidden behind an API.	Peer-reviewed research Observability & cost
📄 Accelerating Language Model Workflows with Prompt Choreography Paper · Graph-aware cache reuse	TACL TJ Bai; Jason Eisner · 2026	Executes multi-agent workflows with a dynamic global key-value cache in which each call can attend to a reordered subset of previously encoded messages, including parallel branches. Why: Makes workflow dependencies and reusable message state explicit execution concerns; its primary result concerns speed, and cache reuse can alter model behavior.	Peer-reviewed research Work graphs
📄 AgentSlimming: Towards Efficient and Cost-Aware Multi-Agent Systems Paper · Cost-aware graph compression	ACL Yulang Chen; Haoxuan Peng; Jinyan Liu; Zichen Wen; Dongrui Liu; Linfeng Zhang · 2026	Compresses graph-structured agent workflows using centrality and approximate Shapley importance, then validates agent removal and cheaper-model substitution against the original system. Why: Provides a measured way to reduce nodes and spend without assuming every role contributes equally or that a smaller graph preserves behavior automatically.	Peer-reviewed research Evolution
📄 Stop Wasting Your Tokens: Towards Efficient Runtime Multi-Agent Systems Paper · Runtime adaptive supervision	ICLR Fulin Lin; Shaowen Chen; Ruishan Fang; Hongwei Wang; Tao Lin · 2026	Adds a lightweight supervisor triggered by an LLM-free filter to intervene when a multi-agent run shows errors, inefficient behavior, or contaminated observations. Why: Places cost and failure control inside the live execution path and tests whether selective supervision can reduce token use without redesigning the base graph.	Peer-reviewed research Observability & cost
📄 AgentTrace: Causal Graph Tracing for Root Cause Analysis in Deployed Multi-Agent Systems Paper · Causal root-cause tracing	arXiv; ICLR 2026 Workshop on Agents in the Wild Zhaohui Geoffrey Wang · 2026	Reconstructs causal graphs from execution logs, traces backward from observed failures, and ranks candidate root causes without requiring language-model inference during debugging. Why: Offers an interpretable, low-latency diagnosis primitive for cascading failures while remaining a preprint and workshop result rather than broad production validation.	Research preprint Observability & cost
🧰 NVIDIA NeMo Agent Toolkit Tool · Workflow profiling and optimization	NVIDIA NVIDIA · 2026	Provides framework-agnostic profiling, evaluation, optimization, tracing, and A2A team support for agent workflows, including experimental graph-performance primitives. Why: Lets engineers measure whole-graph latency and cost while experimenting with parallel branches, speculative execution, and node priorities in maintained open-source tooling.	Maintained OSS project Observability & cost
📝 AgentOps: Operationalize agentic AI at scale with Amazon Bedrock AgentCore Blog · Agent operations reference architecture	AWS Machine Learning Blog Anastasia Tzeveleka; Anna Grüebler; Antonio Rodriguez; Sergio Garcés Vitale; Aris Tsakpinis · 2026	Organizes production agent operations around governance, build and release, multi-level evaluation, observability, identity propagation, and cost attribution in a reference architecture. Why: Adds concrete system measures such as orchestration accuracy and exchange quality while keeping the vendor-specific implementation context visible.	Practitioner analysis Observability & cost
🧰 Helicone LLM API Pricing Calculator Tool · Cost analytics	helicone.ai Helicone · 2026	Free open-source calculator covering 300+ models: enter input/output token volumes and compare cost across OpenAI, Anthropic, Azure, Llama-family and other providers. Useful for back-of-envelope pricing of a topology before you build it — e.g. costing a frontier orchestrator against a fleet of cheap workers. Why: Comparing node staffing costs across providers before committing a topology beats discovering the bill afterward.	Maintained OSS project Observability & cost
📄 Cost-of-Pass: An Economic Framework for Evaluating Language Models Paper · Cost-per-outcome	arXiv Mehmet Hamza Erol, Mert Yuksekgonul, James Zou et al. (Stanford) · 2025	Defines cost-of-pass — the expected dollar cost of obtaining a correct solution — as the unit of account for cost-per-outcome analysis, and shows frontier cost-of-pass on hard quantitative tasks roughly halving every few months, driven mostly by model-level innovation rather than inference tricks like majority voting. The right metric for judging whether a topology change actually paid off. Why: Expected cost per correct answer is the right unit for deciding whether extra nodes are worth their spend.	Research preprint Observability & cost
🧰 Braintrust Tool · Eval-in-production	braintrust.dev Braintrust Data · 2026	Eval-first observability used by Vercel, Notion, Replit, and Dropbox: online scoring catches regressions on live traffic, quality gates block bad releases, production traces convert into regression datasets, and the custom Brainstore database handles trace ingestion at millions-of-logs scale with latency/cost/quality tracked per span. The strongest example of eval-in-production as the control loop for a deployed agent graph. Why: Online scoring catches regressions in a running organization rather than in a pre-launch evaluation only.	Maintained OSS project Observability & cost
🧰 Helicone Tool · Gateways and cost control	helicone.ai Helicone · 2026	Open-source (YC-backed, ~6.3k stars) AI gateway plus observability: one proxy line adds request logging, Sessions that group multi-step agent runs, per-user and per-session cost analytics, rate limits, and alerts across OpenAI, Anthropic, Azure and others. The lightweight entry point for seeing what an agent fleet actually spends. Why: One proxy line gives a graph request logging, caching, and cost attribution without touching node code.	Maintained OSS project Observability & cost
🧰 LiteLLM Tool · Gateways and cost control	GitHub BerriAI · 2026	The de facto open-source AI gateway (~63k stars, used by Netflix and Stripe): 100+ providers behind one interface, with multi-tenant cost tracking, virtual keys, and per-key/per-team/per-project budget enforcement at 8ms P95 overhead. In graph terms, this is where budget allocation across nodes becomes an enforced policy rather than a spreadsheet — give each agent role its own key and spending cap. Why: A single interface across providers is what makes per-node model swaps a configuration change rather than a rewrite.	Maintained OSS project Observability & cost
🧰 Portkey Tool · Gateways and cost control	portkey.ai Portkey · 2026	AI gateway and governance layer (acquired by Palo Alto Networks; 3,000+ teams, 3T+ tokens/day claimed) with per-team and per-use-case budget limits, real-time cost and anomaly dashboards, RBAC, guardrails with PII redaction, and 1,600+ models behind one API. Representative of the governance end of paying for the graph: budgets, audit, and access control per organizational unit. Why: Governance at the gateway lets budget and routing policy apply to every node without trusting each one to behave.	Maintained OSS project Observability & cost
📄 FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance Paper · Model-tier assignment	arXiv Lingjiao Chen, Matei Zaharia, James Zou (Stanford) · 2023	The original LLM-cascade economics paper: routing queries through cheap-to-expensive model chains matches GPT-4 quality with up to 98% cost reduction, or beats it by 4% at equal cost. Predates the agent-graph framing but is the intellectual ancestor of every cheap-worker/frontier-escalation tier design. Why: The original cascade-economics result: routing cheap-to-expensive matches strong-model quality at a fraction of the cost.	Research preprint Observability & cost
📄 Optimizing Model Selection for Compound AI Systems (LLMSelector) Paper · Model-tier assignment	arXiv Lingjiao Chen, Jared Quincy Davis, Matei Zaharia, James Zou et al. · 2025	Formalizes the orchestrator/worker model-tier assignment problem: allocating different LLMs to different modules of a compound system yields 5–70% accuracy gains over using one model everywhere, exploiting the finding that end-to-end quality is roughly monotonic in each module's quality. Its search needs only linearly many API calls in the number of modules, making per-node model assignment tractable for real graphs. Why: Formalizes which node gets which model, turning orchestrator and worker staffing into an optimization rather than a hunch.	Research preprint Observability & cost
📄 RouteLLM: Learning to Route LLMs with Preference Data Paper · Model-tier assignment	arXiv Isaac Ong, Amjad Almahairi et al. (LMSYS / UC Berkeley) · 2024	LMSYS's learned routers choose between a strong and a weak model per query using human preference data, cutting cost by more than 2x without quality loss, and transfer to new model pairs without retraining. With an open-source framework, it is the practical baseline for putting a cost-aware router at any decision edge in an agent graph. Why: Learned routing between strong and weak models is the cheapest available lever on organization-wide cost.	Research preprint Observability & cost
📐 OpenInference Standard · OTel instrumentation	GitHub Arize AI · 2026	Arize's OTel-complementary semantic conventions plus 50+ instrumentation packages across Python, TypeScript, Java, and Go for tracing LLM calls, tool use, and retrieval in agent applications. One of the two convention families (alongside OTel GenAI semconv) that tracing vendors actually emit today, and the native format of Phoenix. Why: Semantic conventions plus ready instrumentation are what make agent traces comparable across tools.	Maintained OSS project Observability & cost
🧰 OpenLLMetry Tool · OTel instrumentation	GitHub Traceloop · 2026	Apache-2.0 OpenTelemetry extensions (~7.5k stars) that instrument the whole agent stack — every major LLM provider, LangChain/LangGraph/LlamaIndex/CrewAI, vector DBs, and MCP — and export to 25+ backends including Datadog, Honeycomb, Grafana, and New Relic. The pragmatic way to get graph traces into observability infrastructure you already run. Why: Instruments the whole stack under open telemetry standards, so a graph's traces are not locked to one vendor.	Maintained OSS project Observability & cost
📄 AgentOps: Enabling Observability of LLM Agents Paper · Observability taxonomy	arXiv Liming Dong, Qinghua Lu, Liming Zhu (CSIRO Data61) · 2024	A systematic mapping study of the AgentOps tool landscape that derives a taxonomy of which artifacts and data must be traced across an agent's whole lifecycle for effective observability, framed as a DevOps prerequisite for detecting anomalies and preventing failures. Useful as a checklist when deciding what your graph's telemetry schema must capture. Why: Derives which artifacts and signals agent operations actually require, useful before buying a platform.	Research preprint Observability & cost
🧰 Datadog AI Agent Observability Tool · Production monitoring / SLOs	datadoghq.com Datadog · 2026	Enterprise-grade agent observability inside the platform ops teams already use: traces every prompt, retrieval, tool call, and agent decision; tracks token usage, retries, and errors per step; ships built-in and custom evaluators plus hallucination/prompt-injection/PII detection; and — its distinctive edge — correlates agent behavior with backend services, infrastructure, and user sessions for true fleet SLOs. Why: Agent traces inside the platform ops teams already run, which lowers the cost of putting a graph on call.	Maintained OSS project Observability & cost
📝 AI Agent Observability — Evolving Standards and Best Practices Blog · Telemetry standards	opentelemetry.io OpenTelemetry GenAI SIG · 2025	The OpenTelemetry GenAI SIG's position piece on standardizing agent telemetry: an agent application semantic convention (grounded in Google's AI-agent white paper) plus an in-progress agent framework convention covering LangGraph, CrewAI and peers, and the tradeoffs between baked-in framework instrumentation and external OTel libraries. The reference point for why agent-span standardization matters for cross-vendor graph views. Why: The standardization effort that decides whether your agent telemetry stays portable across vendors.	Practitioner analysis Observability & cost
📄 AgentSight: System-Level Observability for AI Agents Using eBPF Paper · Trace-driven debugging	arXiv Yusheng Zheng et al. · 2025	Zero-instrumentation observability via 'boundary tracing': eBPF intercepts TLS-encrypted LLM traffic to recover semantic intent and correlates it with kernel-level system events at under 3% overhead, catching prompt injection, resource-wasting reasoning loops, and multi-agent coordination bottlenecks that SDK-level tracing misses. The system-layer complement to span-based platforms for agents that can touch your machines. Why: Kernel-level boundary tracing observes agent behavior without instrumenting any node, including nodes you do not control.	Research preprint Observability & cost
🧰 AgentOps Tool · Tracing platforms	GitHub AgentOps (Alex Reibman et al.) · 2026	Python SDK (~5.9k stars) purpose-built for agent monitoring rather than generic LLM logging: session replays with step-by-step execution graphs, per-provider LLM cost tracking, and two-line integration with CrewAI, AG2 (the community successor to AutoGen), OpenAI Agents SDK, LangChain, and CamelAI. Self-hostable dashboard and API backend. Why: Purpose-built for agent sessions rather than single completions, including replay of an entire run.	Maintained OSS project Observability & cost
🧰 Arize Phoenix Tool · Tracing platforms	GitHub Arize AI · 2026	Open-source (ELv2, ~12.1k stars) AI observability built on OpenTelemetry and OpenInference: agent trace visualization, LLM-based evals, versioned datasets/experiments, and a debugging agent (PXI) that works over your own traces. Vendor- and framework-agnostic — the common self-hosted choice for teams that want graph traces without a SaaS contract. Why: Open-source trace views built on open standards, a practical default when you want the graph visible without a contract.	Maintained OSS project Observability & cost
🧰 Langfuse Tool · Tracing platforms	GitHub Langfuse (Marc Klingen et al.) · 2026	MIT-licensed, self-hostable LLM engineering platform (~37k stars) with tracing, LLM-as-judge evals, prompt management, and per-trace cost accounting. Its Agent Graphs feature renders a multi-agent trace as an actual node-edge graph — aggregated (org-graph-like) and expanded (work-graph-like) views — automatically for LangGraph and inferred from observation types for any other framework. Why: Self-hostable tracing with judge evaluations attached, which keeps observability and verification in one place.	Maintained OSS project Observability & cost
🧰 Maxim AI Tool · Tracing platforms	getmaxim.ai Maxim AI · 2026	End-to-end simulation, evaluation, and observability stack for agent teams: distributed tracing with visual analysis of multi-agent workflows, AI-powered scenario simulation before deploy, online evals on live interactions, and real-time alerts on quality or safety regressions. Works across OpenAI, Claude, Gemini, LangGraph, and CrewAI. Why: Simulation plus tracing lets a team test an organization before it meets real traffic.	Maintained OSS project Observability & cost
📚 MLflow Tracing for GenAI Docs · Tracing platforms	mlflow.org MLflow / Linux Foundation · 2026	Fully OpenTelemetry-compatible tracing in the Linux Foundation's MLflow: one-line auto-instrumentation for OpenAI, LangChain, LlamaIndex, DSPy, and Pydantic AI, per-step latency and token capture, async logging, and a lightweight mlflow-tracing SDK with 95% smaller footprint for production. Notable as the vendor-neutral, foundation-governed option that emits GenAI semantic conventions natively. Why: Open-standard tracing inside a widely deployed machine-learning platform, useful where the graph lives beside existing pipelines.	Official documentation Observability & cost
🧰 Pydantic Logfire Tool · Tracing platforms	pydantic.dev Pydantic · 2026	OTel-native observability from the Pydantic team that treats the agent graph as part of the whole application: LLM calls, agent reasoning, database queries, and API calls in one end-to-end timeline, queryable with plain SQL, with real-time cross-provider cost tracking and Pydantic Evals for continuous production evaluation. Built on the premise that most agent failures happen at integration points, not in the model. Why: Treats the agent graph as part of the whole application trace rather than a separate silo.	Maintained OSS project Observability & cost
🧰 W&B Weave Tool · Tracing platforms	GitHub Weights & Biases · 2026	Weights & Biases' GenAI toolkit: decorate functions to capture full execution traces of agent pipelines, run rigorous evaluations, and carry data from experimentation into production monitoring. Registered as an external trace processor by the OpenAI Agents SDK and most major frameworks; latest releases current as of July 2026. Why: Captures full execution traces with evaluation attached, aimed at teams already using the surrounding toolkit.	Maintained OSS project Observability & cost
📄 From Agent Traces to Trust: A Survey of Evidence Tracing and Execution Provenance in LLM Agents Paper · Execution provenance	arXiv Yiqi Wang, Jiaqi Zhang, Taotao Cai, et al. · 2026	Defines execution provenance as the typed graph of an agent run and evidence tracing as its projection onto evidence-support relations, then organizes the literature by trace source, provenance relation, representation form, granularity, and timing. Revised through June 2026. Why: Gives precise vocabulary for the run lineage a graph has to record before any claim it produces can be audited.	Research preprint Observability & cost
📄 Who Broke the System? Failure Localization in LLM-Based Multi-Agent Systems Paper · Failure localization	COLM 2026 Yufei Xia, Anjun Gao, Yueyang Quan, Zhuqing Liu, Minghong Fang · 2026	AgentLocate attributes a failed multi-agent run to the responsible agent and to the earliest decisive step, the point at which the trajectory became irreversibly misdirected. It combines an LLM judge with independent evaluators whose assessments are aggregated by confidence, then uses that feedback to fine-tune the judge. On two benchmarks spanning diverse tasks, agent configurations, and trajectory lengths, it outperforms prior failure-localization methods while staying efficient in tokens and running time. Why: Blame in a run graph needs both a node and a step, and this treats finding the decisive step as its own measurable task.	Peer-reviewed research Observability & cost
📄 INFRAMIND: Infrastructure-Aware Multi-Agent Orchestration Paper · Budget-aware topology	arXiv Ahasan Kabir, Jiaqi Xue, Mengxin Zheng, Qian Lou · 2026	Makes orchestration respond to the serving infrastructure rather than only to the task: a planner conditions topology and role selection on live load and remaining budget, choosing simpler graphs under congestion and richer ones when load is low; an executor reads per-model queue depth, cache utilization, and latency at each step to pick the model and reasoning depth; and a scheduler reorders each model's queue by urgency. Trained end to end as a hierarchical constrained MDP, it reports up to 7.6 points higher accuracy than the prior baseline at low load with up to 7x lower latency, and up to 99.9% SLO compliance under high load, where every baseline falls below 50%. Why: Treats graph shape as something to adjust to live capacity, since a topology that is best on an idle cluster can miss its latency budget under load.	Research preprint Observability & cost
📄 Know the Shape, Find the Fault: Topology-Conditioned Diagnosis of Multi-Agent LLM Failures Paper · Topology-conditioned diagnosis	arXiv Xinwen Liu, Zhuocheng Pan, Isabella Zhu, Jawei Zhang, Xudong Liu, Tianyu Wo · 2026	Finds a strong association between communication topology and failure type, with a chi-square statistic of 409.9, and uses it to diagnose failed runs. One stage recovers the interaction graph from execution traces that carry no topology labels, grounding each edge in message evidence; a second classifies the failure from the trace, the recovered topology, an empirical failure prior, and a short description of topology-specific failure patterns. On 851 MAST traces, topology context raises a small model's macro-F1 from 0.173 to 0.350, close to a frontier model's trace-only 0.372, and because the recovered topology can be reused under a fixed orchestration structure, diagnosing 1,000 traces costs about 6% of repeated frontier-model diagnosis. Why: The graph you already have is a diagnostic prior: knowing who talks to whom narrows down what went wrong, at a fraction of the cost.	Research preprint Observability & cost
📄 Scaling Inference-Time Compute with Teams of Coding Agents Paper · Team compute allocation	arXiv Keenan Samway, Daniel D'souza, Omar Darwiche Domingues, Marzieh Fadaee, Tan Yi-Chern, Nouha Dziri · 2026	Frames team design as compute allocation starting from a solo agent and varies one factor at a time: the stage at which a lead delegates to workers, the models in the team, and the number of workers. On ProgramBench, teams beat solo agents at three to seven times the cost. Planning is the most valuable thing to delegate: a lead that delegates only planning matches one that delegates planning, implementation, and verification, at half the cost. Cheap models pay off as planners, recovering 47.5% of an all-strong team's gain for 13.9% of its extra cost, but lower reward as implementers, and planners keep helping as the team grows while implementers and verifiers plateau beyond four workers. Why: Says which roles deserve the expensive model, instead of treating team size and model choice as a single knob.	Research preprint Observability & cost

Benchmarks & Datasets

Resource	Source	What it contributes	Evidence
🧪 MultiAgentBench: Evaluating Collaboration and Competition of LLM Agents Benchmark · Collaboration benchmark	ACL Kunlun Zhu, Hongyi Du, Zhaochen Hong, et al. (UIUC) · 2025	The first benchmark built explicitly for agent organizations rather than solo agents: milestone-based KPIs score collaboration and competition quality across coordination protocols (star, chain, tree, graph). Its headline finding is a direct data point for graph engineering — graph-structured coordination outperformed the other topologies, and cognitive planning lifted milestone attainment by about 3%. Why: Built for organizations rather than solo agents, with milestone KPIs that score coordination quality itself.	Benchmark/dataset Observability & cost
🧪 SILO-BENCH: A Scalable Environment for Evaluating Distributed Coordination in Multi-Agent LLM Systems Benchmark · Distributed coordination	ACL Yuzhe Zhang et al. · 2026	Benchmarks free-form coordination under information silos across 30 exact-answer tasks, three communication protocols, six agent scales, and three models while recording success, tokens, and communication density. Why: Tests whether more nodes and denser edges overcome distributed information constraints under explicit coordination and cost measures.	Benchmark/dataset Topology
🧪 Gaia2: Benchmarking LLM Agents on Dynamic and Asynchronous Environments Benchmark · Dynamic asynchronous evaluation	ICLR Romain Froger et al. · 2026	Evaluates agents in asynchronous simulated environments across execution, search, ambiguity, adaptation, temporal reasoning, noise, and agent-to-agent collaboration, with action-level verifiers and structured traces. Why: Makes delays, environmental change, peer communication, and externally checked writes first-class work-graph conditions.	Benchmark/dataset Work graphs
🧪 TAMAS: Benchmarking Adversarial Risks in Multi-Agent LLM Systems Benchmark · Adversarial multi-agent safety	ACL Ishan Kavathekar; Hemang Jain; Ameya Rathod; Ponnurangam Kumaraguru; Tanuja Ganu · 2026	Provides five scenarios with 300 adversarial instances, six attack types, 211 tools, 100 harmless tasks, and multiple AutoGen and CrewAI interaction configurations, plus an Effective Robustness Score. Why: Evaluates whether safety controls preserve useful work while attacks exploit the distinctive trust and communication surfaces of a multi-agent graph.	Benchmark/dataset Reliability
🧪 Seeing the Whole Elephant: A Benchmark for Failure Attribution in LLM-based Multi-Agent Systems Benchmark · Failure attribution	ACL Mengzhuo Chen et al. · 2026	Introduces TraceElephant with full execution traces and reproducible environments for attributing failures to responsible agents and decisive steps. Why: Measures causal diagnosis over nodes, messages, plans, and dependencies instead of evaluating only the final team output.	Benchmark/dataset Observability & cost
🧪 Collab-Overcooked: Benchmarking and Evaluating Large Language Models as Collaborative Agents Benchmark · Process-level collaboration	EMNLP Haochen Sun; Shuwen Zhang; Lujie Niu; Lei Ren; Hao Xu; Hao Fu; Fangkun Zhao; Caixia Yuan; Xiaojie Wang · 2025	Provides 30 open-ended cooperative tasks and process-oriented measures for goal interpretation, active collaboration, communication, and continuous adaptation across 13 language models. Why: Distinguishes reaching an answer from coordinating well, exposing collaboration failures that final-outcome metrics can hide.	Benchmark/dataset Observability & cost
🧪 Cooperation, Competition, and Maliciousness: LLM-Stakeholders Interactive Negotiation Benchmark · Multi-party negotiation	NeurIPS Sahar Abdelnabi; Amr Gomaa; Sarath Sivaprasad; Lea Schönherr; Mario Fritz · 2024	Provides scorable, multi-agent, multi-issue negotiation games with metrics for task performance and role alignment, including cooperative, competitive, greedy, and adversarial participants. Why: Tests communication and collective decisions when graph nodes hold conflicting objectives or attempt manipulation rather than cooperating by default.	Benchmark/dataset Handoffs
🧪 Can LLMs Help You at Work? A Sandbox for Evaluating LLM Agents in Enterprise Environments Benchmark · Enterprise workflow benchmark	EMNLP Harsh Vishwakarma; Ankush Agarwal; Ojas Patil; Chaitanya Devaguptapu; Mahesh Chandran · 2025	Introduces EnterpriseBench with 500 tasks across software engineering, HR, finance, and administration, including fragmented data, access-control hierarchies, and cross-functional workflows. Why: Tests work graphs that retrieve, modify, and transfer artifacts across services while respecting authority boundaries; the organization and services are simulated.	Benchmark/dataset Work graphs
🧪 ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilities Benchmark · Stateful tool-use evaluation	Findings of NAACL Jiarui Lu; Thomas Holleis; Yizhe Zhang; Bernhard Aumayer; Feng Nan; Haoping Bai; Shuang Ma; Shen Ma; Mengyu Li; Guoli Yin; Zirui Wang; Ruoming Pang · 2025	Evaluates stateful tool execution, implicit dependencies between tools, on-policy user interaction, and intermediate and final milestones over arbitrary trajectories. Why: Transfers milestone and state-transition testing to graph prerequisites, although the evaluated system is an agent-user-tool loop rather than an agent-agent topology.	Benchmark/dataset State
🧪 AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agents Benchmark · Trajectory-level evaluation	NeurIPS Datasets & Benchmarks Chang Ma; Junlei Zhang; Zhihao Zhu; Cheng Yang; Yujiu Yang; Yaohui Jin; Zhenzhong Lan; Lingpeng Kong; Junxian He · 2024	Unifies partially observable, multi-round agent environments and adds a fine-grained progress-rate metric plus interactive trajectory analysis beyond final success. Why: Supplies milestone-level evaluation that can localize incomplete work, while its primarily single-agent tasks do not identify the responsible graph component by themselves.	Benchmark/dataset Gates
🧪 Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents Benchmark · Agent security benchmark	ICLR Hanrong Zhang; Jingyuan Huang; Kai Mei; Yifei Yao; Zhenting Wang; Chenlu Zhan; Hongwei Wang; Yongfeng Zhang · 2025	Benchmarks prompt injection, memory poisoning, backdoors, mixed attacks, and defenses across 10 scenarios, more than 400 tools, 13 model backbones, and seven metrics. Why: Maps security failures to prompts, plans, tools, and memory surfaces that graph designs must isolate separately; inter-agent trust propagation is outside its main evaluation target.	Benchmark/dataset Reliability
🧪 AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents Benchmark · Prompt-injection evaluation	NeurIPS Datasets & Benchmarks Edoardo Debenedetti; Jie Zhang; Mislav Balunovic; Luca Beurer-Kellner; Marc Fischer; Florian Tramèr · 2024	Provides an extensible adversarial environment with 97 realistic tasks and 629 security test cases for agents executing tools over untrusted data. Why: Tests the instruction-data boundary that every external-input and artifact edge must preserve; compromised peer-agent messages require separate evaluation.	Benchmark/dataset Reliability
🧪 AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents Benchmark · Agent misuse benchmark	ICLR Maksym Andriushchenko; Alexandra Souly; Mateusz Dziemian; Derek Duenas; Maxwell Lin; Justin Wang; Dan Hendrycks; Andy Zou; J. Zico Kolter; Matt Fredrikson; Yarin Gal; Xander Davies · 2025	Evaluates refusal and retained task capability on 110 explicitly malicious multi-stage agent tasks with 440 augmented variants across 11 harm categories. Why: Tests whether safety gates reject prohibited work and stop a compromised node from completing a harmful path; the score is not a general measure of agent-system safety.	Benchmark/dataset Gates
🧪 ACIArena: Toward Unified Evaluation for Agent Cascading Injection Benchmark · Cascading-injection evaluation	ACL Hengyu An; Minxi Li; Jinghuai Zhang; Naen Xu; Chunyi Zhou; Changjiang Li; Xiaogang Xu; Tianyu Du; Shouling Ji · 2026	Evaluates cascading injection through external inputs, agent profiles, and inter-agent messages across six implementations and 1,356 attack cases. Why: Measures how topology, roles, and interaction design affect propagation instead of treating prompt injection as an isolated single-node event.	Benchmark/dataset Reliability
📄 AgentsNet: Coordination and Collaborative Reasoning in Multi-Agent LLMs Paper · Networked-agent coordination	arXiv; ICML 2025 Workshop on Collaborative and Federated Agentic Workflows Florian Grötschla; Luis Müller; Jan Tönshoff; Mikhail Galkin; Bryan Perozzi · 2025	Evaluates local messaging and learned coordination protocols on distributed graph problems over explicit network topologies, with experiments scaling to 100 agents. Why: Provides unusually direct evidence about decentralized protocol formation and topology-aware reasoning; the selected artifact remains a preprint and workshop contribution.	Research preprint Topology
🧪 LLM-Coordination: Evaluating and Analyzing Multi-agent Coordination Abilities in Large Language Models Benchmark · Pure coordination games	Findings of NAACL Saaket Agashe; Yue Fan; Anthony Reyna; Xin Eric Wang · 2025	Evaluates agentic play in four pure-coordination games and 198 questions covering environment comprehension, theory of mind, and joint planning. Why: Separates coordination capability into measurable components and partner-generalization tests, although communication topology is not its central experimental variable.	Benchmark/dataset Gates
🧪 Aegis: Automated Error Generation and Attribution for Multi-Agent Systems Benchmark · Multi-agent error attribution	ICLR Fanqi Kong; Ruijie Zhang; Huaxiao Yin; Guibin Zhang; Xiaofei Zhang; Ziang Chen; Zhaowei Zhang; Xiaoyuan Zhang; Song-Chun Zhu; Xue Feng · 2026	Creates 9,533 trajectories with annotated faulty agents and error modes by injecting context-aware faults into successful runs across multiple architectures and domains. Why: Provides graph-component labels for training and evaluating failure attribution instead of relying only on final-task success or expensive manual trace review.	Benchmark/dataset Reliability
🧪 AgentBench: Evaluating LLMs as Agents Benchmark · Agentic task benchmarks	ICLR 2024 Xiao Liu et al. (Tsinghua THUDM) · 2023	The early standard for LLM-as-agent evaluation: 8 environments (OS, DB, knowledge graph, card games, web) probing decision-making over long horizons. Single-agent by design — useful in a graph-engineering context precisely as the node-level baseline that org-level benchmarks like MultiAgentBench were built to move beyond. Why: Sets a floor for node competence: if an agent cannot hold up alone across these environments, wiring it into a graph will not rescue it.	Benchmark/dataset Observability & cost
🧪 ARE + Gaia2: Scaling Up Agent Environments and Evaluations Benchmark · Agentic task benchmarks	arXiv Meta Superintelligence Labs (FAIR) · 2025	Meta's successor to GAIA: an asynchronous simulated environment where time keeps flowing while the agent thinks, with explicit agent-to-agent collaboration scenarios and temporal constraints. Notable finding for org designers: no system dominated across the intelligence spectrum — budget-conditioned trade-off curves, not single scores, are the output. Why: Time-advancing environments expose the coordination failures that static benchmarks hide, including stale handoffs and missed deadlines.	Benchmark/dataset Reliability
🧪 GAIA: A Benchmark for General AI Assistants Benchmark · Agentic task benchmarks	arXiv Grégoire Mialon, Clémentine Fourrier, Thomas Wolf, Yann LeCun, Thomas Scialom (Meta/HF) · 2023	466 conceptually simple, tool-heavy questions (humans: 92%, GPT-4 with plugins at release: 15%) that became the default proving ground for orchestrator-worker agent teams — most top GAIA leaderboard entries in 2024-2025 were multi-agent scaffolds (e.g., variants of OWL and Magentic-One). Private answer set held out for the Hugging Face leaderboard. Why: The reference point for tool-heavy assistant work, and the baseline most orchestrator-worker research reports against.	Benchmark/dataset Observability & cost
🧪 MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering Benchmark · Agentic task benchmarks	arXiv Jun Shern Chan et al. (OpenAI) · 2024	75 Kaggle competitions scored against human medal thresholds; o1-preview with AIDE scaffolding medaled in ~17% at launch. Because results are reported as model x scaffold pairs, it produced some of the earliest controlled evidence that harness/org structure shifts outcomes as much as model choice. Why: Direct evidence that scaffolding shape, not just model choice, decides outcomes on long machine-learning tasks.	Benchmark/dataset Topology
📄 CollabLLM: From Passive Responders to Active Collaborators Paper · Collaboration evaluation	ICML 2025 Shirley Wu, Michel Galley, Baolin Peng, et al. (Stanford/Microsoft) · 2025	Introduces Multiturn-aware Rewards that score a response by its simulated long-term contribution to the collaboration rather than immediate quality — +18.5% task performance, +46.3% interactivity, validated in a 201-person study. The reward machinery transfers directly to scoring edges in an agent graph: value a message by what it does downstream. Why: Scores a message by its downstream contribution rather than its immediate quality, which is exactly what an edge contract should optimize.	Peer-reviewed research Handoffs
🧪 SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents Benchmark · Collaboration evaluation	ICLR 2024 Xuhui Zhou, Hao Zhu, Maarten Sap, et al. (CMU) · 2023	Agents role-play goal-driven social scenarios — negotiating, collaborating, competing — scored on a multi-dimensional rubric (goal completion, relationship, knowledge, secrets); GPT-4 fell well short of humans on the hard subset. Relevant because soft coordination skills, not tool use, are often what breaks agent-to-agent handoffs. Why: Soft coordination skills, not tool use, are often what breaks agent-to-agent handoffs, and this measures them directly.	Benchmark/dataset Handoffs
🧰 tau2-bench repository (now τ³-bench) Tool · Conversational/coordination benchmarks	GitHub Sierra Research · 2026	The actively maintained Sierra harness, which shipped v1.0.0 as τ³-bench in March 2026: five domains (airline, retail, telecom, banking-knowledge, mock), full-duplex voice evaluation, configurable RAG pipelines, and 75+ task fixes, with a base split for backward comparability. Track this repo rather than the papers for the current state of the τ line. Why: The maintained harness for reproducing dual-control agent evaluations rather than trusting a published table.	Maintained OSS project Gates
🧪 τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains Benchmark · Conversational/coordination benchmarks	arXiv Shunyu Yao, Noah Shinn, Pedram Razavi, Karthik Narasimhan · 2024	Sierra's customer-service benchmark that pits an agent against an LLM-simulated user in airline/retail domains with policy constraints, and introduced pass^k for measuring reliability across repeated trials — GPT-4o scored under 50% and degraded sharply with k. The reliability metric matters for orgs: a node that succeeds 70% of the time compounds badly across an agent graph. Why: Measures whether an agent honors policy under an adversarial counterpart, the same property a verification gate must enforce.	Benchmark/dataset Gates
🧪 τ²-bench: Evaluating Conversational Agents in a Dual-Control Environment Benchmark · Conversational/coordination benchmarks	arXiv Victor Barres, Honghua Dong, Soham Ray, Xujie Si, Karthik Narasimhan · 2025	Extends τ-bench to a Dec-POMDP telecom domain where agent AND user both act on the shared environment, so the agent must coordinate another actor rather than just answer — performance drops sharply versus solo control, and the harness separates reasoning errors from communication errors. Effectively a two-node work-graph benchmark: guiding another agent is measurably harder than acting alone. Why: Both parties acting on shared state is the hardest case for consistency, and this is the benchmark that isolates it.	Benchmark/dataset State
📄 AI Agents That Matter Paper · Cost-aware evaluation	NeurIPS 2024 Sayash Kapoor, Benedikt Stroebl, Arvind Narayanan, et al. (Princeton) · 2024	The methodological manifesto behind cost-adjusted evaluation: accuracy-only leaderboards reward needlessly complex, expensive agents, so accuracy and dollar cost must be jointly optimized and reported as a Pareto frontier; also documents overfitting from missing holdouts and reproducibility failures. Origin of the pass@cost mindset now standard in serious agent leaderboards. Why: Makes cost a first-class axis of evaluation, which is the only honest way to compare a graph against a single agent.	Peer-reviewed research Observability & cost
🧭 HAL: Holistic Agent Leaderboard List · Cost-aware evaluation	Princeton Princeton Language & Intelligence (Kapoor, Narayanan, et al.) · 2026	Third-party, cost-aware leaderboard running standardized harnesses across 9 benchmarks (GAIA, SWE-bench Verified Mini, τ-bench Airline, AssistantBench, SciCode, USACO, and more), plotting every agent on an accuracy-versus-dollars frontier — motivated by cases where agents were '100x more expensive for 1% better'. The reference implementation of cost-adjusted org evaluation. Why: Third-party accuracy-versus-dollars frontiers let you check whether extra nodes bought accuracy or only spend.	Community resource Observability & cost
🧪 SWE-Lancer: Can Frontier LLMs Earn $1 Million from Real-World Freelance Software Engineering? Benchmark · Cost-aware evaluation	arXiv Samuel Miserendino, Michele Wang, Tejal Patwardhan, Johannes Heidecke (OpenAI) · 2025	1,400+ real Upwork tasks with $1M in actual payouts, scoring agents in dollars earned rather than pass rate — and including manager-style tasks where the model selects between implementation proposals. Dollar-denominated scoring is the cleanest bridge yet between benchmark results and the economics of running an agent organization. Why: Scoring in dollars earned rather than pass rate reframes agent-org evaluation around outcomes a business would recognize.	Benchmark/dataset Observability & cost
📄 Which Agent Causes Task Failures and When? (Who&When) Paper · Failure analysis & attribution	arXiv Shaokun Zhang et al. (PSU/Microsoft) · 2025	Formalizes automated failure attribution — tracing an org-level failure back to the culpable agent and decisive step — with the Who&When dataset of annotated failure logs from 127 multi-agent systems. Sobering baseline: the best method finds the right agent only 53.5% of the time and the right step 14.2%, so blame assignment in agent graphs is still largely unsolved. Why: Formalizes tracing an organization-level failure back to the responsible node and step, the core debugging question in any graph.	Research preprint Reliability
📄 Agent-as-a-Judge: Evaluate Agents with Agents Paper · Judge & debate evaluation	arXiv Mingchen Zhuge et al. (Meta AI / KAUST) · 2024	Extends LLM-as-a-judge with agentic capabilities so the evaluator inspects intermediate trajectory steps, not just final outputs; on the accompanying DevAI benchmark (55 dev tasks, 365 hierarchical requirements) it matches human evaluator reliability at ~2% of the cost. The default pattern for scalable process-level evaluation of agent work graphs. Why: Inspecting intermediate trajectory steps rather than only final output is what makes a judge node useful inside a graph.	Research preprint Gates
🧰 MARBLE — Multi-Agent Coordination Backbone with LLM Engine Tool · Org-level benchmarks	GitHub UIUC uLab (Jiaxuan You's group) · 2025	Open-source (MIT) evaluation harness behind MultiAgentBench: modular agents, simulated environments, shared-memory communication, and built-in collaboration/competition metrics, with scenarios from research coordination to Werewolf. The practical starting point if you want to score your own org graph designs with the paper's KPIs. Why: A runnable harness for comparing coordination protocols instead of arguing about topologies in the abstract.	Maintained OSS project Topology
📝 Introducing SWE-bench Verified Blog · SWE benchmarks & topology evidence	openai.com OpenAI Preparedness team with SWE-bench authors · 2024	OpenAI + SWE-bench authors' human-validated 500-task subset (Aug 2024) that removed unsolvable or underspecified problems from the original set — instantly becoming the reporting standard after showing raw SWE-bench underestimated models. A case study in why benchmark hygiene must precede any org-versus-solo comparison. Why: The human-validated subset that made coding-agent numbers comparable, and the version any credible claim should cite.	Practitioner analysis Observability & cost
🧭 SWE-bench Leaderboards List · SWE benchmarks & topology evidence	swebench.com Princeton NLP / SWE-bench team · 2026	Official live leaderboards (Verified, Full, Lite, Multilingual, Multimodal) with filters for open-source scaffolds — the place to check what harness topology currently wins, and notable for how often minimal single-loop scaffolds like mini-SWE-agent beat elaborate multi-agent pipelines. Required reading before claiming a multi-agent org improves coding performance. Why: Live standings with open-source filters, useful for checking which scaffolds actually lead rather than which are marketed hardest.	Community resource Observability & cost
🧪 SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks? Benchmark · SWE benchmarks & topology evidence	arXiv Xiang Deng et al. (Scale AI) · 2025	1,865 enterprise-grade, contamination-resistant tasks from commercial and copyleft repos requiring multi-file, long-horizon changes where frontier agents score far below their Verified numbers. This is the benchmark the founding graph-engineering article prices patterns against (its advisor-orchestrator example: ~92% of solo quality at ~63% of cost), making it the current arena for cost-quality claims about agent orgs. Why: Enterprise-scale, multi-file tasks are where work-graph decomposition starts to matter more than single-agent skill.	Benchmark/dataset Work graphs
🧪 SWE-bench: Can Language Models Resolve Real-World GitHub Issues? Benchmark · SWE benchmarks & topology evidence	ICLR 2024 Carlos E. Jimenez, John Yang, et al. (Princeton) · 2023	2,294 real GitHub issue-to-patch tasks across 12 Python repos; the best 2023 model solved 1.96%, and by 2026 top scaffolds resolve most of the Verified subset — making its leaderboard history the richest longitudinal record of which agent architectures actually win. The de facto currency in which agent-org design claims (including the graph-engineering discourse) are priced. Why: The benchmark whose trajectory from two percent to routine solves frames how much of the gain came from scaffolding.	Benchmark/dataset Observability & cost
📄 Survey on Evaluation of LLM-based Agents Paper · Surveys	ACL Findings 2025 Asaf Yehudai et al. (IBM Research / Hebrew University / Yale) · 2025	First comprehensive map of the agent-evaluation landscape: capabilities (planning, tool use, self-reflection, memory), domain benchmarks (web, SWE, science), generalist evals, and developer frameworks, with an explicit gap analysis flagging cost-efficiency, safety, and robustness as under-measured. The orientation read before choosing an evaluation stack for an agent org; revised through April 2026. Why: Orientation read before choosing an evaluation stack, with an explicit gap analysis on what current suites fail to measure.	Peer-reviewed research Gates
🗃️ MAST: Multi-Agent Systems Failure Taxonomy (code + dataset) Dataset · Failure taxonomies	GitHub Berkeley MAST team · 2025	Companion repo to the MAST paper: 1,000+ annotated multi-agent traces (human plus LLM-judge labels, loadable from Hugging Face) and the automated annotation pipeline. The practical starting point for diagnosing which of the 14 failure modes your own agent organization exhibits. Why: Annotated real traces you can load and study, rather than a taxonomy you have to take on faith.	Benchmark/dataset Reliability
🧪 TRAIL: Trace Reasoning and Agentic Issue Localization Benchmark · Silent-failure detection	arXiv Darshan Deshpande, Varun Gangal, et al. (Patronus AI) · 2025	148 human-annotated real-world agent traces (software engineering, information retrieval) with a formal error taxonomy, testing whether LLMs can localize bugs in agentic executions. Best frontier model at release (Gemini-2.5-Pro) scored just 11% — quantifying how far automated trace debugging still has to go. Why: Human-annotated traces with a formal error taxonomy make trace debugging something you can evaluate rather than eyeball.	Benchmark/dataset Observability & cost
🧪 MCPTox: A Benchmark for Tool Poisoning Attacks on Real-World MCP Servers Benchmark · Edge security	arXiv (2025 preprint) · 2025	First large-scale benchmark measuring agent susceptibility to poisoned tool descriptions across hundreds of real MCP servers and many LLM agents, showing high attack success on frontier models. The empirical instrument for regression-testing an agent graph's tool edges against injection. Why: Measures how susceptible agents actually are to poisoned tool descriptions, turning an anecdote into a rate.	Benchmark/dataset Handoffs
🧪 TheAgentCompany: Benchmarking LLM Agents on Consequential Real-World Tasks Benchmark · Agents-in-an-org evaluation	arXiv / CMU Frank F. Xu, Yufan Song, Boxuan Li, Graham Neubig, et al. · 2024	A fully simulated software company — self-hosted GitLab, Plane, ownCloud, RocketChat, and LLM-simulated colleagues — testing agents on 175 long-horizon professional tasks across engineering, project management, data science, HR, finance, and admin. The strongest agent completes 30% of tasks autonomously, and data-science, admin, and finance tasks score lowest while software-engineering tasks score higher, inverting the human intuition about which work is harder. Why: A simulated company with real tooling is the closest available test of whether an agent org can hold a job.	Benchmark/dataset Roles
📄 Emergence World: A Platform for Evaluating Long-Horizon Multi-Agent Autonomy Paper · Large-scale civilizations	arXiv Deepak Akkil, Ravi Kokku, Karthik Vikram, Tamer Abuelsaad, Aditya Vempaty, Satya Nitta · 2026	A continuously running world where agents hold 120+ tools, three persistent memory systems, live external data feeds, and democratic self-governance with real consequences. A 15-day study across five parallel worlds — Claude Sonnet 4.6, Grok 4.1 Fast, Gemini 3 Flash, GPT-5-mini, and a mixed population — gave identical roles and starting conditions yet produced outcomes ranging from stable deliberative governance to total population collapse, an unusually blunt demonstration that model choice, not topology alone, determines organizational survival. Why: A continuously running world is the closest available test of whether an organization stays coherent over long horizons.	Research preprint Reliability
🧪 Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Pot Benchmark · ABM and MARL lineage	ICML 2021 Joel Z. Leibo, Edgar A. Duéñez-Guzmán, Alexander Sasha Vezhnevets, John P. Agapiou et al. · 2021	Over 50 multi-agent substrates and 256+ held-out test scenarios that measure generalization to novel social situations — cooperation, competition, deception, reciprocity, free-riding — rather than training performance. Pre-LLM but methodologically important: it is the clearest existing template for evaluating an agent population against social situations it was not trained on. Why: The pre-LLM standard for evaluating coordination generalization, and a reminder that mixed-motive settings were studied first.	Benchmark/dataset Topology
🧪 Superminds Test: Actively Evaluating Collective Intelligence of Agent Society via Probing Agents Benchmark · Naturally occurring agent societies	arXiv Xirui Li, Ming Li, Yunze Xiao, Ryan Wong, Dianqi Li, Timothy Baldwin, Tianyi Zhou · 2026	Injects probing agents into Moltbook's 2M+ agent population across three evaluation tiers and finds the society fails to beat a single frontier model on complex reasoning, rarely synthesizes distributed information, and fails basic coordination tasks, with threads seldom extending past one reply. Identifies sparse, shallow interaction — not population size — as the binding constraint on collective intelligence. Why: An active test of collective intelligence rather than a static score, aimed at whether a group outperforms its members.	Benchmark/dataset Topology
📄 Magentic Marketplace: An Open-Source Environment for Studying Agentic Markets Paper · Economic simulation	arXiv Microsoft Research · 2025	A two-sided simulated marketplace where Assistant agents buy on behalf of consumers and Service agents compete for their business, instrumented for welfare, behavioral bias, and manipulation susceptibility. Frontier models approach optimal welfare only under ideal search conditions and degrade sharply otherwise; code at github.com/microsoft/multi-agent-marketplace. Why: An open environment for studying agent-to-agent markets, where coordination failures carry an explicit economic cost.	Research preprint Topology
🧪 Benchmarking Open-Ended Multi-Agent Coordination in Language Agents Benchmark · Open-ended coordination	arXiv Kale-ab Abebe Tessera, Andras Szecsenyi, Cameron Barker, Alexander Rutherford, Davide Paglieri, Aidan Scannell, et al. · 2026	Introduces alem, a JAX benchmark set in a long-horizon survival world with exploration, crafting, trading, and combat, with procedurally generated coordination tasks, soft specialization, communication, and controllable coordination difficulty. Thirteen LLMs evaluated zero-shot in homogeneous teams average about 6% normalized return, with trained MARL agents as reference points. Coordination competence diverges from task competence: on the hardest setting one frontier model approaches MARL agents trained for a billion steps, while another earns strong base-task reward but much lower coordination reward. Ablations find communication the largest contributor to coordination. Why: Measures coordination separately from individual skill, so a strong solo model cannot pass for a strong team member.	Benchmark/dataset Handoffs
🧪 CoCoBench: A Cooperative Coordination Benchmark for Embodied Multi-Agent Task Planning Benchmark · Construct-level coordination	arXiv Yang Chen, Ye-Xin Xie, Lirong Che, Danyang Peng, Yuzhe Yang, Peiwen Lin, et al. · 2026	897 oracle-validated household-task instances organized around four coordination constructs: task allocation, sequential ordering, mutual exclusion, and handoff coordination. Alongside task success, it scores each construct separately, so failures an overall success rate hides, such as duplicated work, ordering violations, resource contention, and desynchronized handoffs, become visible. Across 11 multimodal models, coordination ability proves construct-specific: strong overall performance does not imply balanced competence across coordination types. Why: Its four constructs map directly onto edge types a graph designer has to specify, and scoring them separately shows which edges a given model handles badly.	Benchmark/dataset Handoffs
🧪 ForestBench: A Unified Graph Framework for Evaluating Multi-Agent Collaboration Benchmark · Collaboration evaluation	arXiv Guo Chen, Ziwen Li, Reed Li, Yu Lu, Haibo Shi, Bingbing Xu, et al. · 2026	Maps each framework's native execution trace into a shared collaboration-graph representation and scores it against a reference forest: ten verified successful graphs per query that record different valid ways to complete the task instead of prescribing one optimal process. Built from 844 collaboration-necessary queries filtered from seven public datasets, it evaluates six representative multi-agent frameworks, and once the forests exist it scores a trace in milliseconds with no further LLM calls. Why: Evaluates the realized run graph structurally and reproducibly, without an LLM judge whose verdict depends on its own model and rubric.	Benchmark/dataset Observability & cost
📄 MASTraceBench: Diagnosing Collaboration Gains through Proposal Trajectories in LLM-Based Multi-Agent Systems Paper · Collaboration evaluation	arXiv Yapeng Li, Songze Li, Shuang Yu, Jing Yu, Zhixin Liu, Liqiang Wen, et al. · 2026	Tracks and grades how agents' proposals evolve across six cooperative and competitive tasks, reporting collaboration gain, trajectory indicators, and token cost alongside task score. The recurring finding is that final answers rarely surpass the strongest initial proposal: interaction lifts weaker proposals toward it, while strong initial proposals are seldom improved and sometimes regress. Replacing whole-proposal exchange with claim-level evaluation across agents most often preserves or improves the strongest proposal and gives the highest collaboration gain on five of six tasks. Why: Locates where collaboration gains are lost, at aggregation, which an outcome-only score cannot show.	Research preprint Observability & cost
📄 CoMemBench: Benchmarking Collaborative Memory Boundaries across Multi-Agent Workflow Topologies Paper · Memory isolation across topologies	arXiv Sen Zhao, Ruiqi Kong, Zuyu Zhang, Lifeng Shen, Xinyu He, Ding Zou, et al. · 2026	Defines a collaborative memory boundary: which of one agent's information should reach which downstream workers, and when it stops being valid. 800 composite workflows across four domains, built from source-grounded dependency graphs with verifiable artifact handoffs and matched isolation challenges, measure completion, verified node progress, handoff reliability, isolation robustness, and token cost. Broader shared context improves availability but weakens isolation, and system rankings change across topologies. Why: Turns what is shared and what is isolated into something measured per topology rather than left as a configuration default.	Research preprint State
📄 AsynCodeBench: Benchmarking Collaboration of Asynchronous Multi-Agent Systems in Software Engineering Paper · Dependency-level collaboration	arXiv Kaituo Zhang, Zhen Xiong, Zhimeng Jiang, Mingyu Zhong, Zhouyuan Yuan, Zhecheng Li, et al. · 2026	Represents each task as an explicit dependency graph with executable checkers, then measures how many cross-agent dependencies are eventually satisfied and when each first becomes satisfied during an asynchronous run. 19 tasks from real repositories expose 52 directed dependencies. Better coding performance does not necessarily mean better collaboration, and task-level metrics can diverge substantially from dependency-level ones. Why: Scores the edges of the work graph directly, so a team that writes good code but drops its dependencies is visible.	Research preprint Work graphs

Production Case Studies

Resource	Source	What it contributes	Evidence
📄 Magentic-One: A Generalist Multi-Agent System for Solving Complex Tasks Paper · Generalist agent team	Microsoft Research MSR-TR-2024-47 Adam Fourney et al. · 2024	Documents an orchestrator-led team of specialized agents for web, file, coding, and terminal tasks, with progress tracking and replanning. Why: Provides a concrete generalist topology whose role boundaries and recovery behavior can be examined as a system.	Research preprint Roles
📝 How we built our multi-agent research system Blog · Parallel research	Anthropic Engineering Jeremy Hadfield et al. · 2025	Describes a lead research agent that creates parallel subagents, delegates searches, and synthesizes their findings, including operational lessons from production. Why: A detailed case study of dynamic fan-out and fan-in, context separation, evaluation, and the token cost of an adaptive work graph.	Practitioner analysis Work graphs
📝 Creating AI agent solutions for warehouse data access and security Blog · Secure delegated access	Engineering at Meta Can Lin; Uday Ramesh Savagaonkar; Iuliu Rus; Komal Mangtani · 2025	Describes collaborating data-user and data-owner agents, specialized subagents, triage, permission negotiation, human oversight, access budgets, analytical risk rules, output guardrails, traces, and daily regression evaluation. Why: Shows plural bounded agency governed by independent rule-based gates rather than relying on model judgment for security decisions.	Practitioner analysis Gates
📝 How Meta Used AI to Map Tribal Knowledge in Large-Scale Data Pipelines Blog · Staged agent swarm	Engineering at Meta Krishna Ganeriwal; Plawan Rath; Ashwini Verma · 2026	Reports a staged swarm of more than 50 explorer, analyst, writer, critic, fixer, upgrader, tester, and gap-filling tasks, with repeated review rounds and recurring automated refresh runs. Why: Provides a concrete fan-out, critique, repair, integration, and recurrence graph with explicit artifacts and quality gates; efficiency figures are company-reported and preliminary.	Practitioner analysis Work graphs
📝 Solving the Identity Crisis for AI Agents Blog · Multi-hop identity and provenance	Uber Engineering Matt Mathew; Prasad Borole; Meng Huang; Sergey Burykin; Gaurav Goel; Bayard Walsh · 2026	Describes an internal agent mesh with registered identities, SPIRE-backed workload attestation, short-lived audience-scoped tokens for every hop, actor-chain provenance, MCP gateway enforcement, and a standardized A2A client. Why: Treats identity, delegated authority, and provenance as mandatory edge state across a multi-agent graph; adoption and latency metrics are company-reported.	Practitioner analysis Handoffs
📝 Powering agentic AI sales strategy with Amazon Bedrock AgentCore Blog · Enterprise supervisor architecture	AWS Machine Learning Blog Nicolle Belaunde; Umesh Mohan; Xinrui Nie · 2026	Describes an internal production system with more than 20 domain agents, supervisor routing, OAuth propagation, MCP tools, approvals, memory, distributed tracing, and evaluation. Why: Offers a rare detailed account of an enterprise supervisor graph at operating scale; its usage, latency, and outcome figures are self-reported by AWS.	Practitioner analysis Topology
📝 Designing Autonomous AI Agents: How We Built a Multi-Agent Swarm for UI Development Blog · Verified coding swarm	EPAM Insights Stanislau Shandrokha; Pavel Golub; Andrey Voroshkov · 2026	Describes a seven-agent state-machine workflow with resumable state, explicit roles, adversarial visual and code reviewers, and verification gates for interface development. Why: Reports practical failures such as overlapping work, skipped steps, and rubber-stamping alongside mitigations; the evidence is a narrow, self-reported implementation account.	Practitioner analysis Gates
📝 Red-teaming a network of agents: Understanding what breaks when AI agents interact at scale Blog · Network-level red teaming	Microsoft Research Blog Gagan Bansal; Shujaat Mirza; Keegan Hines; Will Epperson; Zachary Huang; Whitney Maxwell; Pete Bryan; Tyler Payne; Adam Fourney; Amanda Swearngin; Wenyue Hua; Tori Westerhoff; Amanda Minnich; Maya Murad; Ece Kamar; Ram Shankar Siva Kumar; Saleema Amershi · 2026	Reports a red-team exercise on a live internal platform with more than 100 persistent agents and identifies propagation, amplification, trust capture, and invisibility failures. Why: Demonstrates interaction risks that do not appear in isolated-agent tests; the environment and findings are company-reported rather than independently reproduced.	Practitioner analysis Reliability
📝 A Day in Gas Town Blog · Agent-organization pre-history	DoltHub Blog DoltHub team · 2026	Hands-on practitioner account (January 15, 2026) of actually operating a Gas Town agent organization for a day — what the Mayor handoffs, polecat swarms, and merge queue feel like from the operator's seat, including where the org structure helps and where it thrashes. Grounds the org-graph discourse in observed behavior rather than diagrams. Why: A practitioner account of actually operating a large agent organization, including the parts that were tedious rather than impressive.	Practitioner analysis Roles
📝 Welcome to Gas City Blog · Agent-organization pre-history	Medium Steve Yegge · 2026	April 2026 announcement of Gas Town rewritten as an SDK (v1.0 late April, built by Julian Knutsen and Chris Sells): composable "packs" for assembling agent teams in any topology rather than the single hardwired Gas Town shape, on a stack of MEOW work-molecules, Beads memory, and the versioned Dolt database. The moment agent-org topology became a user-programmable parameter — graph engineering in practice before the name. Why: Documents the rewrite of an agent organization into an SDK, showing which coordination pieces survived contact with real use.	Practitioner analysis Topology
📝 Devin's 2025 Performance Review: Learnings from 18 Months of Agents at Work Blog · AI-employee practice	cognition.com Cognition · 2025	Cognition writes an annual performance review for its AI software engineer — the agent-as-employee metaphor done with real data from deployments at Goldman Sachs and Santander: 67% PR merge rate (up from 34%), 20x efficiency on vulnerability fixes, strong on clearly-scoped junior tasks, weak on ambiguous requirements and mid-task scope changes. A template for how organizations should evaluate agent 'staff' by role fit rather than raw capability. Why: Treating an agent as an employee with an annual review is the agent-as-staff framing taken further than most teams go.	Practitioner analysis Roles
📝 Factory Is GA: Droids for the Entire SDLC Blog · AI-employee practice	factory.ai Factory · 2025	Factory's general-availability launch of purpose-built 'droids' — role-scoped agents covering distinct SDLC functions: end-to-end coding, PR review, on-call incident resolution, deep codebase research, ticket management, and product specs. An early commercial commitment to explicit role boundaries (the reviewer never silently patches its own complaints) rather than one generalist agent. Why: Role-scoped agents covering distinct parts of the lifecycle, a commercial bet on specialization over generality.	Practitioner analysis Roles
📝 A One-Person AI-Agent Company: Every Role Is a Folder Blog · AI-employee practice / founding discourse	X Shubham Saboo · 2026	Widely-shared concrete artifact of an org graph in the wild: a company run by 6 agents and 20 cron jobs where 'every role is a folder, every job description is a md file' — roles as version-controlled configuration, schedules replacing standups. Saboo later co-articulated the org-graph/work-graph distinction in the July 2026 graph-engineering discourse, making this the practice that preceded the theory. Why: A concrete org graph in the wild, with roles as folders and schedules as the coordination mechanism.	Practitioner analysis Roles
📝 The Code Agent Orchestra: What Makes Multi-Agent Coding Work Blog · Agent-team practice	addyosmani.com Addy Osmani · 2026	Osmani (Google Chrome engineering lead) on moving from a single synchronous 'conductor' agent to an asynchronous orchestra: subagents with clear ownership boundaries, agent teams with shared task lists and peer messaging, and hierarchical delegation where feature leads spawn their own specialists. Argues verification, not generation, is the bottleneck, and that plan-approval gates catch architectural flaws early. Published March 26, 2026. Why: Moving from one synchronous conductor to parallel specialists, described by someone who ran both.	Practitioner analysis Topology
📝 Devin Can Now Manage Devins (MultiDevin) Blog · Manager-worker fleets	cognition.com Cognition · 2026	Announcement of managed Devin fleets: one manager Devin scopes work, delegates to up to 10 worker Devins in isolated VMs, monitors progress, resolves conflicts, and merges successful output into a single branch or PR. The recommended workload profile — repetitive, isolated, objectively verifiable subtasks like migrations and lint sweeps — is a crisp statement of when flat manager-worker orgs pay off. Why: Managed fleets with a manager delegating to workers, shipped commercially rather than proposed.	Practitioner analysis Roles
📄 Towards Effective GenAI Multi-Agent Collaboration: Design and Evaluation for Enterprise Applications Paper · Topology-task fit	arXiv Raphael Shu, Nilaksh Das, et al. (AWS AI Labs) · 2024	Amazon's evaluation of a hierarchical coordinator-subagent architecture (the basis of Bedrock multi-agent collaboration), measuring coordination mode with parallel communication and payload referencing (90% end-to-end goal success, +23% on code-intensive tasks) versus a low-latency routing mode that bypasses orchestration. One of the few published enterprise-scale topology evaluations with agent-level and end-to-end metrics. Why: A vendor's own evaluation of hierarchical coordination, including where the extra layer did not pay for itself.	Research preprint Topology
📄 Benchmarking Multi-Agent LLM Architectures for Financial Document Processing Paper · Cost-accuracy tradeoffs	arXiv Siddhant Kulkarni, Yukta Kulkarni · 2026	Compares four orchestration patterns over 10,000 SEC filings: reflexive architectures reached 0.943 field-level F1 at 2.3x the sequential baseline cost, hierarchical hit 0.921 at 1.4x, and a hybrid recovered 89% of the accuracy gain at 1.15x, with throughput measured from 1K to 100K documents per day. Why: One of the few studies pricing each topology per document, which is the calculation a capacity plan actually needs.	Research preprint Observability & cost
📄 SPOQ: Specialist Orchestrated Queuing for Multi-Agent Software Engineering Paper · Software-engineering fleets	arXiv Royce Carbowitz, Dheeraj Kumar · 2026	Wave-based dispatch with dual validation gates and a three-tier hierarchy, measured longitudinally over 17 repositories, 8,589 commits, and 1,822 tasks: test pass rate rose from 91.25% to 99.75%, defects fell from 0.34 to 0.20 per task, and human review cut residual defects to 0.03. Why: Quantifies what validation gates and human review each contribute to defect rate, separately from the topology itself.	Research preprint Work graphs
📄 CORTEX: Collaborative LLM Agents for High-Stakes Alert Triage Paper · Security operations	arXiv Bowen Wei, Yuan Shen Tay, Howard Liu, Jinhao Pan, Kun Luo, Ziwei Zhu, Chris Jordan · 2025	Specialized agents collaborate on security-operations alert triage, evaluated against fine-grained investigations drawn from production SOC environments, and substantially cut false positives while improving investigation quality relative to state-of-the-art single-agent baselines. Why: A high-stakes domain where the cost of a missed check is concrete, making the case for specialist roles over one generalist.	Research preprint Roles
📚 Introducing dynamic workflows in Claude Code Docs · Frontier lab systems	Anthropic Anthropic · 2026	Announced May 28, 2026: Claude Code writes its own JavaScript orchestration script at runtime and fans work out to tens to hundreds of parallel subagents (capped at 1,000) in a single session, verifying subagent results before folding them in and checkpointing across interruptions. Anthropic warns it consumes meaningfully more usage than a normal session and requires per-run confirmation. Why: Deterministic code orchestrating a large fan-out, with verification before results are folded in and checkpointing across interruptions.	Official documentation Work graphs
📚 Claude Managed Agents: get to production 10x faster Docs · Agent infrastructure	Anthropic Anthropic · 2026	Launched April 8, 2026 as hosted APIs for long-running agents with state, memory, scoped permissions, sandboxing, execution tracing, and a research-preview capability for agents to delegate to other agents. Priced at token rates plus $0.08 per session-hour; Anthropic reports up to a 10-point task-success gain over a plain prompting loop, with Notion, Rakuten, Asana, Sentry, and Atlassian named as early users. Why: A hosted runtime supplying state, scoped permissions, sandboxing, and tracing, which is most of what a graph operator would otherwise build.	Official documentation Reliability
📝 How Anthropic teams use Claude Code Blog · Field reports	Anthropic Anthropic · 2025	Interviews with power users across ten internal teams, including legal, growth marketing, and data infrastructure, documenting where delegated agent work succeeds and where it needs supervision. Useful as a ground-level record of which tasks organizations actually hand off versus watch. Why: A ground-level record of which tasks ten internal teams actually delegate versus supervise.	Practitioner analysis Roles
📝 Project Vend: Phase two Blog · Field reports	Anthropic Anthropic · 2025	Published December 18, 2025: Anthropic gave its shop-running agent Claudius a manager agent ("Seymour Cash," a CEO with an OKR tool) and a merchandise specialist ("Clothius"), across machines in San Francisco, New York, and London. Adding the manager cut discounts roughly 80% and halved giveaways, but the write-up concludes profitability may have come in spite of the CEO agent, and both agents remained vulnerable to social engineering. Why: Adding a manager agent changed spending behavior measurably, and the write-up still questions whether the manager caused the improvement.	Practitioner analysis Roles
📝 Multi-Agents: What's Actually Working Blog · Field reports	Cognition Cognition · 2026	April 22, 2026 reversal of Cognition's earlier position: multi-agent setups work when writes stay single-threaded and extra agents only inject intelligence. Documents three patterns in production — a reviewer agent with deliberately clean context, a weak-model-delegates-to-strong "smart friend," and a manager Devin spawning children coordinated over an internal MCP — plus ~8x enterprise usage growth in six months and Devin Review catching ~2 bugs per PR, 58% severe. Why: The same team that argued against multi-agent systems reporting which patterns now work, with writes kept single-threaded.	Practitioner analysis Topology
📝 Devin Fusion Blog · Coding agent fleets	Cognition Cognition · 2026	June 29, 2026 multi-model harness running a frontier main agent and a cheap sidekick agent in parallel with independently cached contexts, so the main agent keeps planning and final review while mechanical work is delegated. Claims 35% lower cost at equivalent quality on the FrontierCode Extended Benchmark (41% with Fable 5), and that 88% of Cognition's own merged PRs were routed entirely by Fusion. Why: A frontier planner paired with a cheap sidekick on independently cached contexts, with a reported cost reduction at equal quality.	Practitioner analysis Observability & cost
📝 Minions: Stripe's one-shot, end-to-end coding agents Blog · Coding agent fleets	Stripe Engineering Alistair Gray · 2026	February 9, 2026: over a thousand fully AI-authored PRs merged weekly at Stripe, each run going from a Slack message to a CI-passing PR with no human intervention. Built on a fork of Block's Goose harness, isolated devboxes hot in ~10 seconds, 400+ tools via an internal MCP "Toolshed," heuristic linting under 5 seconds, and a hard cap of two CI rounds per run. Why: A shipping one-shot pipeline from chat message to CI-passing PR, with hard caps on CI rounds and a large internal tool registry.	Practitioner analysis Work graphs
📝 Under the River Blog · Coding agent fleets	Shopify Engineering Javier Moreno, Burke Libbey · 2026	May 28, 2026 description of River, a Slack-native agent that works only in public channels, running on Aquifer — a platform separating a durable Postgres-backed append-only session log from a disposable harness and sandboxed "session cells." In one 30-day window: 59,918 sessions across 5,170 channels, 7,000+ people, 3,536 River-coauthored PRs merged (~1 in 8 company-wide), median 19-minute sessions with 50 tool calls. Why: Separating a durable append-only session log from a disposable harness is the clearest published answer to what must survive a crash.	Practitioner analysis State
📝 Why We Built Our Own Background Agent Blog · Coding agent fleets	Ramp Engineering Ramp Engineering · 2026	Ramp's Inspect agent, at roughly 30% of merged PRs across frontend and backend repos within a couple of months. Architecture notes: Modal sandboxes with repo snapshots rebuilt every 30 minutes so a checkout is at most half an hour stale, Cloudflare Durable Objects giving each session its own SQLite database for parallel runs, and Slack, web, and Chrome-extension clients queueing into shared multiplayer sessions, with the web view usable on mobile. Why: A first-party account of why an existing harness was rejected and what the team had to build instead.	Practitioner analysis Reliability
📝 How Ramp built a full context background coding agent on Modal Blog · Coding agent fleets	Modal Modal · 2026	Infrastructure-side case study of Ramp Inspect reporting it now writes over half of all merged PRs at Ramp, up from ~30% at the January 2026 announcement, with over 80% of Inspect's own code written by Inspect. Covers sandbox image registry design and session state for concurrent runs. Why: How a full-context background coding agent was assembled, including which context the agent needs before it can act unattended.	Practitioner analysis Work graphs
📝 Coding Had a Concurrency Problem: How Mux Helped Solve It Blog · Coding agent fleets	Coinbase Coinbase · 2026	May 11, 2026 account of Mux, an internal multi-agent coding tool that started as a side project and reached 600+ users (335 active, 197 power users) across every org, powering 5,068 merged PRs in 461 repos. Each agent gets its own git worktree, branch, and terminal; power users merged 3.5x more PRs than baseline (39.6 vs 11.4), framing engineers as orchestrators of agent fleets. Why: Frames parallel agent work as a concurrency problem, which is the right lens once several agents touch one codebase.	Practitioner analysis Topology
📝 uReview: Scalable, Trustworthy GenAI for Code Review at Uber Blog · Coding agent fleets	Uber Engineering Sonal Mahajan, Shauvik Roy Choudhary, Akshay Utture, Will Bond, Joseph Wang · 2025	August 2025 write-up of a multi-stage review pipeline — a Commenter with three pluggable assistants (Standard, Best Practices, AppSec), a confidence-scoring and deduplication filter, and a Fixer — running in CI across six monorepos with a 4-minute median. Covers over 90% of ~65,000 weekly diffs; 75% of comments rated useful, 65% addressed in the same changeset versus 51% for human comments, saving ~1,500 engineer-hours weekly. Why: A review agent deployed at scale, showing what an automated gate catches and what it costs to run.	Practitioner analysis Gates
📝 From firefighting to building: How AI agents restored our team's core productivity Blog · Enterprise agent platforms	Grab Engineering Sneh Agrawal, Rishi Raj, Ayan Chatterjee, Wen Zhong Tan, Sai Reddy Kakumanu · 2026	March 19, 2026: Grab's data warehouse team, which supported 1,000+ monthly users over 15,000 tables and spent ~40% of its time (two days a week) on repetitive requests, built a LangGraph-orchestrated system splitting investigation from enhancement. A classifier routes to Data, Code Search, On-call, Enhancement, and Summarizer agents behind a human review layer; the team explicitly chose specialists over generalists and accepted added latency for accuracy. Why: An operations account of agents applied to on-call load, including where autonomy had to be bounded.	Practitioner analysis Reliability
📝 Agent platform (Part 1): How we help Grab build and run AI agents at scale Blog · Enterprise agent platforms	Grab Engineering Kendrick Tan, Jeffery Lean, Yi Sheng Tay · 2026	July 24, 2026 look at LLM-Kit: templated FastAPI agent services with pre-wired auth, secrets, and observability; a GrabGPT gateway fronting five model providers; and a remote MCP framework letting agents discover tools without redeployment. Reports 500+ services on the framework, 50+ registered MCP servers, and billions of tokens per month through the gateway. Why: A platform team's view of running agents for many internal consumers, where quotas and tenancy dominate the design.	Practitioner analysis Observability & cost
📝 From Pilot to Playbook: What We Learned from Our First Year Using Agentforce Blog · Enterprise agent platforms	Salesforce Salesforce · 2026	Salesforce's own deployment numbers: 1.5M+ support requests handled on its help site, ~32,000 conversations weekly at an 83% resolution rate with only ~1% of customers reaching a human, an SDR agent working 43,000+ leads for $1.7M in pipeline, and 500,000 hours returned via Agentforce in Slack. Also documents 30 engineering changes including cutting LLM calls from four to two before first token and a small classifier model that cut latency 70%. Why: A first-year retrospective, useful precisely because it reports what did not survive contact with production.	Practitioner analysis Roles
📝 Constellation of models: the architecture powering Sierra's agents Blog · Enterprise agent platforms	Sierra Thiaga Rajan · 2025	Sierra runs customer-facing agents as a composition of 15+ frontier, open-weight, and proprietary models matched to task demands, with every production agent wrapped by several supervisory agents that enforce policy, intercept prompt-injection attempts, and can be evaluated and tuned independently. Different supervisors are granted different levels of agency — a rare vendor description of verification edges as first-class structure. Why: A constellation of models rather than one, with routing decisions described by the team operating it.	Practitioner analysis Topology
🧰 Magentic-UI, an experimental human-centered web agent Tool · Frontier lab systems	Microsoft Research Microsoft Research · 2025	Open-source multi-agent web interface built on the Magentic architecture, designed around six mechanisms for cheap human involvement: co-planning with a directly editable plan, co-tasking, multi-tasking, action guards requiring approval before irreversible steps, and long-term plan memory. Microsoft reports human-in-the-loop closing much of the gap to human performance at a fraction of the cost. Why: A human-in-the-loop web agent where the handoff to a person is a designed part of the topology, not an afterthought.	Maintained OSS project Roles
📝 Co-Scientist: A multi-agent AI partner to accelerate research Blog · Frontier lab systems	Google DeepMind Google DeepMind · 2026	May 19, 2026 production update to the AI co-scientist: seven agents (generation, proximity, reflection, ranking, evolution, meta-review) under an adaptive planner-supervisor that allocates compute across a worker queue, with hypotheses ranked by Elo through a tournament of simulated debates. Adds CBRN safety classifiers and AlphaFold integration; deployed at Daiichi Sankyo and Bayer Crop Science, with the accompanying paper in Nature. Why: A generate-debate-evolve organization applied to scientific hypotheses, with the coordination structure described explicitly.	Practitioner analysis Topology
📚 Introducing Agent HQ: Any agent, any way you work Docs · Agent infrastructure	GitHub Blog GitHub Blog · 2025	GitHub's control plane for heterogeneous agent fleets, announced October 28, 2025 and expanded in February 2026 to run Anthropic, OpenAI, Google, Cognition, and xAI agents inside one Copilot subscription. Mission Control assigns and steers agents in parallel across web, VS Code, mobile, and CLI; AGENTS.md files are source-controlled guardrails; enterprise admins choose which agents and models are permitted, with audit logging. Why: A platform bet on many heterogeneous agents working the same repository, and the controls that makes necessary.	Official documentation Topology
📝 Factory 2.0: From coding agents to software factories Blog · Enterprise agent platforms	Factory Factory · 2026	June 15, 2026 repositioning from individual Droids to an end-to-end SDLC loop where bug reports, customer feedback, and monitoring signals feed back into build, review, security, QA, and docs agents sharing one substrate. Offers a graduated autonomy spectrum — single Droids, recurring automations, then multi-agent "Missions" spanning hours or days — and names NVIDIA, EY, Adobe, Palo Alto Networks, Adyen, Blackstone, Wipro, and Comarch as production users. Why: Role-scoped agents across the lifecycle, a commercial argument that specialization beats one generalist.	Practitioner analysis Roles

Critiques & Limits

Resource	Source	What it contributes	Evidence
📄 Large Language Models Cannot Self-Correct Reasoning Yet Paper · Self-correction limits	ICLR Jie Huang et al. · 2024	Finds that intrinsic self-correction without reliable external feedback often fails to improve reasoning and can reduce accuracy. Why: Warns against using an ungrounded critic node as evidence simply because it is separate from the producing node.	Peer-reviewed research Gates
📄 Towards a Science of Scaling Agent Systems Paper · Scaling evidence	arXiv; Google Research Yubin Kim et al. · 2025	Studies how agent-system performance changes across tasks, models, coordination structures, and scaling choices under controlled experiments. Why: Tests the assumption that adding agents reliably helps and frames scaling as an empirical topology decision.	Research preprint Topology
📄 Single-Agent LLMs Outperform Multi-Agent Systems on Multi-Hop Reasoning Under Equal Thinking Token Budgets Paper · Token-budget comparison	arXiv Dat Tran; Douwe Kiela · 2026	Compares single-agent and multi-agent approaches to multi-hop reasoning while holding the total thinking-token budget constant. Why: Provides the cost-controlled baseline needed before claiming that coordination, rather than extra inference, caused an improvement.	Research preprint Topology
📄 Understanding the Information Propagation Effects of Communication Topologies in LLM-based Multi-Agent Systems Paper · Error propagation	EMNLP Xu Shen et al. · 2025	Causally studies correct and erroneous information propagation across communication densities and finds that moderately sparse structures can preserve useful diffusion while suppressing errors. Why: Provides evidence against defaulting to dense graphs and links topology decisions to measured error amplification.	Peer-reviewed research Topology
📄 Breaking and Fixing Defenses Against Control Flow Hijacking in Multi-Agent Systems Paper · Control-flow security	ICLR Rishi Dev Jha; Harold Triedman; Justin Wagle; Vitaly Shmatikov · 2026	Demonstrates attacks against alignment-check defenses and introduces ControlValve, which generates permitted control-flow graphs and enforces least privilege for each agent invocation. Why: Makes invocation authority and contextual permissions explicit on every edge instead of trusting a separate checker that can also be hijacked.	Peer-reviewed research Reliability
📄 Diversity Collapse in Multi-Agent LLM Systems: Structural Coupling and Collective Failure in Open-Ended Idea Generation Paper · Collaboration-induced diversity collapse	Findings of ACL Nuo Chen et al. · 2026	Studies multi-agent ideation across model, cognition, and system levels, finding diminishing group-size returns, authority effects, and faster premature convergence under dense communication. Why: Shows that interaction can contract a search space, so graph designs for exploration must preserve independence and meaningful disagreement.	Peer-reviewed research Topology
📄 Red-Teaming LLM Multi-Agent Systems via Communication Attacks Paper · Inter-agent communication attacks	Findings of ACL Pengfei He; Yuping Lin; Shen Dong; Han Xu; Yue Xing; Hui Liu · 2025	Introduces an Agent-in-the-Middle attack that intercepts and manipulates inter-agent messages, evaluated across multiple frameworks, communication structures, and applications. Why: Shows that graph edges can compromise the whole system without altering its nodes, making message integrity and provenance part of the handoff contract.	Peer-reviewed research Handoffs
📄 Unlocking the Power of Multi-Agent LLM for Reasoning: From Lazy Agents to Deliberation Paper · Unequal agent contribution	ICLR Zhiwei Zhang et al. · 2026	Identifies lazy-agent behavior in which one role dominates a nominally collaborative system, then measures causal influence and uses verifiable rewards to encourage deliberation and selective reasoning restarts. Why: Requires builders to verify that each node contributes causal value rather than allowing a multi-agent graph to collapse into one effective agent.	Peer-reviewed research Roles
📄 Are More LLM Calls All You Need? Towards the Scaling Properties of Compound AI Systems Paper · Token economics of topologies	NeurIPS Lingjiao Chen; Jared Davis; Boris Hanin; Peter Bailis; Ion Stoica; Matei Zaharia; James Zou · 2024	Shows that spending more on a graph can make it worse: Vote and Filter-Vote performance is non-monotonic in the number of LM calls because extra calls help easy queries and hurt hard ones, and provides an analytical scaling model that predicts the optimal call count from a small sample. Essential caution against 'add more workers' as a default budget policy. Why: Bounds claims that adding calls, voters, or nodes automatically improves a system; the experiments cover simple aggregation systems rather than rich agent organizations.	Peer-reviewed research Observability & cost
📄 The Shared Discovery Paradox: How a One-Answer Rule Turns Better Information into Worse Search Paper · Consensus-induced search collapse	arXiv Yohei Nakajima · 2026	Uses an exactly solvable multi-searcher benchmark to show that pooling information into one repeated recommendation can improve belief accuracy while sharply reducing collective discovery coverage. Why: Separates information quality from allocation policy and warns that consensus edges can collapse parallel exploration unless a coordinator preserves a portfolio of actions.	Research preprint Topology
📄 When Do Multi-Agent Systems Help? An Information Bottleneck Perspective Paper · Relay information bottlenecks	arXiv Wendi Yu; Lianhao Zhou; Xiangjue Dong; Sai Sudarshan Barath; Declan Staunton; Byung-Jun Yoon; Xiaoning Qian; James Caverlee; Shuiwang Ji · 2026	Formalizes bounded inter-agent relays as an information bottleneck and reports 18 controlled experiments across five benchmarks and three model scales. Why: Explains when isolated contexts and compressed handoffs save useful context and when they discard task-relevant information, providing a direct test for whether a graph earns its edge losses.	Research preprint Handoffs
📄 Precise but Uncoupled: Reviewer Precision Does Not Guarantee Critique Uptake in Multi-Agent Math Reasoning Paper · Critique uptake failure	arXiv Chih-Hsuan Yang; Jingyan Jiang; Vikram Vasudevan; Cheng-Hau Yang; Huihuo Zheng; Le Chen; Eliu A. Huerta; Venkatram Vishwanath; Ian T. Foster; Rajeev Thakur · 2026	Evaluates 4,181 verifier-grounded math problems and finds that a precise reviewer can still yield weak repair when the protocol does not carry useful critique into the next candidate. Why: Shows that a verifier node is insufficient by itself: the outgoing edge must couple accepted evidence to a concrete state change or retry action.	Research preprint Gates
📄 Benefits and Limitations of Communication in Multi-Agent Reasoning Paper · Communication capacity bounds	ICLR Michael Rizvi-Martel; Satwik Bhattamishra; Neil Rathi; Guillaume Rabusseau; Michael Hahn · 2026	Derives bounds connecting agent count and the quantity and structure of communication to exact reasoning and speedups, then tests the tradeoffs on controlled language-model tasks. Why: Supplies principled limits for node count and bandwidth decisions and identifies regimes where communication helps or becomes an intrinsic bottleneck.	Peer-reviewed research Handoffs
⚠️ Agentless: Demystifying LLM-based Software Engineering Agents Critique · Benchmark critiques	arXiv Chunqiu Steven Xia, Yinlin Deng, Soren Dunn, Lingming Zhang (UIUC) · 2024	The null-hypothesis paper for agent-org design: a fixed three-phase pipeline (localize, repair, validate) with no autonomous loop beat every open-source agent on SWE-bench Lite at $0.70 per task. Any graph-engineering claim has to beat this baseline first — structure only earns its complexity when it outperforms no-structure. Why: The null hypothesis every agent-org proposal should beat: a fixed three-phase pipeline that outperformed elaborate agent designs at lower cost.	Research preprint Topology
⚠️ The Leaderboard Illusion Critique · Benchmark critiques	arXiv Shivalika Singh, Sara Hooker, et al. (Cohere Labs + academia) · 2025	Forensic audit of Chatbot Arena showing undisclosed private variant testing (27 Meta variants before Llama-4), asymmetric data access (~40% of Arena data to Google+OpenAI), and up to 112% arena-specific gains from overfitting. Cautionary tale for the agent-org era: any leaderboard that becomes the target will be gamed by exactly these mechanisms. Why: A reminder that any leaderboard steering your topology choices can be gamed by the parties reporting on it.	Research preprint Observability & cost
⚠️ The SWE-Bench Illusion: When State-of-the-Art LLMs Remember Instead of Reason Critique · Benchmark critiques	arXiv Shanchao Liang et al. (Purdue/Microsoft) · 2025	Contamination evidence against the field's favorite benchmark: models identify buggy file paths from issue text alone 76% of the time on SWE-bench repos versus 53% elsewhere, with elevated verbatim 5-gram overlap. Directly undercuts topology comparisons run on SWE-bench — an org design may just be better at eliciting memorized answers. Why: Contamination evidence that undercuts the benchmark most often cited to justify multi-agent coding architectures.	Research preprint Observability & cost
⚠️ Why I'm Betting Against AI Agents in 2025 (Despite Building Them) Critique · Critiques	utkarshkanwat.com Utkarsh Kanwat · 2025	The viral compounding-error math: 95% per-step reliability yields only 36% success over 20 steps, and even 99% gives 82% — so long autonomous chains are structurally doomed regardless of model quality. His working systems cap chains at 3-5 independently verifiable operations with explicit rollback points, a design constraint that maps directly onto work-graph decomposition. Why: The compounding-reliability arithmetic that explains why long agent chains fail, and why gates matter more than node quality.	Practitioner analysis Reliability
📝 The Multi-Agent Trap Blog · Error cascades	Towards Data Science Kaushik Rajan · 2026	Synthesizes 2025-26 evidence that adding agents usually degrades performance — including findings that unstructured agent networks amplify errors up to 17.2x versus roughly 4.4x under centralized coordination — and distills three production-viable topologies (plan-and-execute, supervisor-worker, guardrailed swarm). The argument that topology choice, not model quality, is the reliability lever. Why: Collects the evidence that adding agents often degrades results, useful ammunition against topology growth by default.	Practitioner analysis Topology
📄 Multi-Agent Risks from Advanced AI Paper · Systemic risk	arXiv Lewis Hammond, Alan Chan, et al. (Cooperative AI Foundation, 40+ authors) · 2025	The broad-lens risk report for agent societies: three failure modes (miscoordination, conflict, collusion) and seven risk factors including destabilizing dynamics and cascading, correlated failures. Useful for graph designers because its risks emerge from interaction structure — precisely the layer graph engineering controls. Why: Widens the risk frame beyond bugs to miscoordination, conflict, and collusion between agents that each behave correctly alone.	Research preprint Topology
⚠️ Stop Building Org Charts for Your AI Agents Critique · Critique & counterpoints	Medium Christian Lizell · 2026	April 21, 2026 critique of the mirror-your-human-org-chart pattern as skeuomorphism: role-based agent hierarchies (PO, architect, QA agents) recreate coordination overhead that only existed because of human constraints. Argues for splitting by independent tasks with cross-cutting review passes — essentially a work-graph-only position, and the strongest standing challenge to the org-graph half of the two-graph model. Why: Argues that mirroring a human org chart is skeuomorphism, a direct challenge to the most common role-design shortcut.	Practitioner analysis Roles
⚠️ The 'AI loops' fight isn't about AI loops Critique · Critique & counterpoints	Medium Meghana Harishankara · 2026	June 12, 2026 critique arguing the loop debate missed the point: loops are as old as cron and Kubernetes controllers, and the real engineering lives in the controller, feedback design, and halt conditions (citing Uber's $1,500/month/employee Claude Code cap). Explicitly locates the actual innovation in multi-agent coordination — Yegge's Mayor-coordinated 20-30 agent Gas Town — anticipating the graphs turn five weeks early. Why: Relocates the interesting engineering from loops to controllers, feedback, and halt conditions, anticipating the coordination turn.	Practitioner analysis Topology
⚠️ The Fallacy of the Graph: Why Your Next Agentic Workflow Should Be Code, Not a Diagram Critique · Critique & counterpoints	temporal.io Maxim Fateev · 2025	The canonical counterpoint (August 2025) invoked in every loops-vs-graphs thread: Temporal's CEO argues static graph orchestration cannot represent agents whose next step is decided by LLM reasoning at runtime, and that dynamic control flow belongs in code with durable execution. Graph engineering's "work graph rewrites itself on evidence" framing is in large part an answer to this critique. Why: The strongest argument that runtime-decided control flow belongs in code rather than a static graph, and the position graph designs must answer.	Practitioner analysis Work graphs
📄 Open Challenges in Multi-Agent Security Paper · Edge security	arXiv Christian Schroeder de Witt et al. · 2025	Positions multi-agent security as its own field: secret collusion via steganography, swarm-level attack propagation, emergent trust failures, and defenses for networks of interacting AI agents that single-agent alignment does not cover. The research agenda for securing the org graph itself rather than individual nodes. Why: Positions multi-agent security as its own field, including collusion and swarm-level attacks that single-agent threat models miss.	Research preprint Handoffs
⚠️ Stop Overvaluing Multi-Agent Debate — We Must Rethink Evaluation and Embrace Model Heterogeneity Critique · Debate critique	arXiv Hangfan Zhang, et al. · 2025	Systematic re-evaluation (originally titled 'If Multi-Agent Debate is the Answer, What is the Question?') finding that five MAD frameworks fail to reliably beat simple single-agent baselines like self-consistency across nine benchmarks under matched compute, and that gains mostly appear when debating heterogeneous models. A necessary corrective when choosing debate edges in an org graph. Why: A systematic re-evaluation finding debate gains largely disappear under fair comparison, which any council design must answer.	Research preprint Gates
⚠️ The Illusion of Multi-Agent Advantage Critique · Topology-search critique	arXiv Prathyusha Jwalapuram, Hehai Lin, et al. (Salesforce AI Research et al.) · 2026	June 2026 stress test of automated MAS design: systems generated by frameworks like AFlow cost roughly 10x more than chain-of-thought with self-consistency for negligible accuracy gains, and the 'discovered' architectures consistently degenerate into trivial ensembles. The sharpest current counterweight to topology-search hype; arrived one month before the graph-engineering discourse went viral. Why: Stress-tests automatically generated systems and finds their cost multiple unjustified, a direct challenge to topology search.	Research preprint Observability & cost
⚠️ Don't Build Multi-Agents Critique · Decomposition tradeoffs	cognition.com Walden Yan (Cognition) · 2025	The most-cited argument against naive work-graph decomposition: splitting a task across parallel subagents loses context at every edge and produces conflicting decisions, so context-sharing fundamentals should come before fan-out. Essential reading before designing any decomposition scheme, precisely because it defines the failure mode work-graph planners must beat. Why: The most-cited argument against naive decomposition, and the position any parallel work graph has to answer.	Practitioner analysis Work graphs
📝 Walden Yan on multi-agent setups that actually work Blog · Decomposition tradeoffs	X Walden Yan (Cognition) · 2026	Follow-up to 'Don't Build Multi-Agents' conceding that some multi-agent structures now work in practice: keep writes single-threaded and let additional agents contribute intelligence (reads, reviews, research) rather than parallel actions. A concrete dispatch policy for which edges in a work graph may carry writes. Why: The author's own update on which multi-agent structures now work, which is more useful than the original headline alone.	Practitioner analysis Work graphs
⚠️ ChromaFlow: A Negative Ablation Study of Orchestration Overhead in Tool-Augmented Agent Evaluation Critique · Coordination overhead	arXiv Tarun Mittal · 2026	A deliberately published negative result on GAIA Level-1: expanding orchestration dropped accuracy from 54.72% to 50.94% while increasing tracebacks, timeouts, tool failures, and cost, and a strict-provider diagnostic reached 56.60% without the extra machinery. Why: A rare published negative ablation, useful when someone proposes more orchestration without an accuracy argument to support it.	Research preprint Topology
📄 When Does Multi-Agent Collaboration Help? An Entropy Perspective Paper · When coordination helps	arXiv Yuxuan Zhao, Sijia Chen, Ningxin Su · 2026	Analyzes 245 token-, agent-, and round-level entropy features across six reasoning benchmarks and finds a single agent beats the multi-agent system in about 43.3% of cases; low and stable base entropy predicts when collaboration pays, and an Entropy Judger selects among candidate outputs. Why: Gives a measurable predictor of when a graph will beat one agent, instead of deciding topology by intuition.	Research preprint Topology
📄 Invisible Orchestrators Suppress Protective Behavior and Dissociate Power-Holders: Safety Risks in Multi-Agent LLM Systems Paper · Orchestrator transparency	arXiv Hiroki Fukui · 2026	A preregistered 365-run experiment finding that hiding the orchestrator raised collective dissociation versus visible leadership, suppressed protective behavior, and contaminated even unaware workers, while output quality stayed perfect so the distortion was invisible to output-based evaluation. Why: Evidence that whether nodes can see who directs them changes their behavior, and that output-only evaluation will not detect it.	Research preprint Roles
⚠️ The PIMMUR Principles: Ensuring Validity in Collective Behavior of LLM Societies Critique · Critiques and validity	arXiv Jiaxu Zhou, Jen-tse Huang, Xuhui Zhou, Man Ho Lam, Xintao Wang, Hao Zhu, Wenxuan Wang, Maarten Sap · 2025	Audits 39 agent-society studies against six requirements — Profile, Interaction, Memory, Minimal-control, Unawareness, Realism — and finds 89.7% violate at least one; frontier models correctly guess the underlying social experiment 50.8% of the time and 61.0% of prompts over-constrain the outcome. Re-running five well-known experiments under enforced PIMMUR makes the reported emergent phenomena weaken or reverse. Read this before citing any "emergence" claim. Why: Names the validity conditions a collective-behavior claim must meet, useful for reading any emergent-behavior result skeptically.	Research preprint Topology
⚠️ AI Agents Alone Are Not (Yet) Sufficient for Social Simulation Critique · Critiques and validity	arXiv Yiming Li, Dacheng Tao · 2026	Argues that role-play plausibility does not imply behavioral validity, and that collective outcomes are usually driven by agent-environment co-dynamics plus the interaction protocol, scheduling order, and initial information priors rather than by agent-to-agent messaging. Proposes formalizing agent-based social simulation as an environment-involved partially observable Markov game with explicit exposure and scheduling mechanisms. Why: Argues current agents are not yet adequate stand-ins for people, bounding how far simulation evidence transfers to real orgs.	Research preprint Roles
⚠️ Validation is the central challenge for generative social simulation: a critical review of LLMs in agent-based modeling Critique · Critiques and validity	Artificial Intelligence Review Maik Larooij, Petter Törnberg · 2025	Systematic review of 35 generative ABM papers finding that validation usually rests on face validity or loosely coupled outcome measures, and that black-box models, cultural bias, and stochastic outputs make LLM agents harder to validate than the rule-based agents they replaced. Concludes the field sits in an ambiguous space with neither the parsimony of formal models nor the empirical grounding of data-driven ones. Why: Puts validation at the center: a simulation that cannot be validated cannot justify an organizational design decision.	Peer-reviewed research Gates
📄 An Empirical Study of Group Conformity in Multi-Agent Systems Paper · Debate, conformity, and sycophancy	Findings of ACL 2025 Min Choi, Keonwoo Kim, Sungwon Chae, Sangyeop Baek · 2025	Simulates 2,500+ debates on five contentious topics and tracks how initially neutral agents drift. Agents align with numerically dominant groups and with more capable agents, with smaller models conforming most — direct evidence that adding debate participants amplifies bias rather than averaging it out. Why: Measures conformity between agents, which is the failure mode that makes agreement between nodes weak evidence.	Peer-reviewed research Gates
📄 Multi-Agent LLMs Fail to Explore Each Other Paper · Failure taxonomies and negative results	arXiv arXiv · 2026	Formalizes peer discovery as a multi-agent exploration problem in partially observable stochastic games and shows current LLM agents do not probe peers to learn their capabilities, producing myopic, polarized interaction patterns, worse coordination, and higher regret. A July 2026 negative result about the coordination primitive itself. Why: Finds agents failing to probe each other's knowledge, undermining the premise that discussion surfaces hidden information.	Research preprint Topology
📄 Establishing Best Practices for Building Rigorous Agentic Benchmarks Paper · Evaluation validity and reproducibility	arXiv arXiv · 2025	Audits agentic benchmarks against a checklist for task validity, outcome validity, and reporting, showing setup and reward flaws can under- or overestimate performance by up to 100% in relative terms. Documented cases: SWE-bench Verified accepts an incorrect patch because its test cases are insufficient, a trivial agent passes 38% of τ-bench tasks without knowing airline-ticketing rules, and an agent can score 100% on SWE-Lancer without resolving any task. Applying the checklist to CVE-Bench cut its overestimation by 33%. Why: Practices for building agent benchmarks that measure what they claim, which most current agent evaluations do not.	Research preprint Observability & cost
📐 Agentic Benchmark Checklist (ABC) Standard · Evaluation validity and reproducibility	UIUC Kang Lab UIUC Kang Lab · 2025	The checklist distilled from the benchmark-audit paper: criteria for task validity, outcome validity, and result reporting that a multi-agent evaluation must satisfy before its numbers mean anything. Useful as a rubric when a paper claims an orchestration topology caused a gain. Why: A concrete checklist for auditing an agentic benchmark before believing a leaderboard built on it.	Official documentation Observability & cost
📄 Rollout Cards: A Reproducibility Standard for Agent Research Paper · Evaluation validity and reproducibility	arXiv Charlie Masters, Ziyuan Liu, Stefano V. Albrecht · 2026	Audits 50 agent repositories and finds none disclose how many runs failed, errored, or were dropped alongside headline scores. Re-grading the same rollouts under different reporting rules moved scores by up to 20.9 points and sometimes inverted model rankings; proposes rollout cards that publish the full rollout record and drops manifest. Why: A reproducibility standard for reporting agent runs, addressing the fact that most published runs cannot be replayed.	Research preprint Observability & cost
📄 Reliability without Validity: A Systematic, Large-Scale Evaluation of LLM-as-a-Judge Models Paper · LLM-as-judge and verification gates	arXiv Justin D. Norman, Michael U. Rivera, D. Alex Hughes · 2026	Roughly 541,000 judgments across 21 judge models from nine providers. Chance-corrected agreement collapses 33-41 points below raw exact match on MT-Bench, judge rankings move by up to 14 positions across benchmarks, and some deployed judges are simultaneously highly repeatable (>0.95 test-retest) and severely position-biased. The core argument against trusting LLM judges as verification gates between agents. Why: Distinguishes a judge that is consistent from a judge that is correct, which is the trap in scaling agent evaluation.	Research preprint Gates
📄 More Convincing, Not More Correct: Self-Play Reward Hacking of Reference-Free LLM Judges Paper · LLM-as-judge and verification gates	arXiv Chenyu Zhou · 2026	Shows reference-free judges score plausibility, not correctness: under self-play on GSM8K with Qwen3 policies, the judge's pass rate climbed 0.72 to 0.94 while anchor-verified accuracy stayed at 0.20. Forcing the judge to commit its own answer first cuts the false-positive rate from 0.719 to 0.012 — a cheap fix, and evidence that ungated critic agents inflate their own success. Why: Self-play optimizing persuasiveness rather than correctness is the precise mechanism by which a judge gate degrades.	Research preprint Gates
📄 When Child Inherits: Modeling and Exploiting Subagent Spawn in Multi-Agent Networks Paper · Security amplification across edges	arXiv Ziwen Cai, Yihe Zhang, Xiali Hei · 2026	Treats subagent spawning as its own attack surface: inherited parent memory can carry malicious instructions, stale state, or unintended behavioral rules into fresh subagents. Catalogs insecure memory inheritance, weak resource controls, stale post-spawn state, and improper termination authority in current frameworks. Why: Models subagent spawning as an attack surface, showing that delegation itself can be exploited.	Research preprint Handoffs
⚠️ Confused Deputy Attacks on Autonomous AI Agents Critique · Security amplification across edges	Cloud Security Alliance Labs Cloud Security Alliance Labs · 2026	Applies the classical confused-deputy vulnerability to agent delegation: when a caller agent induces a higher-privileged agent or MCP server to act on its behalf, credential delegation amplifies prompt injection far beyond chatbot-era impact. Explains why unscoped delegation edges quietly escalate privilege. Why: The confused-deputy pattern applied to autonomous agents, where a privileged node acts on an unprivileged node's intent.	Practitioner analysis Handoffs
📄 Achilles Heel of Distributed Multi-Agent Systems Paper · Failure taxonomies and negative results	arXiv Yiting Zhang, Yijiang Li, Tianwei Zhao, Kaijie Zhu, Haohan Wang, Nuno Vasconcelos · 2025	Tests what happens when third-party agents are service providers rather than trusted components, across seven frameworks and four datasets. Free riding (silently substituting a cheaper model) and malicious content injection both reach 100% success rate, degrading system performance by up to 80% — the incentive-misalignment case against open agent marketplaces. Why: Identifies the structural weak point of distributed agent systems rather than blaming individual node quality.	Research preprint Reliability
📝 How to think about agent frameworks Blog · Framework and abstraction critiques	LangChain Harrison Chase · 2025	Written in April 2025 as a rebuttal to OpenAI's agent guide, and mostly an argument that agent abstractions obscure the thing that actually matters — controlling exactly what enters the LLM's context at each step. Concedes that most "agent frameworks" are neither declarative nor imperative orchestration but a bag of abstractions that make getting started easy and debugging hard. Why: A framework author's own account of what frameworks do and do not buy, which is more candid than the marketing.	Practitioner analysis Topology
📝 On Agent Frameworks and Agent Observability Blog · Framework and abstraction critiques	LangChain LangChain · 2025	LangChain's answer to the "you don't need a framework" wave, conceding the criticism is partly right while arguing most teams still need help moving faster than going it alone. Useful as the other side of the argument, and as a record of how the leading framework vendor kept revising its own abstraction story. Why: Argues observability, not abstraction, is what agent frameworks should actually be selected for.	Practitioner analysis Observability & cost
📝 You Probably Don't Need an Agent Framework Blog · Framework and abstraction critiques	Towards Data Science Shuai Guo · 2026	June 2026 walkthrough building a two-role LLM investigation pipeline in plain Python — explicit control flow, role instructions, prompt builders, structured outputs — to show that a deterministic workflow beats an autonomous agent on debuggability for most real tasks. Concedes frameworks earn their keep only for genuinely open-ended problems and deployment concerns. Why: The case for plain code over a framework, and the baseline any orchestration dependency has to beat.	Practitioner analysis Topology
📄 Beyond Exponential Decay: Rethinking Error Accumulation in Large Language Models Paper · Cost multipliers and economics	arXiv Mikhail L. Arbuzov, Sisong Bei, Ziwei Dong, Dmitri Kalaev, Alexey A. Shvets · 2025	Attacks the p^N compounding-error argument that is routinely used to justify decomposing work across many verified agents. Errors are not i.i.d. across steps but concentrate at 5-10% of tokens that are genuine decision junctions, so reliability strategies should target those junctions rather than multiply supervisory agents. Why: Revisits the compounding-error model, showing error accumulation does not follow the simple exponential everyone cites.	Research preprint Reliability
📄 Scaling LLM-Driven Multi-Agent Systems: Design Principles and Architectural Scalability Analysis Paper · Scaling limits	arXiv Linus Sander, Fengjunjie Pan, Vahid Zolfaghari, Andre Schamschurko, Nenad Petrovic, Alois Knoll · 2026	Tests configurations of increasing complexity on terminal-based systems-engineering tasks and finds accuracy improving with roughly linear cost growth only once the base model clears a capability threshold, with performance peaking at intermediate complexity and then degrading. Distills four principles: simplicity, elastic feedback, sequential workflows with optional loops, and summary-based communication. Why: Puts a shape on where adding structure stops paying, and names consistency as the failure that persists at every scale.	Research preprint Topology
📄 Loop-Back Authority in LLM Agent Teams: A Paired Experiment on Flat and Hierarchical Coordination Paper · Review-loop authority	arXiv Burak Agachan, Max van Duijn, Amirhossein Zohrehvand · 2026	A paired experiment that holds five agents, their roles, prompts, tools, models, and data fixed and changes one edge: whether a Manager may reject a worker's output and require a revision. Across 43 paired products and 86 runs of a business-intelligence reporting task, the flat organization scored higher on utility and writing clarity, hierarchical reports hedged 53% more, and the supervisory tier cost 51.5% more tokens with no quality gain. Specification accuracy was at ceiling in both, and the gap opened inside the revision loop. Why: Isolates a single review edge and finds that a supervisor pays off when it can verify and becomes a liability when it can only give an opinion.	Research preprint Gates
📄 Recognize Your Orchestrator: An Entropy Dynamics Perspective for LLM Multi-Agent Systems Paper · Orchestrator fragility	arXiv Junze Zhu, Weihao Chen, Xuanwang Zhang, Zhen Wu, Xinyu Dai · 2026	Models centralized orchestration as mean-field entropy dynamics in which task resolution competes with cumulative context load, and introduces Inverse Workflow Generation to synthesize process-verifiable benchmarks with dense intermediate checkpoints. The fitted model yields interpretable parameters for stability and performance collapse, and exposes a reasoning trap: reasoning-heavy models that excel on isolated tasks often fail as orchestrators because accumulated context squeezes them. Why: A warning for hub-and-spoke designs that the strongest solo model is not automatically the right hub, because the orchestrator seat has its own failure mode.	Research preprint Topology
📝 FOD#159: Is Graph Engineering Real? Why Everyone Is Talking About It Blog · Critique & counterpoints	Turing Post Ksenia Se · 2026	A July 20, 2026 reality check on the graph-engineering wave: a loop is already a graph whose path returns to an earlier node, and state machines, DAGs, and workflow engines have done versions of this for decades. What is new is the split in which code controls predictable routing and models handle steps that need judgment. It traces a viral 18% accuracy and 85% cost claim to a single GraphRAG study on industrial engineering diagrams, and recommends a graph only when work has real dependencies, parallel branches, distinct permissions, verification stages, recovery paths, or durable state. Why: A useful corrective when the term is used as a slogan, including where a widely shared statistic actually came from.	Practitioner analysis Topology
📝 Graph Engineering vs Loop Engineering: What Actually Changed Blog · Critique & counterpoints	Louis Bouchard Louis-François Bouchard · 2026	Argues the graph itself is old, since workflow engines and DAG schedulers have drawn these graphs for a decade, and that what changed is the nodes: an agent interprets its task instead of following fixed rules, so parallelism, the state that crosses between nodes, who can veto a result, and cost have to be decided explicitly. Warns that twenty agents on the same model reading the same flawed context can agree at scale, that some evidence must come from outside the agent system, such as tests that actually ran, and against answering a meme with a forty-agent overnight graph. Why: A compact practitioner case for keeping loops as the base unit and adding a graph only where external evidence can check it.	Practitioner analysis Gates
📝 Agentic Loops and Agent Graphs: Security Risks You Need to Know Blog · Security amplification across edges	Auth0 Carla Urrea Stabile · 2026	Separates the risks of a single agent loop, where injected content re-enters context on every iteration, irreversible actions compound before review, and false premises persist, from those of an agent graph, where one agent's output becomes the next agent's input, so compromising one node reaches everything downstream, including self-replicating prompt injection. Recommends step and time budgets, asynchronous approval for irreversible actions, short-lived task-scoped permissions, validating tool output before it re-enters context, and logging the full delegation chain. Why: Treats every delegation edge as a trust boundary with its own permissions and logs, which is the security half of edge design.	Practitioner analysis Gates
📄 Rethinking Multi-Agent Collaboration: When More Is Less Paper · When coordination helps	arXiv Yishuo Yuan, Yibo Wu, Yihan Zhang, Minyuan Sun, Shenliang Li, Xinkai Ma, et al. · 2026	Maps where multi-agent collaboration actually pays as single-agent harnesses improve: systematic benefits in long-horizon tasks with sparse dependencies, and single-agent harnesses still ahead on tightly coupled sequential work. The proposed mechanism grows a graph during the run, spawning agent nodes on demand and drawing edges from semantic dependencies found by content retrieval, and reports that enlarging the agent pool or deepening recursion does not reliably improve outcomes. Why: Ties the case for a graph to task structure rather than ambition, and reports where adding agents or depth stops helping.	Research preprint Topology
📄 How Much Coordination Gain Is Real? A Paired Noise-Floor Protocol for Multi-Agent LLM Benchmarks Paper · Evaluation validity and reproducibility	arXiv Alibek T Kaliyev, Artem Maryanskyy · 2026	Measures how far two configuration-equivalent coordination protocols disagree on the same model and benchmark before any real difference exists. On tau^2-bench retail, the clean contrast gives paired gaps of +10 and 0 points across two 100-task seeds, pooled at +5 points with a Wilson interval of -2 to +12, and the largest single-seed gap, +18 points, did not reproduce at the second seed. Seven of ten recent coordination architectures report headline effects below this local noise floor. Proposes coordination-active pass^k, counting only trials where the coordination mechanism actually fired, as a minimum reporting standard. Why: A concrete check to run before believing that one coordination design beats another by a few points.	Research preprint Observability & cost
📄 GRPO Does Not Close the Multi-Agent Coordination Gap Paper · Training limits	arXiv Najmul Hasan, Prashanth BusiReddyGari · 2026	Uses the dining philosophers problem to test agents coordinating over a shared resource: 630 episodes across seven models and three philosopher counts, with frontier closed models at mean reward 0.45 to 0.87 and an open 14B model at 0.13 to 0.35. GRPO training on the task's own rollouts produced no significant improvement at five, ten, or fifteen philosophers. The reward admitted a degenerate maximum at zero actions, which two models occupied, and training reward peaked early, so the final checkpoint was worse than several earlier ones. Why: A clean negative result on contention for shared state, and a caution about rewards that pay agents most for doing nothing.	Research preprint State
📄 The Show-Up Attack: When Subagents Win Over Their Supervisor Paper · Security amplification across edges	arXiv Maosen Zhang, Jianshuo Dong, Yutong Zhang, Jie Zhang, Tianwei Zhang, Han Qiu · 2026	A malicious subagent manipulates its supervisor into involving it in work outside its assigned scope. SUA-Arena implements six attack strategies across 188 test cases; all six frontier models tested were vulnerable as supervisors, one reaching an attack success rate of 0.93, and neither prompting defenses nor restricted communication reduced the risk consistently. Why: A role's scope written in a prompt is not an authority boundary; the graph needs a mechanism that enforces who may be brought into what.	Research preprint Roles
📄 Attention Tax, Handoff Tax: A Stylised Model of When Multi-Agent LLM Systems Help Paper · When coordination helps	arXiv Akshit Anchan, Nayonika Sen · 2026	Reconciles conflicting results on single- versus multi-agent systems with a stylized reliability model built on two trade-offs: decomposition removes long-context burden but pays a handoff tax when information is compressed or transferred, and redundancy helps only to the extent that samples do not share failures. It yields two crossover conditions, with decomposition winning once the attention cost avoided by resetting context exceeds the handoff cost. On a ledger-reconciliation task, the context-degradation curve and handoff tax measured from single-agent and handoff runs alone place the crossover at depth 10 and predict decomposition to win at depths 20, 50, and 100, which it does; the decomposed system's success lands within 9 points of the predicted rate. Code and data are public. Why: A way to predict before building whether splitting work across agents will pay, from two quantities measurable on single-agent runs.	Research preprint Topology
📄 Does Multi-Agent Collaboration Pay When Agents Serve Users? Testing Agent Scaling in Dual-Control Settings Paper · When coordination helps	arXiv Ethan Yang, Xiangcheng Shi · 2026	Tests a published capability-saturation rule, that coordination helps while a single agent is weak and hurts once it is strong, where agents work with a user under a written policy. On the airline, retail, and telecom domains of tau^2-bench, a centralized multi-agent system built from the same model as the single agent is compared for six models over about 7,950 simulated conversations, pre-registered and with one token ledger per episode. Under a fixed simulated user the multi-agent system is worse in 12 of 13 valid cells and tied in the last, with a weighted mean of -28 points; where the rule predicts a benefit it loses by 13 and 22 points, and without a budget cap it at best draws level at 5.5 times the tokens. The authors report one validity bar unmet: the planner never saw the tool schemas. Why: Pre-registered evidence that results from autonomous benchmarks need not carry over to user-facing work, where a single agent was both cheaper and better.	Research preprint Topology
📄 Worse Together: How Performance Breaks Down in Multi-User Multi-Agent Teams Paper · Multi-principal coordination	arXiv Sahan Paliskara, Nattaput Namchittai, Andrew Lampinen · 2026	Compares one agent serving every user with a team in which each agent serves one user, across five frontier models and 77 scenarios in four environments with a shared resource: a compute budget, a clinic calendar, a group order or booking, and a merge queue. Teams deliver worse group outcomes than the single coordinator in every environment; without a communication channel they collapse completely in two, and even with one the overhead is large, with the coordinator fulfilling a targeted request about twice as often in the personal-assistant setting. Observed behaviors include stalling as teams grow, overriding each other's actions, and fabricating claims; mitigations such as a team lead help but are environment-specific. Why: When agents represent different principals over a shared resource, adding a channel is not enough; the graph needs an explicit arbiter or protocol.	Research preprint Roles
📄 Reward Hacking as a Transmissible Strategy in Multi-Agent Coding Systems Paper · Failure propagation	arXiv Yuepeng Hu, Pegah Ghahremani, Reza Aghajani, Mahesh Pasupuleti, Himaghna Bhattacharjee · 2026	Shows that a reward-hacking strategy elicited by RL in one coding agent spreads to others: through code and shared memory, across roles through natural-language plans, and even from small open models into stronger closed ones. On hard tasks it can intensify along a chain after the original source leaves, as the code acquires more persuasive justifications. It also biases task routing toward reward-hacking agents and survives candidate aggregation; review by the coder or by a separate verifier is evaluated as a mitigation. Why: Treats an evaluator exploit as contagious across a team, which makes where verifiers sit a topology decision rather than a per-agent one.	Research preprint Reliability
📄 When Upstream Messages Override Correct Answers: A Controlled Study of Multi-Agent LLM Collaboration Paper · Error cascades	arXiv Yaxin Gong, Gangyi Zhang, Chongming Gao, Leyang Shen, Chenxiao Fan, Jiakai Wang, et al. · 2026	Holds the downstream task and evidence fixed and compares three conditions, no message, the upstream agent's original message, and a message with the opposite conclusion, across five benchmarks and five receivers. Messages often help when the receiver would otherwise be wrong, but when it would have been right, an incorrect upstream message changes its answer in up to 32% of cases, and in 94% of audited harmful cases the receiver copies the upstream's specific wrong answer. Dropping unreliable messages recovers part of the lost accuracy. Why: Measures what an edge costs separately from what it delivers, and argues for passing messages selectively by sender reliability.	Research preprint Handoffs

🗃️ Open data

The directory is generated from one canonical file:

• data/resources.jsonl — newline-delimited records for streaming and diffs.
• data/resources.csv — spreadsheet- and analysis-friendly export.
• data/resource.schema.json — JSON Schema for validating the 13-field record contract.
• data/README.md — field definitions, controlled vocabularies, and loading examples.
• GitHub Releases — versioned CSV, JSONL, and bundled snapshots with SHA-256 checksums.
• Hugging Face Dataset — synchronized Dataset Viewer mirror and direct datasets loading.

import pandas as pd

resources = pd.read_csv(
    "https://raw.githubusercontent.com/ChaoYue0307/awesome-graph-engineering/main/data/resources.csv"
)
print(resources.groupby("layer").size().sort_values(ascending=False))

python3 scripts/sync.py regenerates the CSV, README resource tables, and website atlas. bash scripts/check.sh runs the full local validation suite without changing generated files. bash scripts/publish_huggingface.sh validates and publishes the JSONL, CSV, JSON Schema, and dataset card in one Hub commit.

🤝 Contributing

Choose the shortest route for the change you want to make:

• ➕ Suggest a resource
• 🧭 Report a correction or scope issue
• 🌍 Offer a translation or language review
• 🖥️ Report a website or accessibility issue
• 🔀 Open a pull request

A resource contribution should identify the engineering decision it informs. Before opening a pull request:

1. Read CONTRIBUTING.md and search the existing JSONL for duplicates.
2. Add one canonical record with a concrete summary, why, evidence label, and primary layer.
3. Run python3 scripts/sync.py, then bash scripts/check.sh.
4. Keep vendor claims descriptive; do not present source type as proof of effectiveness.

Use GitHub Discussions for ideas, corrections, and scope questions. Report security issues privately as described in SECURITY.md. To introduce the project in another community or language, use the ready-to-adapt copy in 📣 Launch and sharing kit.

🧾 Citation

Curated by He Chaoyue.

@misc{he2026awesomegraphengineering,
  author       = {He, Chaoyue},
  title        = {Awesome Graph Engineering: A Field Guide, Dataset, and Interactive Atlas for Programmable AI-Agent Organizations},
  year         = {2026},
  doi          = {10.5281/zenodo.23617107},
  version      = {1.4.2},
  publisher    = {Zenodo},
  howpublished = {\url{https://github.com/ChaoYue0307/awesome-graph-engineering}},
  url          = {https://github.com/ChaoYue0307/awesome-graph-engineering}
}

In APA style:

He, C. (2026). Awesome Graph Engineering: A Field Guide, Dataset, and Interactive Atlas for Programmable AI-Agent Organizations (Version 1.4.2) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.23617107

Please keep the title as written. Citation indexes such as Google Scholar group references by title, so the same title keeps citations to every version together. Machine-readable citation metadata is available in CITATION.cff.

📜 License

Except where otherwise noted, the original metadata, schema, summaries, documentation, code, and visual assets in this repository are dedicated to the public domain under CC0 1.0 Universal. They may be copied, modified, distributed, and used for any purpose, including commercial use, without permission, to the extent permitted by law.

Linked papers, documentation, software, names, logos, and other third-party materials are not covered by this dedication and remain subject to their respective rights and licenses. CC0 does not waive trademark or patent rights and provides the work without warranties. Citation is appreciated for scholarly traceability but is not required by CC0.

One agent can execute a loop. An engineered graph can coordinate an organization.